CN101848393B - Telescopic video sparse information processing system - Google Patents
Telescopic video sparse information processing system Download PDFInfo
- Publication number
- CN101848393B CN101848393B CN 201010194922 CN201010194922A CN101848393B CN 101848393 B CN101848393 B CN 101848393B CN 201010194922 CN201010194922 CN 201010194922 CN 201010194922 A CN201010194922 A CN 201010194922A CN 101848393 B CN101848393 B CN 101848393B
- Authority
- CN
- China
- Prior art keywords
- module
- submodule
- time domain
- links
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention provides a telescopic video sparse information processing system, belonging to the technical field of information processing. The system comprises a time domain resolving module, a double multi-resolution ratio geometry resolving module, an entropy coding module, a bit stream multiplexer module and a rate distortion optimization controlling module, wherein the time domain resolving module is connected with the double multi-resolution ratio geometry resolving module to transmit time domain high-frequency video frame information and time domain low-frequency video frame information; the double multi-resolution ratio geometry resolving module is connected with the entropy coding module to transmit double multi-resolution ratio geometry-transformed resolution ratio information; the entropy coding module is connected with the bit stream multiplexer module to transmit configured multi-layer bit stream information; the rate distortion optimization controlling module is connected with the time domain resolving module to transmit motion evaluating control information; and the rate distortion optimization controlling module is connected with the entropy coding module to transmit coding control information. The system has the advantages of high compatibility, large compression ratio, high coding efficiency and small code word overhead, can obtain better visual effect, and has wide applicability.
Description
Technical field
What the present invention relates to is a kind of system of technical field of information processing, specifically is a kind of telescopic video sparse information processing system.
Background technology
In Internet video information processing and transmission, because the isomerism of the network architecture and terminal, networks of different type has the different characteristics of channel, and these factors make different users or same user be not quite similar in the bandwidth that difference obtains constantly.In order to make video bit stream can adapt to different network channel environment, people have proposed SVC (telescopic video information processing) technology, promptly " once the coding; as required the intercepting " the embedded video information technology, decoder offers terminal by the required code stream with different spatial and temporal resolution/code checks of user.
In the WSVC system based on 3-D Wavelet (3 D wavelet) scheme in the prior art, frame of video is after having passed through the Time Domain Decomposition module, time domain high-frequency frame that obtains and time domain low-frequency frame are input in the two-dimensional wavelet transformation module of rear end decomposes, the coefficient that obtains be input to again carry out in the sub-band coding module embedded encoded.Wavelet transformation is because its distinctive multi-resolution signals space expression mode and based on the quick decomposition and the perfect reconstruction method of boosting algorithm, the important tool of space stage when enabling to become natively in the telescopic video information processing; In addition, wavelet transformation coefficient later also can use as EBCOT (the embedded block encoding that optimization is blocked) or ESCOT embedded encoded technology such as (the embedded sub-band codings that optimization is blocked), obtains meticulous grading effect.
The document in many signal analysis field pointed out that wavelet transformation can not provide the signal indication method of " sparse " on general 2D signal but in the last few years.Obtain because two-dimentional separable wavelets base is the tensor product of the one dimension wavelet basis by level and vertical direction, the best that therefore can only reach " point is unusual " family of functions is approached.And in natural image and video, have a large amount of " line is unusual " functions, directly adopt two-dimentional separable wavelets conversion meeting to produce the coefficient of can not ignore in a large number.Therefore if under the situation of low code check, realize higher coding and decoding video efficient, must adopt inseparable two dimensional basis functions to come signal is carried out rarefaction representation.In addition, physiology points out that also the corticocerebral received field of vision has local, orientation and bandpass characteristics, and therefore the basic function of new two-dimensional transform also should satisfy these characteristics.
Through existing literature search is found, the Contourlet ( profile wave convert ) that M.N.Do and M.Vetterli propose in the paper of the 12nd phase in 2005 " IEEE Transactionson Image Processing " ( " the Institute of Electrical and Electric Engineers image is processed transactions " ) " The contourlettransform:an efficient directional multiresolution image representation ( profile wave convert: a kind of effective directional multi-resolution rate image expression mode ) " is that the inseparable anisotropic filter group of first discrete two-dimensional is in picture breakdown with the successful Application in representing. It has used double filter group structure; At first with the laplacian-pyramid filter group with signal decomposition in multiple dimensioned subspace; the singular point that uses DFB ( directional filter group ) will be distributed on equidirectional then in the logical subspace of band is merged into a coefficient, and final effect is exactly that the basic function of Contourlet has " strip " structure that length-width ratio changes with yardstick.BambergerSmith19924《IEEE Transactions on Signal Processing ( ) 》“Afilter bank for the directional decomposition of images：theory and design” ( “：” ) ； Do and Vetterli have simplified the method for designing of anisotropic filter group, have adopted based on the method for Quincunx Sampling (five strains sampling) and have come the design direction bank of filters. But the shortcoming of such technology is:because the redundancy that the laplacian-pyramid filter group has makes Contourlet be not suitable for being directly used in image and video are carried out the information compressed processing procedure.
Find by retrieval again, Eslami and Radha have proposed the Contourlet conversion (Wavelet-basedContourlet Transform is designated hereinafter simply as WBCT) based on multiple dimensioned nonredundancy small echo in the paper " A New Family of NonredundantTransforms Using Hybrid Wavelets and Directional Filter Banks (based on gang's nonredundancy conversion of hybrid wavelet and directional filter group) " of 2007 the 4th phases " IEEE Transactions on ImageProcessing (Institute of Electrical and Electric Engineers image processing transactions) ".The difference of WBCT and Contourlet is to have used wavelet transformation that signal decomposition is arrived multiple dimensioned subspace, has therefore eliminated redundancy, is adapted at image and video are carried out using in the information compressed processing procedure.Yet the defective that these systems exist is that all used the directional filter group of even decomposition, this bank of filters has been used the full binary tree structure.Directional characteristic in the natural image frequency spectrum obviously can not be equally distributed, and direction decomposition uniformly may cause the energy leakage of the comparatively intensive direction of some spectrum distribution to go to adjacent sub-bands, and it obviously is not the optimal image is olation that therefore this uniform direction is decomposed.In addition, because edge of image all is positioned at different metric spaces with texture, and along with the speed of the variation of its curvature, its directional resolution also changes taking place, and in existing multiple dimensioned how much decomposition methods, directional resolution often depends on yardstick resolution.
Summary of the invention
The objective of the invention is to overcome above shortcomings in the prior art, a kind of telescopic video sparse information processing system is provided.The present invention proposes a kind of how much is olations of two multiresolutions at image and video, under the condition of equal code check, signal energy can be accumulated in the low frequency sub-band in the concentrated area more, thereby realize sparse expression to signal, and finally show as bigger compression ratio, and the quality height of the receiving terminal frame of video recovering out.
The present invention is achieved by the following technical solutions:
The present invention includes: the Time Domain Decomposition module, how much decomposing module of two multiresolutions, the entropy coding module, bit stream multiplexer module and rate-distortion optimization control module, wherein: Time Domain Decomposition module and how much decomposing module of two multiresolutions link to each other and transmit time domain high frequency and time domain low frequency frame of video information, how much decomposing module of two multiresolutions decomposition coefficient information after the transmission pair multiresolution geometric transformations that links to each other with the entropy coding module, link to each other with the bit stream multiplexer module multi-layer bitstream information of transmission configuration of entropy coding module, the rate-distortion optimization control module transmitting moving that links to each other with the Time Domain Decomposition module is estimated control information, the rate-distortion optimization control module transfer encoding control information that links to each other with the entropy coding module.
Described Time Domain Decomposition module is carried out Time Domain Decomposition based on motion compensation to frame of video, comprise: the estimation submodule, motion compensated temporal filter submodule and pattern and motion vector encoder submodule, wherein: estimation submodule input original video frame, the estimation submodule links to each other transmission based on the motion vector information of macro block with the motion compensated temporal filter submodule, the estimation submodule links to each other transmission based on the motion vector information of macro block with pattern and motion vector encoder submodule, the optimum-size information that the motion compensated temporal filter submodule links to each other and transmits macro block with pattern and motion vector encoder submodule, the motion compensated temporal filter submodule transmitting moving that links to each other with the rate-distortion optimization control module is estimated control information, and how much decomposing module of motion compensated temporal filter submodule and two multiresolutions link to each other and transmit time domain high frequency and time domain low frequency frame of video information.
How much decomposing module of described pair of multiresolution are carried out multiple dimensioned non-homogeneous direction and are decomposed, comprise: two-dimentional overcomplete wavelet decomposes submodule, the phase equalization detection sub-module, non-homogeneous directional filter group design submodule, spatial decomposition submodule and resolution model coding submodule, wherein: two-dimentional overcomplete wavelet decomposition submodule links to each other with the Time Domain Decomposition module and transmits time domain high frequency and time domain low frequency frame of video information, two dimension overcomplete wavelet transformation submodule links to each other with the phase equalization detection sub-module and transmits two-dimentional overcomplete wavelet decomposition coefficient, the phase equalization detection sub-module links to each other with non-homogeneous directional filter group design submodule and transmits the phase equalization coefficient, the directional filter group information that non-homogeneous directional filter group design submodule links to each other and transmits non-homogeneous binary tree structure with spatial decomposition submodule and resolution model coding submodule respectively, the spatial decomposition submodule propagation and transformation non-homogeneous direction decomposition coefficient information later that links to each other with the entropy coding module, the spatial decomposition submodule links to each other with the Time Domain Decomposition module and transmits time domain high frequency and time domain low frequency frame of video information.
Described two-dimentional overcomplete wavelet decompose submodule with frame of video be decomposed into four onesize but comprise the subband of different frequency bands information, comprise: one dimension line direction resolving cell and one dimension column direction resolving cell, wherein: one dimension line direction resolving cell links to each other with the Time Domain Decomposition module and transmits time domain low frequency and time domain high frequency frame of video information, one dimension line direction resolving cell links to each other with one dimension column direction resolving cell and transmits L (low pass) subband and H (high pass) subband, and one dimension column direction resolving cell links to each other with the phase equalization detection sub-module and transmits LL (low) subband, LH (low high) subband, HL (just) subband and HH (high) subband.
Described spatial decomposition submodule is decomposed into direction subband heterogeneous under the different scale with frame of video, comprise: 2-d wavelet resolving cell and convolutional filtering unit, wherein: the 2-d wavelet resolving cell links to each other with the Time Domain Decomposition unit and transmits time domain low frequency and time domain high frequency frame of video information, the 2-d wavelet resolving cell multiple dimensioned sub-band information of transmission 2-d wavelet after decomposing that link to each other with the convolutional filtering unit, the convolutional filtering unit links to each other with the entropy coding module and transmits the non-direction decomposition coefficient information of converging.
The difference of described two-dimentional overcomplete wavelet resolving cell and 2-d wavelet resolving cell is: the former comprises the sub sampling operation, and the latter does not comprise the sub sampling operation.
Described entropy coding module is handled the sub-band coefficients that obtains with how much decomposing module of two multiresolutions and is carried out the binary system compressed encoding, comprise: Bit-Plane Encoding submodule and arithmetic coding submodule, wherein: how much decomposing module of Bit-Plane Encoding submodule and two multiresolutions link to each other to transmit and divide even direction decomposition coefficient information, the Bit-Plane Encoding submodule links to each other with the arithmetic coding submodule and transmits bit plane numerical value and corresponding context flag information, the arithmetic coding submodule links to each other transmission through the binary code stream after the lossy compression method, the code check working point parameter that the Bit-Plane Encoding submodule links to each other with the rate-distortion optimization control module and transmits entropy coding with the bit stream multiplexer.
Described rate-distortion optimization control module comprises: estimation parameter control submodule and coding work point parameter control submodule, wherein: the estimation parameter control submodule transmitting moving that links to each other with the Time Domain Decomposition module is estimated control information, and coding work point parameter is controlled the code check working point parameter that submodule links to each other with the entropy coding module and transmits entropy coding.
Described bit stream multiplexer module will be under the parameter condition of different code checks working point the telescopic video code stream that generates of encoding process simply merge.
Compared with prior art, the invention has the beneficial effects as follows: at the directional characteristic of 2-d spectrum in the frame of video, can select for use more any multiple dimensioned and multi-direction is olation that image is carried out adaptive conversion, finally be adapted to the natural image of " line is unusual " and the more sparse sub-band coefficients of vision signal, helped the compression and the transmission of telescopic video information; The present invention can seamlessly combine with WSVC (based on the telescopic video information processing of wavelet transformation) system, and use at SVC under the situation of same code check working point, compare with the result who obtains with WSVC merely, since at the coefficient of how much decomposing module rear ends outputs of two multiresolutions than common wavelet decomposition more sparse (quantity that is the nonzero coefficient that obtains than common wavelet decomposition of the quantity of nonzero coefficient is lacked a lot), therefore the telescopic video code stream that can obtain to have bigger compression ratio; On visual quality, with existing WSVC systematic comparison, system of the present invention can obtain the lifting on the objective effect of 1.2dB at most, and higher structural similarity, finally obtains better visual effect.
Description of drawings
Fig. 1 is that system of the present invention forms connection diagram.
Fig. 2 is the schematic diagram that two-dimentional overcomplete wavelet decomposes a direction decomposition of submodule.
Fig. 3 is the schematic diagram that a direction of 2-d wavelet resolving cell is decomposed.
Fig. 4 is the HH subband schematic diagram of test pattern Foreman.
Fig. 5 is each the interval interior phase equalization distribution histogram after Fig. 4 is decomposed into 32 intervals.
Fig. 6 is the approximate uniform phase equalization distribution histogram that Fig. 5 obtains after merging into 8 subbands.
Fig. 7 is the non-homogeneous binary tree structure schematic diagram of Fig. 6 correspondence.
Fig. 8 is that the embodiment bank of filters realizes schematic diagram;
Wherein: figure (a) is that directional resolution is 1 bank of filters realization schematic diagram; Figure (b) is that directional resolution is 2 bank of filters realization schematic diagram.
Fig. 9 is the adaptive direction filter bank structure schematic diagram of embodiment.
Figure 10 is that embodiment decomposes the subband distribution map that obtains.
Figure 11 is the schematic diagram of the four directions of embodiment to even decomposition.
Embodiment
Below in conjunction with accompanying drawing system of the present invention is further described: present embodiment is being to implement under the prerequisite with the technical solution of the present invention, provided detailed execution mode and concrete operating process, but protection scope of the present invention is not limited to following embodiment.
Embodiment
As shown in Figure 1, present embodiment comprises: the Time Domain Decomposition module, how much decomposing module of two multiresolutions, the entropy coding module, bit stream multiplexer module and rate-distortion optimization control module, wherein: Time Domain Decomposition module and how much decomposing module of two multiresolutions link to each other and transmit time domain high frequency and time domain low frequency frame of video information, how much decomposing module of two multiresolutions decomposition coefficient information after the transmission pair multiresolution geometric transformations that links to each other with the entropy coding module, link to each other with the bit stream multiplexer module multi-layer bitstream information of transmission configuration of entropy coding module, the rate-distortion optimization control module transmitting moving that links to each other with the Time Domain Decomposition module is estimated control information, the rate-distortion optimization control module transfer encoding control information that links to each other with the entropy coding module.
Described Time Domain Decomposition module is carried out Time Domain Decomposition based on motion compensation to frame of video, comprise: the estimation submodule, motion compensated temporal filter submodule and pattern and motion vector encoder submodule, wherein: estimation submodule input original video frame, the estimation submodule links to each other transmission based on the motion vector information of macro block with the motion compensated temporal filter submodule, the estimation submodule links to each other transmission based on the motion vector information of macro block with pattern and motion vector encoder submodule, the optimum-size information that the motion compensated temporal filter submodule links to each other and transmits macro block with pattern and motion vector encoder submodule, the motion compensated temporal filter submodule transmitting moving that links to each other with the rate-distortion optimization control module is estimated control information, and how much decomposing module of motion compensated temporal filter submodule and two multiresolutions link to each other and transmit time domain high frequency and time domain low frequency frame of video information.
Described estimation submodule is used for seeking optimum motion vector in the macro block of the identical size of adjacent front and back two interframe.
Described motion compensated temporal filter submodule is used for original video frame is carried out one dimension time domain wavelet decomposition after the motion compensation, and original video frame is carried out temporal scalability.
Described pattern and motion vector encoder submodule are used for the macro block search pattern is carried out UVLC (universal variable is long) coding, and the motion vector unification is converted into 4 * 4 sports ground.
How much decomposing module of described pair of multiresolution are carried out multiple dimensioned non-homogeneous direction and are decomposed, comprise: two-dimentional overcomplete wavelet decomposes submodule, the phase equalization detection sub-module, non-homogeneous directional filter group design submodule, spatial decomposition submodule and resolution model coding submodule, wherein: two-dimentional overcomplete wavelet decomposition submodule links to each other with the Time Domain Decomposition module and transmits time domain high frequency and time domain low frequency frame of video information, two dimension overcomplete wavelet transformation submodule links to each other with the phase equalization detection sub-module and transmits two-dimentional overcomplete wavelet decomposition coefficient, the phase equalization detection sub-module links to each other with non-homogeneous directional filter group design submodule and transmits the phase equalization coefficient, the directional filter group information that non-homogeneous directional filter group design submodule links to each other and transmits non-homogeneous binary tree structure with spatial decomposition submodule and resolution model coding submodule respectively, the spatial decomposition submodule propagation and transformation non-homogeneous direction decomposition coefficient information later that links to each other with the entropy coding module, the spatial decomposition submodule links to each other with the Time Domain Decomposition module and transmits time domain high frequency and time domain low frequency frame of video information.
Described two-dimentional overcomplete wavelet decompose submodule with frame of video be decomposed into four onesize but comprise the subband of different frequency bands information, comprise: one dimension line direction resolving cell and one dimension column direction resolving cell, wherein: one dimension line direction resolving cell links to each other with the Time Domain Decomposition module and transmits time domain low frequency and time domain high frequency frame of video information, one dimension line direction resolving cell links to each other with one dimension column direction resolving cell and transmits L subband and H subband, and one dimension column direction resolving cell links to each other with the phase equalization detection sub-module and transmits LL subband, LH subband, HL subband and HH subband.
Described one dimension line direction resolving cell carries out two passage wavelet filter group filtering with a frame video signal on line direction, obtain L subband and H subband.
Described one dimension column direction resolving cell carries out two passage wavelet filter group filtering with a frame video signal on column direction, obtain L subband and H subband.
High-frequency sub-band after described phase equalization detection sub-module is decomposed two-dimentional overcomplete wavelet is carried out the phase equalization distribution statistics based on the Boundary Detection of two-dimensional Gabor small echo.
Described non-homogeneous directional filter group design submodule generates the non-homogeneous directional filter group of the phase equalization distribution that is adapted to each small echo high-frequency sub-band.
Described spatial decomposition submodule is decomposed into direction subband heterogeneous under the different scale with frame of video, comprise: 2-d wavelet resolving cell and convolutional filtering unit, wherein: the 2-d wavelet resolving cell links to each other with the Time Domain Decomposition unit and transmits time domain low frequency and time domain high frequency frame of video information, the 2-d wavelet resolving cell multiple dimensioned sub-band information of transmission 2-d wavelet after decomposing that link to each other with the convolutional filtering unit, the convolutional filtering unit links to each other with the entropy coding module and transmits the non-direction decomposition coefficient information of converging.
Described convolutional filtering unit is used for high frequency wavelet sub-band coefficients (being the coefficient of LH, HL, HH subband) and non-homogeneous directional filter are carried out convolution, obtains having non-homogeneous directional characteristic sparse sub-band coefficients.
The difference of described two-dimentional overcomplete wavelet resolving cell and 2-d wavelet resolving cell is: the former comprises the sub sampling operation, and the latter does not comprise the sub sampling operation.
Described entropy coding module is handled the sub-band coefficients that obtains with how much decomposing module of two multiresolutions and is carried out the binary system compressed encoding, comprise: Bit-Plane Encoding submodule and arithmetic coding submodule, wherein: how much decomposing module of Bit-Plane Encoding submodule and two multiresolutions link to each other to transmit and divide even direction decomposition coefficient information, the Bit-Plane Encoding submodule links to each other with the arithmetic coding submodule and transmits bit plane numerical value and corresponding context flag information, the arithmetic coding submodule links to each other transmission through the binary code stream after the lossy compression method, the code check working point parameter that the Bit-Plane Encoding submodule links to each other with the rate-distortion optimization control module and transmits entropy coding with the bit stream multiplexer.
Described Bit-Plane Encoding submodule is used for being divided into the sub-band coefficients after the process quantification independently on length and width and time shaft cube encoding block (64 * 64 * 4) carries out absolute coding and carries out laminated tissue according to the rate distortion characteristic of code check working point.
Described arithmetic coding submodule is used for bitplanes numerical value and corresponding context sign carries out arithmetic coding.
Described rate-distortion optimization control module comprises: estimation parameter control submodule and coding work point parameter control submodule, wherein: the estimation parameter control submodule transmitting moving that links to each other with the Time Domain Decomposition module is estimated control information, and coding work point parameter is controlled the code check working point parameter that submodule links to each other with the entropy coding module and transmits entropy coding.
Described bit stream multiplexer module will be under the parameter condition of different code checks working point the telescopic video code stream that generates of encoding process simply merge.
Introduce the concrete course of work of each module in the present embodiment below in detail:
The concrete course of work of estimation submodule is in the present embodiment: each frame of video in the original video stream of input system is input to according to the order of Y (brightness), U (colourity) and V (concentration) in the estimation submodule of Time Domain Decomposition module carries out estimation.Estimation is based on that macro block carries out, and the different sizes along with the original video frame of input are provided with different initial macroblock sizes: for CIF (general image transformat, size are 352 * 288), using size is 32 * 32 macro block; For QCIF (1/4th big or small general image transformats, size is 176 * 144), the use size is 16 * 16 macro block; For 4CIF (four times of general image transformats of size, size is 704 * 576), the use size is 64 * 64 macro block.When specifically carrying out estimation, in each initial macroblock, can among four kinds of mode of operations such as the whole pixel search of 16 * 16 big or small macro blocks, the half pixel searching of 16 * 8 big or small macro blocks, the half pixel searching of 8 * 16 big or small macro blocks and 1/4th pixel search of 8 * 8 big or small macro blocks, determine the motion vector and the optimum search work pattern of optimal estimation.
The motion vector of optimal estimation can obtain by minimizing following Lagrangian rate distortion (R-D) cost function:
Wherein: distortion function is
Mv represents motion vector, the required bit number of R (mv) representative transmission mv, S
CurAnd S
RefRepresent the current macro block that carries out estimation respectively and calculate with reference to required macro block as estimation.
Optimum search work pattern can be obtained by minimizing following Lagrangian rate distortion (R-D) cost function:
Wherein: the concrete search work pattern of using of mode representative, the required bit number of R (mode) representative transmission mode.
The concrete course of work of motion compensated temporal filter submodule is in the present embodiment: after the estimation submodule had obtained motion vector information, the motion compensated temporal filter submodule carried out based on the time domain wavelet decomposition of boosting algorithm to realize the temporal scalability of telescopic video original video frame.Time domain wavelet-decomposing method based on boosting algorithm comprises division, prediction and upgrades three phases.In the division stage, original video frame is broken down into odd-numbered frame { F
2i+1And even frame { F
2iTwo groups of disjoint set; At forecast period, the prediction that odd-numbered frame deducts the neighbouring even-numbered frame obtains a time domain high-frequency frame; In update stage, even frame adds the renewal of this time domain high-frequency frame, obtains a time domain low-frequency frame.So-called prediction and renewal process, can be understood as be exactly between adjacent odd-numbered frame and the even frame through the operation of the pixel mapping behind the compensation motion vector.In the present embodiment, use 5/3 small echo to carry out operating based on the time domain wavelet decomposition of boosting algorithm, concrete decomposable process is as follows:
1) division: even frame A
i=F
2i, odd-numbered frame B
i=F
2i+1
2) forecast period: high-frequency frame
3) update stage: low-frequency frame
Wherein: mv
A, bRepresentative is by F
aSome macro blocks in the frame point to F
bThe motion vector of corresponding macro block in the frame.
The concrete course of work of two-dimentional overcomplete wavelet decomposition submodule is in the present embodiment: two-dimentional overcomplete wavelet decomposes submodule at first to be handled the time domain high and low frequency frame of exporting in the Time Domain Decomposition module.Decomposition of two dimension overcomplete wavelet and building-up process are as follows:
Wherein:
[n] and
[n] represents respectively and decomposes and the required low pass filter of synthetic end,
[n] and
[n] then represents corresponding high pass filter, and symbol * represents convolution.In order to satisfy perfect reconstruction condition, high pass filter
[n] and
[n] needs to satisfy
Present embodiment is chosen biorthogonal " 9-7 " bank of filters, promptly uses the bank of filters of coefficient as shown in table 1.
Table 1
Fig. 2 has provided the schematic diagram that two-dimentional overcomplete wavelet decomposes a direction decomposition of submodule: in filtering, at first on line direction, the frame of video of input is carried out low pass and high-pass filtering, obtain L subband and H subband, then in the low pass and the high-pass filtering of L subband and the enterprising ranks direction of H subband, with picture breakdown in LL, LH, four subbands of HL, HH; Wherein: the LL subband has comprised topmost low frequency energy in the image, and the LH subband has comprised the high frequency coefficient of basic vertical phase place, and the HL subband has comprised the high frequency coefficient of basic horizontal phase place, and the HH subband has comprised basic high frequency coefficient to angular phasing.
The concrete course of work of phase equalization detection sub-module is in the present embodiment: two-dimentional overcomplete wavelet decomposes after submodule decomposes frame of video in LL, LH, HL and four subbands of HH, and the phase equalization detection sub-module is carried out phase equalization distribution statistics based on the Boundary Detection of two-dimensional Gabor small echo to LH, HL and three high-frequency sub-band of HH.Present embodiment selects for use the method for PeterKovesi to calculate phase equalization, that is:
Wherein: W
d(x) expression weight factor, many more these weight factors of frequency that embody phase equalization are big more, A
Dn(x) represent the mould of the component on the direction d of two-dimensional Gabor Wavelet Component, the phase sensitive function
Be used for a phase place at x place is positioned T
DnThe expression weight is used for eliminating the influence that noise brings court verdict in formula, ∈ is a small constant, and being used for avoiding denominator is zero.
The concrete course of work of non-homogeneous directional filter group design submodule is in the present embodiment: non-homogeneous directional filter group design submodule is after the phase equalization coefficient that has obtained the output of phase equalization detection sub-module, according to the inceptive direction resolution r=2 that sets
-nGo up required final number of sub-bands l with yardstick s
s(l
s≤ 2
n, s=1,2 ...), with initial 2
nPhase place PC (x) distributes and merges between two the most sparse adjacent regions in the individual direction interval, whenever once merge, two leaf nodes of its correspondence just merge disappearance, father node under these two leaf nodes has become new leaf node, and the directional resolution in new interval is exactly original twice, until final l
sPhase equalization in the individual interval distributes near till even.This moment, each direction interval was one by one corresponding to a leaf node in the binary tree structure.For this problem clearly is described, Fig. 4～Fig. 7 has showed the HH subband of test pattern Foreman respectively, with phase place [pi/2, pi/2] resolve into each interval interior phase equalization distribution histogram of 32 interval backs, through merging into approximate uniform phase equalization distribution histogram and the corresponding non-homogeneous binary tree structure that obtains behind 8 subbands.At each yardstick s=1,2 ... in, according to its i direction subband (0＜i≤l
s) directional resolution r
i(T
iThe i.e. inverse of the power of this node degree of depth of 2 time), use five strains sampling fan-filter group or parallelogram bank of filters, obtain being adapted to the directional filter group of the non-homogeneous binary tree structure of LH, HL and HH subband in the yardstick s of this frame of video, and binary tree imported into to the resolution model coding submodule of rear end encode.When the degree of depth of node is 1 or 2, so only need to adopt five strains sampling fan-filter group to get final product, Fig. 8 (a) and Fig. 8 (b) have provided the fan-filter group realization figure when the node degree of depth is 1 and 2 respectively, and this fan-filter group adopts respectively
With
Five strain sampling matrixs; When the degree of depth of node more than or equal to 3 the time, so then on the basis of adopting five strains sampling fan-filter, adopt the parallelogram bank of filters that the passband of bank of filters is pushed toward respective direction (positive direction of x axle and y axle or negative direction) again, wherein sampling matrix is
The parallelogram bank of filters with image project under the constant situation of ordinate with x axle positive direction angle be that sampling matrix is on the directions of 45 degree
The parallelogram bank of filters with image project under the constant situation of ordinate with x axle negative direction angle be that sampling matrix is on the directions of 45 degree
The parallelogram bank of filters with image project under the constant situation of abscissa with y axle positive direction angle be that sampling matrix is on the directions of 45 degree
The parallelogram bank of filters with image project under the constant situation of abscissa with y axle negative direction angle be on the directions of 45 degree.In the present embodiment, Fig. 9 has provided to use based on what test video sequence Foreman obtained and has been adapted to the directional filter group realization figure of the non-homogeneous binary tree structure of a certain frame wherein, comprising the degree of depth be the node of 1,2,3 non-homogeneous binary tree structure bank of filters.
The concrete course of work of space decomposition submodule is in the present embodiment: the spatial decomposition submodule is after the directional filter group information of the non-homogeneous binary tree structure that has obtained the output of non-homogeneous directional filter group design submodule, at first this frame of video is carried out the two-dimentional common wavelet transformation of k yardstick altogether, the basic principle of common wavelet transformation of two dimension and two-dimentional overcomplete wavelet conversion is identical with method, and difference is that the filtering of two-dimentional common small echo exists the step of sub sampling and zero padding to eliminate spatial scalability redundant and the realization telescopic video.Present embodiment is chosen biorthogonal " 9-7 " bank of filters, and Fig. 3 is the schematic diagram that a direction of two-dimensional wavelet transformation unit is decomposed.The spatial decomposition submodule is behind the common wavelet transformation that frame of video has been passed through k yardstick, obtain 3k+1 wavelet sub-band, in 3k high-frequency sub-band, use non-homogeneous directional filter group to carry out further convolutional filtering then, finally obtain together together with low frequency sub-band
Individual subband.These subbands have mutually independently yardstick resolution and directional resolution.Figure 10 has provided the distribution map that a certain frame among the test video sequence Foreman decomposes the subband that obtains.
The concrete course of work of entropy coding module is in the present embodiment: after the multiple dimensioned non-homogeneous direction sub-band coefficients that has obtained the output of spatial decomposition submodule, use 3-D ESCOT (the three-dimensional embedded sub-band coding that optimization is blocked) coding module as the entropy coding module sub-band coefficients to be encoded.This module specifically can be divided into two submodules again: Bit-Plane Encoding submodule and arithmetic coding submodule.The Bit-Plane Encoding submodule at first will be divided into independently on length and width and time shaft through the sub-band coefficients after quantizing, and cube encoding block (64 * 64 * 4) carries out absolute coding, numerical value D on the carry-out bit plane and corresponding contextual tagging CX, they are admitted to the arithmetic coding that code check parameter that the arithmetic coding submodule transmits according to the coding work point parameter submodule in reasonably information source probabilistic model and the rate-distortion optimization control module is carried out lossy compression method again.Used four kinds of coding primitive (being Zero Coding (Zero-code), Sign Coding (symbolic coding), Magnitude Refinement Coding (value improves coding) and Run-Length Coding (Run-Length Coding)) to come to generate contextual tagging CX in the whole Bit-Plane Encoding process as bit plane numerical value D.In the present embodiment, for the directivity of outstanding subband, Zero Coding is partly revised.Because each encoding block has all used direction resolution model that may be different adaptively, the present invention is directed to each encoding block and is provided with new Zero Coding context model table.Figure 11 is the schematic diagram of the four directions of embodiment to even decomposition, and table 2 has provided the four directions that is adapted to describe among Figure 11 to the Zero of even resolution model Coding context model table.
The concrete course of work of bit stream multiplexer module is in the present embodiment: behind the telescopic video code stream that has obtained under the different code checks working point parameter condition of required configuration, bit stream multichannel (multiplexing) device merges them simply becomes a total code stream.
How much decomposition techniques of two multiresolutions of the image that present embodiment proposes can be controlled yardstick resolution and directional resolution simultaneously independently, the directional filter group of choosing the different topology structure at different input video frame carries out sub-band division and sends handling to image adaptively, finally obtains more sparse sub-band coefficients.
Table 2
Implementation result
According to above step, adopt test video sequence harbour_4cif_60.yuv (size is 4CIF, i.e. 704 * 576 4:2:0 form, frame per second is the YUV file of 60 frame per seconds (60Hz)), 90 frames of encoding altogether.Present embodiment has added how much decomposing module of two multiresolutions in the WSVC of Microsoft Research, Asia (based on the telescopic video information processing of wavelet transformation) reference software Vidwav, and replaced original wavelet decomposition module, this test video is sent and receives processing.It is as follows that parameter specifically is set:
The temporal scalability rank is set to 5 grades (supporting 60Hz, 30Hz, 15Hz, 7.5Hz, the frame per second of 3.75Hz), and GOP (groups of pictures) size is set to 32; The gradable spatial rank be set to 3 grades (support 4CIF, CIF, QCIF);
Use glug Lang Ri multiplier method to come the optimization motion vector, wherein lagrangian multiplier and hunting zone SR are set at: 4CIF: λ=128, SR=128, CIF: λ=32, SR=64, QCIF: λ=16, SR=32.
For 4CIF, in the range of code rates of 512kbps～3072kbps, set a working point every 256kbps.For CIF, in the range of code rates of 128kbps～768kbps, set a working point every 64kbps.For QCIF, in the range of code rates of 32kbps～192kbps, set a working point every 16kbps.
Input video is carried out 5 grades of wavelet decomposition, on 4 grades of the meticulousst small echo high-frequency sub-band, again it is decomposed into 1,2,4,8 direction subbands from the most coarse.
In the elimination ringing effect filter in the reprocessing end of decoder, the filter coefficient weight is set to 0.35, and attenuation parameter is set at 3.0.
PSNR (Y-PSNR) and SSIM (structural similarity) index have been used, to the final video sequence and original not comparing of describing based on multiple dimensioned direction-adaptive and obtaining based on the scalable video coding method of wavelet transformation through the video sequence that sends and receive processing.Wherein the PSNR index is usually used in weighing the objective quality of video after treatment, and high more its objective quality of numerical value is good more; The SSIM index is used to weigh the structural similarity of two width of cloth images, and it is worth between 0～1, and 1 expression is identical, and more near 1, the expression structural similarity is high more.
When the decoding working point is 4CIF﹠amp; During 512kbps, the PSNR that present embodiment system and WSVC system obtain is respectively: 25.33dB and 24.18dB; SSIM is respectively: 0.737 and 0.685;
When the decoding working point is 4CIF﹠amp; During 1024kbps, the PSNR that present embodiment system and WSVC system obtain is respectively: 28.97dB and 28.65dB; SSIM is respectively: 0.875 and 0.863;
When the decoding working point is 4CIF﹠amp; During 1536kbps, the PSNR that present embodiment system and WSVC system obtain is respectively: 30.62dB and 30.37dB; SSIM is respectively: 0.909 and 0.904;
When the decoding working point is 4CIF﹠amp; During 2048kbps, the PSNR that present embodiment system and WSVC system obtain is respectively: 31.70dB and 31.51dB; SSIM is respectively: 0.927 and 0.926;
When the decoding working point is CIF﹠amp; During 128kbps, the PSNR that present embodiment system and WSVC system obtain is respectively: 25.54dB and 24.58dB; SSIM is respectively: 0.796 and 0.715;
When the decoding working point is CIF﹠amp; During 256kbps, the PSNR that present embodiment system and WSVC system obtain is respectively: 28.22dB and 27.71dB; SSIM is respectively: 0.890 and 0.855;
When the decoding working point is CIF﹠amp; During 384kbps, the PSNR that present embodiment system and WSVC system obtain is respectively: 29.61dB and 29.22dB; SSIM is respectively: 0.919 and 0.898;
When the decoding working point is CIF﹠amp; During 512kbps, the PSNR that present embodiment system and WSVC system obtain is respectively: 30.57dB and 30.30dB; SSIM is respectively: 0.936 and 0.922.
By above-mentioned data as can be known: than existing WSVC system, the present embodiment system can improve the objective quality of video effectively, and structurally more near original video sequence.
Claims (5)
1. telescopic video sparse information processing system, it is characterized in that, comprise: the Time Domain Decomposition module, how much decomposing module of two multiresolutions, the entropy coding module, bit stream multiplexer module and rate-distortion optimization control module, wherein: Time Domain Decomposition module and how much decomposing module of two multiresolutions link to each other and transmit time domain high frequency and time domain low frequency frame of video information, how much decomposing module of two multiresolutions decomposition coefficient information after the transmission pair multiresolution geometric transformations that links to each other with the entropy coding module, link to each other with the bit stream multiplexer module multi-layer bitstream information of transmission configuration of entropy coding module, the rate-distortion optimization control module transmitting moving that links to each other with the Time Domain Decomposition module is estimated control information, the rate-distortion optimization control module transfer encoding control information that links to each other with the entropy coding module;
How much decomposing module of described pair of multiresolution comprise: two-dimentional overcomplete wavelet decomposes submodule, the phase equalization detection sub-module, non-homogeneous directional filter group design submodule, spatial decomposition submodule and resolution model coding submodule, wherein: two-dimentional overcomplete wavelet decomposition submodule links to each other with the Time Domain Decomposition module and transmits time domain high frequency and time domain low frequency frame of video information, two dimension overcomplete wavelet decomposition submodule links to each other with the phase equalization detection sub-module and transmits two-dimentional overcomplete wavelet decomposition coefficient, the phase equalization detection sub-module links to each other with non-homogeneous directional filter group design submodule and transmits the phase equalization coefficient, the directional filter group information that non-homogeneous directional filter group design submodule links to each other and transmits non-homogeneous binary tree structure with spatial decomposition submodule and resolution model coding submodule respectively, the spatial decomposition submodule propagation and transformation non-homogeneous direction decomposition coefficient information later that links to each other with the entropy coding module, the spatial decomposition submodule links to each other with the Time Domain Decomposition module and transmits time domain high frequency and time domain low frequency frame of video information;
High-frequency sub-band after described phase equalization detection sub-module is decomposed two-dimentional overcomplete wavelet is carried out the phase equalization distribution statistics based on the Boundary Detection of two-dimensional Gabor small echo;
Described non-homogeneous directional filter group design submodule generates the non-homogeneous directional filter group of the phase equalization distribution that is adapted to each small echo high-frequency sub-band;
Described spatial decomposition submodule comprises: 2-d wavelet resolving cell and convolutional filtering unit, wherein: the 2-d wavelet resolving cell links to each other with the Time Domain Decomposition module and transmits time domain low frequency and time domain high frequency frame of video information, the 2-d wavelet resolving cell multiple dimensioned sub-band information of transmission 2-d wavelet after decomposing that link to each other with the convolutional filtering unit, the convolutional filtering unit links to each other with the entropy coding module and transmits non-homogeneous direction decomposition coefficient information;
Described convolutional filtering unit is used for high frequency wavelet sub-band coefficients and non-homogeneous directional filter are carried out convolution, obtains having non-homogeneous directional characteristic sparse sub-band coefficients.
2. telescopic video sparse information processing system according to claim 1, it is characterized in that, described Time Domain Decomposition module comprises: the estimation submodule, motion compensated temporal filter submodule and pattern and motion vector encoder submodule, wherein: estimation submodule input original video frame, the estimation submodule links to each other transmission based on the motion vector information of macro block with the motion compensated temporal filter submodule, the estimation submodule links to each other transmission based on the motion vector information of macro block with pattern and motion vector encoder submodule, the optimum-size information that the motion compensated temporal filter submodule links to each other and transmits macro block with pattern and motion vector encoder submodule, the motion compensated temporal filter submodule transmitting moving that links to each other with the rate-distortion optimization control module is estimated control information, and how much decomposing module of motion compensated temporal filter submodule and two multiresolutions link to each other and transmit time domain high frequency and time domain low frequency frame of video information.
3. telescopic video sparse information processing system according to claim 1, it is characterized in that, described two-dimentional overcomplete wavelet decomposes submodule and comprises: one dimension line direction resolving cell and one dimension column direction resolving cell, wherein: one dimension line direction resolving cell links to each other with the Time Domain Decomposition module and transmits time domain low frequency and time domain high frequency frame of video information, one dimension line direction resolving cell links to each other with one dimension column direction resolving cell and transmits L subband and H subband, and one dimension column direction resolving cell links to each other with the phase equalization detection sub-module and transmits the LL subband, the LH subband, HL subband and HH subband.
4. telescopic video sparse information processing system according to claim 1 and 2, it is characterized in that, described rate-distortion optimization control module comprises: estimation parameter control submodule and coding work point parameter control submodule, wherein: the estimation parameter control submodule transmitting moving that links to each other with the Time Domain Decomposition module is estimated control information, and coding work point parameter is controlled the code check working point parameter that submodule links to each other with the entropy coding module and transmits entropy coding.
5. telescopic video sparse information processing system according to claim 1 is characterized in that, described bit stream multiplexer module will be under the parameter condition of different code checks working point the telescopic video code stream that generates of encoding process simply merge.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010194922 CN101848393B (en) | 2010-06-08 | 2010-06-08 | Telescopic video sparse information processing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010194922 CN101848393B (en) | 2010-06-08 | 2010-06-08 | Telescopic video sparse information processing system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101848393A CN101848393A (en) | 2010-09-29 |
| CN101848393B true CN101848393B (en) | 2011-08-31 |
Family
ID=42772822
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201010194922 Active CN101848393B (en) | 2010-06-08 | 2010-06-08 | Telescopic video sparse information processing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101848393B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2492330B (en) * | 2011-06-24 | 2017-10-18 | Skype | Rate-Distortion Optimization with Encoding Mode Selection |
| RU2612624C2 (en) * | 2012-07-02 | 2017-03-09 | Самсунг Электроникс Ко., Лтд. | Method and device for entropy video encoding and entropy video decoding method and device |
| CN108289043B (en) * | 2017-01-10 | 2020-06-05 | 工业和信息化部电信研究院 | Data consistency detection method and device |
| CN107016656B (en) * | 2017-04-01 | 2020-06-19 | 中国科学院光电技术研究所 | Wavelet sparse basis optimization method in image reconstruction based on compressed sensing |
| CN108259910B (en) * | 2018-02-09 | 2021-05-07 | 上海小蚁科技有限公司 | Video data compression method and device, storage medium and computing equipment |
| CN112184840B (en) * | 2020-09-22 | 2022-09-02 | 上海交通大学 | 3D point cloud compression system based on multi-scale structured dictionary learning |
| CN113079378B (en) * | 2021-04-15 | 2022-08-16 | 杭州海康威视数字技术股份有限公司 | Image processing method and device and electronic equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101427573A (en) * | 2006-02-16 | 2009-05-06 | 维德约股份有限公司 | System and method for thinning of scalable video coding bit-streams |
| CN101449588A (en) * | 2006-10-31 | 2009-06-03 | 株式会社Ntt都科摩 | Spatial sparsity induced temporal prediction for video compression |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| BRPI0812191A2 (en) * | 2007-06-08 | 2014-11-18 | Thomson Licensing | METHOD AND APPARATUS FOR SPARSITY-BASED MULTI-TREAD FILTERING |
-
2010
- 2010-06-08 CN CN 201010194922 patent/CN101848393B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101427573A (en) * | 2006-02-16 | 2009-05-06 | 维德约股份有限公司 | System and method for thinning of scalable video coding bit-streams |
| CN101449588A (en) * | 2006-10-31 | 2009-06-03 | 株式会社Ntt都科摩 | Spatial sparsity induced temporal prediction for video compression |
Non-Patent Citations (2)
| Title |
|---|
| Minh N. Do, Martin Vetterli.The contourlet transform: an efficient directional multiresolution image representation.《IEEE TRANSACTIONS ON IMAGE PROCESSING》.2005,第14卷(第12期),2091-2106. * |
| Roberto H. Bamberger, Mark J. T. Smith.A filter bank for the directional decomposition of images: theory and design.《IEEE TRANSACTIONS ON SIGNAL PROCESSING》.1992,第40卷(第4期),882-893. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101848393A (en) | 2010-09-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101848393B (en) | Telescopic video sparse information processing system | |
| US6519284B1 (en) | Encoding method for the compression of a video sequence | |
| CN1722838B (en) | Scalable video coding method and apparatus using base-layer | |
| Juliet et al. | A novel medical image compression using Ripplet transform | |
| CN103581691B (en) | A kind of towards sparse coefficient efficiently can parallel image coding method | |
| CN1319382C (en) | Method for designing architecture of scalable video coder decoder | |
| CN101631243B (en) | Image encoding/decoding method based on wavelet transformation | |
| Boucetta et al. | DWT based-approach for color image compression using genetic algorithm | |
| Inouri et al. | A fast and efficient approach for image compression using curvelet transform | |
| CN101783939A (en) | Picture coding method based on human eye visual characteristic | |
| Brahimi et al. | An efficient JPEG-2000 based multimodal compression scheme | |
| CN1825894A (en) | All phase cosine double orthogonal transformation and JPEG improving method | |
| Kim et al. | Wavelet transform image compression using human visual characteristics and a tree structure with a height attribute | |
| KR20080021268A (en) | Method and apparatus of image encoding/decoding based on 3d wavelet transform | |
| Liu et al. | Zerotree wavelet image compression with weighted sub-block-trees and adaptive coding order | |
| CN100477789C (en) | A method of inter-frame reinforcement based on Moving Joint Picture Experts Group 2000 | |
| Jilani et al. | A fuzzy neural networks based ezw image compression system | |
| Kim | Progressive Image Coding using Wavelet Transform | |
| Hassen et al. | The 5/3 and 9/7 wavelet filters study in a sub-bands image coding | |
| KR20030048719A (en) | Image Compression Technique based on Wavelet Transform for Image Transmission and Storage in the DVR(Digital Video Recording) System | |
| KR0170935B1 (en) | Method and apparatus for coding a fractal image in a wavelet transform domain | |
| Guleryuz et al. | Image compression with a geometrical entropy coder | |
| Fukuma et al. | An ROI image coding based on switching wavelet transform | |
| Dabhole et al. | Performance evaluation of traditional and adaptive lifting based wavelets with spiht for lossy image compression | |
| SVEC et al. | Image compression using transform coding methods |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |