CN103152581B - The error recovery method that a kind of spatial domain is multiple description coded - Google Patents
The error recovery method that a kind of spatial domain is multiple description coded Download PDFInfo
- Publication number
- CN103152581B CN103152581B CN201310085279.3A CN201310085279A CN103152581B CN 103152581 B CN103152581 B CN 103152581B CN 201310085279 A CN201310085279 A CN 201310085279A CN 103152581 B CN103152581 B CN 103152581B
- Authority
- CN
- China
- Prior art keywords
- spatial domain
- lost part
- distortion
- spatial
- time domain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000011084 recovery Methods 0.000 title claims abstract description 39
- 238000001514 detection method Methods 0.000 claims abstract description 4
- 230000033001 locomotion Effects 0.000 claims description 23
- 230000000295 complement effect Effects 0.000 description 12
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000013459 approach Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 239000012141 concentrate Substances 0.000 description 2
- 238000009792 diffusion process Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses the error recovery method that a kind of spatial domain is multiple description coded, comprise the following steps: (1) input original video stream, this original video stream is adopted and there are two spatial domains described multiple description codedly to decode; (2) error detection is carried out in two descriptions for described original video stream, when a description occurs to lose wherein, does to judge as follows: if a) described lost part has relativity of time domain, described lost part is carried out to the recovery of movable information; If b) described lost part has spatial correlation, spatial domain interpolation method is adopted to recover lost part; C) the time domain distortion degree of more described lost part and spatial domain distortion level: if time domain distortion degree is less than spatial domain distortion level, then adopt the method for step a) to recover; If time domain distortion degree is more than or equal to spatial domain distortion level, then the method for step b) is adopted to recover.Error recovery method provided by the invention, can strengthen the Fault recovery performance that spatial domain is multiple description coded.
Description
Technical field
The present invention relates to field of video transmission, be specifically related to the error recovery method that a kind of spatial domain is multiple description coded.
Background technology
In multimedia application, there is various network, unreliable due to Internet Transmission, fault-tolerant encoding (Error-resilientcoding) and Fault recovery (Error-concealment) become urgent problem in Video coding and decoding.
Have stronger fault-tolerance due to multiple description coded, become the hot research field of fault-tolerant encoding, be mainly divided into spatial domain multiple description coded, time domain is multiple description coded and frequency domain is multiple description coded.
Sampling algorithm under multiple description coded use spatial domain, spatial domain, image equalization is divided into two descriptions, between these two descriptions, there is stronger correlation, therefore, when a description is lost wherein, another can be utilized to describe part recover, thus realize certain wrong rejection ability.
The existing research about the multiple description coded algorithm in spatial domain mainly concentrates on coding side, and for the strategy that the Fault recovery of decoding end only takes spatial domain to recover, due to the strategy that decoding end uses single spatial domain to recover, the optimization of the multiple description coded algorithm in spatial domain is made to concentrate on the spatial correlation of raising two description, improve the complexity of encryption algorithm, reduce the quality of coding simultaneously, and Fault recovery can not be realized preferably.
Field picture coding is the specific coding mode H.264/AVC to field picture sequence in standard.Field picture coding is used to realize spatial domain to frame image sequence multiple description coded, two field picture coding corresponding two descriptions respectively.
Authorization Notice No. a kind of H.264 multi code Rate of Chinese character video flowing implementation method based on field coding (PAFF) that has been the disclosure of the invention of CN102118618A, comprise the H.264 video flowing that a roadbed is encoded in field or encoded in units of field by frame input source, to give up the mode of all strange field or whole even field to code stream, or in the mode of discard portion P field and B field, or give up the combination of mode with above-mentioned two kinds, give up the part field of code stream; Give up mode according to difference and respectively Reseal is carried out to the code stream of discard portion field, form the H.264 standard video stream of various code rate and/or resolution.
For the error recovery method of field picture, the multiple description coded error recovery method in spatial domain also can be considered as.There are some problems in the fault-tolerance approach of existing field picture, such as fault-tolerance approach does not consider the pixel difference between image, do not consider the motion relevance between two field picture, and for the frame image sequence of different characteristic, adaptability is not strong.
Summary of the invention
The invention provides the error recovery method that a kind of spatial domain is multiple description coded, significantly can strengthen the Fault recovery performance that spatial domain is multiple description coded, improve quality when video is unreliable to be transmitted, the method complexity is low, and the memory space needed for image is little.
The error recovery method that spatial domain is multiple description coded, comprises the following steps:
(1) input original video stream, this original video stream is adopted and there are two spatial domains described multiple description codedly to decode;
(2) error detection is carried out in two descriptions for described original video stream, when a description occurs to lose wherein, does to judge as follows:
If a) described lost part has relativity of time domain, then utilize the movable information in the description not occurring to lose, described lost part is carried out to the recovery of movable information, otherwise go to step b);
If b) described lost part has spatial correlation, then utilize the description not occurring to lose, adopt spatial domain interpolation method to recover lost part, otherwise go to step c);
C) the time domain distortion degree of more described lost part and spatial domain distortion level:
If time domain distortion degree is less than spatial domain distortion level, then the method for step a) is adopted to carry out the recovery of movable information to described lost part;
If time domain distortion degree is more than or equal to spatial domain distortion level, then the method for step b) is adopted to recover described lost part.
When one of them described in the present invention describes and occur to lose, all refer to that situation when losing occurs P frame.
As preferably, calculate two pixel difference distortion values described and differences in motion opposite sex distortion value, carry out the comparison of time domain distortion degree and spatial domain distortion level according to pixel difference distortion value and differences in motion opposite sex distortion value.
As preferably, utilize the pixel difference distortion value D of former frame
eCwith residual energy D
res, calculate current pixel difference distortion value and the differences in motion opposite sex distortion value with lost part description.
If the residual energy D that lost part describes
res<1, utilizes spatial domain interpolation method to recover lost part;
If the pixel difference distortion value D that lost part describes
eC<3, utilizes the movable information in the description not occurring to lose, described lost part is carried out to the recovery of movable information;
If meet D
res<D
eC-T, then utilize spatial domain interpolation method to recover lost part, and wherein, T is 8.
The error recovery method that a kind of spatial domain of the present invention is multiple description coded, significantly can strengthen the Fault recovery performance that spatial domain is multiple description coded, and improve quality when video is unreliable to be transmitted, the method complexity is low, and the memory space needed for image is little.
Accompanying drawing explanation
Fig. 1 is the flow chart of the multiple description coded error recovery method in spatial domain of the present invention;
Fig. 2 is Coastguard cycle tests the simulation experiment result figure of the present invention.
Embodiment
Below in conjunction with accompanying drawing, the error recovery method multiple description coded to spatial domain of the present invention is described in detail.
As shown in Figure 1, the error recovery method that a kind of spatial domain is multiple description coded, comprises the following steps:
(1) input original video stream, this original video stream is adopted and there are two spatial domains described multiple description codedly to decode;
(2) error detection is carried out in two descriptions for described original video stream, chooses different recovery policies according to type of error:
If when A two describes and occurs to lose, carry out Fault recovery by following method:
If A-1 is current is I frame, spatial domain interpolation method is adopted to recover;
If A-2 is current is P frame, the method for Block-matching is used to recover;
When B, a wherein description occurs to lose, recover by following method:
For convenience of description, two macro blocks of same position in two code streams formed in describing are defined as a complementary macro block pair.
Movable information is used to recover to describe, namely complementary macro block centering is utilized not lose the macro block (mb) type of macro block, segmentation type, motion vector and reference frame to recover the macro block information that complementary macro block centering loses macro block completely, then, normal decode process is used, reconstructing lost macro block.
Use Pixel Information to recover to describe, namely utilize complementary macro block centering not lose the Pixel Information of macro block, obtain losing macro block by spatial domain interpolation method.
If the macro block that the complementary macro block centering of B-1 is not lost is inter-coded macroblocks (INTRA), spatial domain interpolation method is adopted to recover;
On the channel occurring mistake, inter-coded macroblocks can stop the error diffusion process because time domain prediction causes, and recovers distortion relative to the time domain error of accumulation, and two spatial domain differences described are less.
Inter-coded macroblocks and inter-coded macroblocks may be comprised in the description of not losing, when do not lose be inter-coded macroblocks when, use spatial domain interpolation method to recover the macro block of complementary macro block centering loss, the error diffusion process that another one can be made to describe is effectively suppressed.
If the macro block that the complementary macro block centering of B-2 is not lost is inter-coded macroblocks (INTER), relativity of time domain and the spatial correlation of foundation lost part can only use Pixel Information to recover, and also movable information and Pixel Information can be combined and recover.
For two descriptions taking interframe encode, because the relativity of time domain of its pixel is strong, movable information is also corresponding has very strong correlation, for inter-coded macroblocks, describing the distortion level after adopting Pixel Information and movable information to recover by comparing two, selecting the restoration methods that distortion level is little.
D
min(x,y,n)=min(D
1(x,y,n),D
2(x,y,n))
Wherein, n is the sequence number of present frame in original video stream;
X, y are the coordinate (i.e. lost part position in the current frame) losing macro block;
M, N are width and the height of losing macro block;
The span of i is the integer of 0 ~ M;
The span of j is the integer of 0 ~ N;
D
1(x, y, n) is pixel difference distortion value, D
2(x, y, n) is differences in motion opposite sex distortion value;
F is for rebuilding macro block;
F
eC cfor the recovery macro block gone out by complementary macro block spatial domain interpolation;
V
x c, V
y cfor the motion vector recovered.
Because the reconstruction of inter-coded macroblocks is by forecasted variances distortion D
mforming with residual energy R two parts, is forecasted variances distortion D by the differences in motion of macro block opposite sex distortion value Further Division
mwith residual energy R:
Image pixel otherness distortion value D in two descriptions
1(x, y, n), by the interpolation distortion value D of complementary macro block centering two mutual demosaicings of macro block
eCobtain.
D
1(x,y,n)=D
EC(x,y,n)
Wherein, n is the sequence number of present frame in original video stream;
X, y are the coordinate (i.e. lost part position in the current frame) losing macro block;
M, N are width and the height of losing macro block;
The span of i is the integer of 0 ~ M;
The span of j is the integer of 0 ~ N;
F is for rebuilding macro block;
F
eC cfor the recovery macro block gone out by complementary macro block spatial domain interpolation.
Because current macro centering has a macro block to lose, therefore, the interpolation distortion value D of these two mutual demosaicings of macro block directly cannot be calculated
eC.
The interpolation distortion value D of macro block
eChave very large correlation with the texture of image, and the texture of front and back two two field picture has higher similitude, therefore, the pixel difference distortion value of the pixel difference distortion value calculation present frame of former frame can be used.
Use the motion vector of former frame, pixel difference distortion value is extrapolated, obtains the pixel difference distortion value D of present frame
1the estimated value of (x, y, n):
D
1(x,y,n)≈D
1(x+v
x,y+v
y,n-1)=D
EC(x+v
x,y+v
y,n-1)
Wherein, V
x, V
yit is the motion vector of the (n-1)th frame;
The differences in motion opposite sex distortion D of two codings described and information
2(x, y, n), by differentiating the residual energy R of code stream, the movable information such as motion vector and cut zone obtains, because have information dropout in describing, so the reconstruction macro block described cannot be obtained, the differences in motion opposite sex distortion of losing macro block directly cannot be calculated, because two motions described have larger similitude, therefore, in the present invention, motion prediction difference distortion D is set
mbe zero, the differences in motion opposite sex distortion D of macro block
2(x, y, n) is namely converted into residual energy D
res(R) value:
If a) lost part has relativity of time domain, then utilize the movable information in the description not occurring to lose, lost part is carried out to the recovery of movable information;
If time domain distortion degree is less than spatial domain distortion level, i.e. residual energy D
resduring <1, think that lost part has relativity of time domain, use complementary macro block centering not lose the motion vector of macro block and reference frame as the motion vector of current loss macro block and reference frame, then use normal decode process to decode.
If b) lost part has spatial correlation, then utilize the description not occurring to lose, adopt spatial domain interpolation method to recover lost part;
If time domain distortion degree is more than or equal to spatial domain distortion level, i.e. pixel difference distortion value D
eCduring <3, think that lost part has spatial correlation, the picture element interpolation after the macroblock reconstruction using complementary macro block centering not lose, generate the reconstruction pixel value losing macro block, no longer carry out decode procedure.
C) time domain distortion degree and the spatial domain distortion level of lost part is compared:
If time domain distortion degree and spatial domain distortion level are all comparatively large, then compare residual energy D
reswith pixel difference distortion value D
eCsize, due under same packet loss condition, for same code stream, the quality that the mode taking merely motion vector to recover does not adopt spatial domain interpolation method to recover is high, therefore, under normal circumstances, the method of preferential use spatial domain interpolation, concrete Rule of judgment is as follows:
If meet D
res<D
eC-T, just takes motion vector restoration methods, and wherein, T is 8.
The inventive method (TASTEC) is adopted to carry out Fault recovery to the coastguard video flowing taked based on the spatial domain of field coding is multiple description coded.The method of contrast is traditional spatial domain restoration methods (MDCEC) (list of references Mao-QuanLi, Zheng-QuanXu, andYan-YanXu.ARobustVideoCodingSchemeforMobileWirelessCo mmunications [J] .JournalofMultimedia, 2009, 4 (6): 405-411) and revert to self-adapting airspace symbol recovery method (ECSD) (the list of references M.Q.Li of benchmark with spatial domain, Z.Q.Xu.AnErrorResilientCodingSchemeforVideoTransmissionB asedonPixelLineDecimation [C] .SecondInternationalSymposiumonIntelligentInformationTec hnologyApplication, Shanghai, 2008, 2:315-320).
Video flowing is all encoded by JM14.0, takes a coding mode, and use cbr (constant bit rate) 1Mbps, frame per second is that 30 frames are per second; Use many slice pattern, all macro blocks separate size and sub-block segmentation size is all used; The network packet loss rate of test is 5%, 10% and 20%.
The method of three kinds of Fault recovery is all integrated in the decoder of JM14.0, for same packet loss code stream, different error recovery procedure is used to recover respectively, then the PSNR quality of restoration result is compared, as shown in Figure 2, the improvement amplitude of the present invention to video quality can reach 0.7 ~ 1.3dB.
Claims (4)
1. the error recovery method that spatial domain is multiple description coded, is characterized in that, comprises the following steps:
(1) input original video stream, this original video stream is adopted and there are two spatial domains described multiple description codedly to decode;
(2) error detection is carried out in two descriptions for described original video stream, and a description is wherein lost, and does not lose when being described as inter-coded macroblocks, does to judge as follows:
If a) described lost part has relativity of time domain, then utilize the movable information in the description not occurring to lose, described lost part is carried out to the recovery of movable information, otherwise go to step b);
If b) described lost part has spatial correlation, then utilize the description not occurring to lose, adopt spatial domain interpolation method to recover lost part, otherwise go to step c);
C) the time domain distortion degree of more described lost part and spatial domain distortion level:
If time domain distortion degree is less than spatial domain distortion level, then step method a) is adopted described lost part to be carried out to the recovery of movable information;
If time domain distortion degree is more than or equal to spatial domain distortion level, then adopt step b) method described lost part is recovered.
2. the error recovery method that spatial domain as claimed in claim 1 is multiple description coded, it is characterized in that, calculate two pixel difference distortion values described and differences in motion opposite sex distortion value, carry out the comparison of time domain distortion degree and spatial domain distortion level according to pixel difference distortion value and differences in motion opposite sex distortion value.
3. the error recovery method that spatial domain as claimed in claim 2 is multiple description coded, is characterized in that, utilize the pixel difference distortion value D of former frame
eCwith residual energy D
res, calculate current pixel difference distortion value and the differences in motion opposite sex distortion value with lost part description.
4. the error recovery method that spatial domain as claimed in claim 3 is multiple description coded, is characterized in that, if the residual energy D that lost part describes
res<1, utilizes spatial domain interpolation method to recover lost part;
If the pixel difference distortion value D that lost part describes
eC<3, utilizes the movable information in the description not occurring to lose, described lost part is carried out to the recovery of movable information;
If meet D
res<D
eC-T, then utilize spatial domain interpolation method to recover lost part, and wherein, T is 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310085279.3A CN103152581B (en) | 2013-03-15 | 2013-03-15 | The error recovery method that a kind of spatial domain is multiple description coded |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310085279.3A CN103152581B (en) | 2013-03-15 | 2013-03-15 | The error recovery method that a kind of spatial domain is multiple description coded |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103152581A CN103152581A (en) | 2013-06-12 |
| CN103152581B true CN103152581B (en) | 2016-04-13 |
Family
ID=48550421
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310085279.3A Active CN103152581B (en) | 2013-03-15 | 2013-03-15 | The error recovery method that a kind of spatial domain is multiple description coded |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103152581B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105245908B (en) * | 2015-10-27 | 2018-06-29 | 大连海事大学 | A kind of video fault-tolerant coding method based on error correction preferred value feedback |
| CN110740380A (en) * | 2019-10-16 | 2020-01-31 | 腾讯科技（深圳）有限公司 | Video processing method and device, storage medium and electronic device |
-
2013
- 2013-03-15 CN CN201310085279.3A patent/CN103152581B/en active Active
Non-Patent Citations (4)
| Title |
|---|
| Error Concealment For Slice Group Based Multiple Description Video Coding;D. Wang等;《IEEE》;20051231;全文 * |
| Error-Resilient Video Coding Using Multiple Description Motion Compensation;Yao Wang等;《IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY》;20040630;第12卷(第6期);全文 * |
| Joint Temporal and Spatial Error Concealment for Multiple Description Video Coding;Wen-Jiin Tsai等;《IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY》;20101231;第20卷(第12期);全文 * |
| 视频多描述编码与差错隐藏的联合抗误码算法;刘杰平，余英林;《华南理工大学学报(自然科学版)》;20040731;第32卷(第7期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103152581A (en) | 2013-06-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101729892B (en) | Coding method of asymmetric stereoscopic video | |
| CN102630012B (en) | Coding and decoding method, device and system based on multiple description videos | |
| CN101163250B (en) | Boundary gradient based video stream fault tolerance method | |
| CN102026001B (en) | Method for evaluating importance of video frame based on motion information | |
| CN102256133A (en) | Distributed video coding and decoding method based on side information refining | |
| CN109905717A (en) | A kind of H.264/AVC Encoding Optimization based on Space-time domain down-sampling and reconstruction | |
| CN103370934A (en) | Multi-view video encoding/decoding apparatus and method | |
| CN103139571A (en) | Video fault-tolerant error-resisting method based on combination of forward error correction (FEC) and WZ encoding and decoding | |
| CN103475884A (en) | HBP coding format oriented stereoscopic video B frame loss error concealment method | |
| CN101242540B (en) | Error control method for video decoder chip | |
| CN103152581B (en) | The error recovery method that a kind of spatial domain is multiple description coded | |
| CN102625104A (en) | Video coding method | |
| CN102223544B (en) | Method for processing error after detecting error in H264 video stream | |
| Kulupana et al. | Error resilience aware motion estimation and mode selection for HEVC video transmission | |
| CN104363461B (en) | The error concealing method of frame of video and apply its video encoding/decoding method | |
| Tang et al. | End-to-end distortion modeling for error-resilient screen content video coding | |
| CN103313064B (en) | Based on the time domain error hidden method of inter-frame mode and movement repair | |
| CN101888552B (en) | Local compensation based frame-skipping coding and decoding methods and devices | |
| CN103747261A (en) | Layered perceptual encryption method of H.264 compressed domain based on motion reference structure | |
| CN102404568A (en) | P-frame interpolating method aimed at whole frame loss error concealment in video coding and decoding | |
| CN111726636A (en) | HEVC (high efficiency video coding) coding optimization method based on time domain downsampling and frame rate upconversion | |
| Pang et al. | Relativity analysis-based error concealment algorithm for entire frame loss of stereo video | |
| Chen et al. | An Adaptive Spatio-Temporal Error Concealment Scheme Based on H. 264/AVC. | |
| CN104144347A (en) | H.264/AVC video I-frame error recovery method based on reversible data hiding | |
| CN107835422B (en) | HEVC multi-description image coding algorithm based on significance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |