CN100486338C - Fault disguising method for dynamic image decoding by using multi-frame buffer - Google Patents
Fault disguising method for dynamic image decoding by using multi-frame buffer Download PDFInfo
- Publication number
- CN100486338C CN100486338C CN 200510029453 CN200510029453A CN100486338C CN 100486338 C CN100486338 C CN 100486338C CN 200510029453 CN200510029453 CN 200510029453 CN 200510029453 A CN200510029453 A CN 200510029453A CN 100486338 C CN100486338 C CN 100486338C
- Authority
- CN
- China
- Prior art keywords
- frame
- mentioned
- image
- macroblock
- image frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
This invention relates to error hiding method by use of multiple buffer in dynamic image process, which comprises the following steps: correcting hiding module on above image modules; referring above mobile correction before image macro module whether there is error; according to above results, under situation of error before, through above macro module mobile vector values to compute; using above formula to read out reference macro module from front images; then using above macro module to move and correct current image.
Description
[technical field]
The invention relates to the error concealing method when utilizing the multi-screen buffering area that dynamic image is decoded.Specifically, be exactly about the error concealing method when MPEG-4 (Motion Picture Experts Group-4) dynamic image is decoded.Error concealing method when utilizing according to the decoding of MPEG-4 of the present invention (Motion PictureExperts Group-4) dynamic image, to the MPEG-4 (Motion Picture Experts Group-4) by data network or line network when bit stream information is transmitted, just can effectively prevent the problem that descends in the quality of image that MPEG-4 (Motion Picture ExpertsGroup-4) is occurred when decoding that causes because of network error.
[background technology]
For the system of transmitting or carrying out encoding process after digital multimedia data is compressed again, for example: H.263 or under the situation of MPEG4, when dynamic image being compressed and encode, just the data with digitized video are that unit handles with module (macro block).In it is encoded (encoding), it comprises following process: i.e. mensuration and correction to moving; Calculate the variation (DCT) of cosine, the quantization of DCT coefficient is handled, and to the encode process of (VLC) of variable-length.In it is decoded (decoding), the process that experience is exactly above-mentioned inverse process of encoding, and promptly it comprises following process: inverse guantization (IQ) is handled, and inverse DCT is to processes such as the correction of action and mensuration.As mentioned above, for the system that digital image signal is transmitted and receives, can the mistake that be produced by channel circumstance be detected.Then, to taking place, this module that sustains damage when wrong revises.Simultaneously, utilize the method for error detection and error concealing for the image quality that improves recovery (reception) image.
For example:, when mistake is detected, will utilize coding grammer (syntax) information for being the system that digital image signal is transmitted and receives of unit with the module.Break away from data under the situation of grammer (syntax) information, just look it mistake has taken place.For the module that makes a mistake, can utilize the hidden method in time or space again it to be revised.Like this, just can improve the image quality of reception image greatly.
As mentioned above, for the error-detecting method that utilizes syntactic information, it is applicable to following several situation.For example: when being that unit is when handling digital image signal with the module, break away from mobile vector under the situation of specific scope, or break away from when variable-length under the situation of coding (VLC) form, or varies with cosine (DCT) coefficient breaks away under the situation of specific scope, and the number of the DCT coefficient in a module to surpass the situation of specific number inferior.
Fig. 1 is the ordinary construction component drawings of expression according to the wrong corresponding MPEG-4 decoder of conventional art.
With reference to Fig. 1 as can be seen, according to the wrong corresponding MPEG-4 decoder (decoder) of conventional art when the bit stream information of input is carried out VLD (Variable Length Decoding), will pass through error detection and corresponding process, and remaining process is exactly to pass through the process identical with the process that existing dynamic image is decoded.
Generally speaking, for the MPEG-4 decoder, it will use the method for error concealing (error concealment) when the module that makes a mistake is detected.
When the encoded dynamic image signal of handling the back and transmitting was decoded, in general, its employed error concealing method roughly can be divided into 2 kinds.The error concealing method (TemporalError concealment) that promptly utilizes the homophylic error concealing method in space (Spatial Error Concealment) and utilize time correlation to concern.
For with the module being the system that unit carries out the digital image signal processing, above-mentioned 2 kinds of methods are to have bigger correlation between the adjacent block of supposition (feature) signal of video signal, or have between the picture in the adjacent time period under the situation of bigger relevance and the method for using.
Fig. 2 a and Fig. 2 b are the schematic diagrames of representing respectively utilizing the homophylic error concealing method in space (SpatialError Concealment) and utilizing the error concealing method (Temporal ErrorConcealment) of time correlation relation to describe.
With reference to Fig. 2 as can be seen, at first, Fig. 2 a method that the module of damaged utilizes the value of the adjacent module that does not sustain damage of its periphery to carry out interpolation to making a mistake is illustrated.That is to say, the above-mentioned homophylic error concealing method in space (Spatial Error Concealment) that utilizes is to be illustrated in the homophylic method of utilizing the space in the signal of video signal, and it is the information of the normal macroblock (macro block) that utilizes its periphery and method that the macroblock (macro block) that makes a mistake is hidden.
Here, above-mentioned macroblock (macro block) means that it is the part suitable with the 16*16 picture element of a frame.
In addition, the method that the module of picture relevant position is filled up before the module utilization of diagram 2b damaged to making a mistake is illustrated.That is to say, the above-mentioned error concealing method (Temporal Error Concealment) that utilizes time correlation relation be expression utilize from before the temporal homophylic method that begins of image, it is to be illustrated under the current image state, if mistake has taken place, with regard to utilize from before the information of the regulation that begins of image method that the mistake that is taken place is hidden.
In this case, the information of the regulation that begins for the image that utilizes before above-mentioned, (Motion Vector, value MV) is inferred with mobile vector and utilize the value of mobile vector (MV) of macroblock (macro block).Then, utilize with from before the data error of the corresponding macroblock of value (macro block) of the image mobile vector that begins to infer hide.
In other words, the above-mentioned error concealing method (Temporal ErrorConcealment) of time correlation relation that utilizes has utilized mobile vector (MV), that is to say, if mistake has taken place in the regulation mobile vector (MV) of current picture,
It is hidden mistake with regard to mobile vector (MV) information of the same position of picture before utilizing.
Here, the above-mentioned homophylic error concealing method in space (Spatial ErrorConcealment) that utilizes mainly utilizes I-frame (Intra frame).That is to say, only can carry out decoded pictures by the data in the bit stream information of current image, the above-mentioned homophylic error concealing method in space (Spatial Error Concealment) that utilizes is suitable for when picture makes a mistake in this case.And the above-mentioned error concealing method (Temporal ErrorConcealment) of time correlation relation that utilizes mainly utilizes P-frame (Predictive frame).That is to say that for carrying out decoded pictures with reference to former image, the above-mentioned error concealing method (Temporal Error Concealment) of time correlation relation that utilizes is suitable for when picture makes a mistake in this case.
But, as mentioned above, if mistake is hidden processing by the error concealing method of foundation conventional art, so, because may there be situation about suffering damage in the original data of institute's reference.Therefore, the module of institute's reference will produce error propagation.This is its existing problem.
Fig. 3 is expression according to traditional technology schematic diagram with reference to the error propagation phenomenon of the module that made a mistake the in the past when shift action of image is revised.
With reference to Fig. 3 as can be seen, in foundation the 1st embodiment of the present invention, under the situation that occurs P-frame continuously, in the process that image information is decoded, just mistake is detected, and the macroblock (macro block) that makes a mistake is carried out error concealing.Then, under the situation that the bit stream information of the picture after it all normally occurs, even normally decode, because the macroblock (macro block) through error concealment process in the image of institute's reference has been carried out reference, therefore, will store the possibility of proceeding error propagation (error propagation).
If this error propagation phenomenon has taken place, destination's the quality of image will be quite bad.This is its existing problem.
Therefore, just having pressed for a kind of error concealing method, is exactly in wrong the generation, not only can hide mistake, and can prevent generation to the propagation phenomenon of this mistake effectively.
[summary of the invention]
Therefore, the present invention researches and develops in order to address the above problem.The objective of the invention is to provides a kind of error concealing method when utilizing the multi-screen buffering area that dynamic image is decoded for the user.According to the error concealing method when utilizing the multi-screen buffering area that dynamic image is decoded of the present invention, not only can store, and can store the former image frame (n-2_frame) of the image frame before it to current image frame (n-frame) and the image frame (n-1_frame) before it.Then, when current normal image frame is decoded, part through error concealment process in to reference picture (n-1_frame) is carried out under the situation of reference, just utilize the multi-screen buffering area, the macroblock (macro block) of the picture (n-2_frame) before the reference picture (n-1_frame) is carried out reference.Like this, just can prevent wrong the propagation, and finally can improve the quality of image.
To achieve these goals, has the feature that constitutes by following step according to the error concealing method when utilizing the multi-screen buffering area that dynamic image is decoded of the present invention: that is to say, for according to the coding/decoding method of the current image frame (n_frame) concerning current normal input of the present invention, it has the feature that is made of following step, to the mobile preparation process of revising of the hiding module (inter macro) of above-mentioned current image frame (n_frame); When above-mentioned shift action was revised, the macroblock (macro block) of the image frame (n-1_frame) that reference was former was to confirm whether to have taken place wrong step; According to above-mentioned affirmation result, image frame has taken place under the wrong situation before above-mentioned, the step of the value of stipulating being calculated by mobile vector (MV) value of the macroblock (macro block) around the above-mentioned module that makes a mistake; The value of utilizing the regulation that aforementioned calculation comes out before above-mentioned image frame (n-1_frame) before will can be used as reference the image frame (n-2_frame) macroblock (macro block) read out, then, utilize above-mentioned macroblock (macro block) and to the mobile step of revising of above-mentioned current image frame (n_frame).
Also have following feature according to the error concealing method when utilizing the multi-screen buffering area that dynamic image is decoded of the present invention here: above-mentioned current image frame (n_frame) and in the past image frame (n-1_frame) be exactly P-frame (Predictive-frame).
In addition, also have following feature according to the error concealing method when utilizing the multi-screen buffering area that dynamic image is decoded of the present invention: before above-mentioned the macroblock (macroblock) of image frame (n-1_frame) go up the mistake that takes place also comprise above-mentioned before in the image frame (n-1_frame) through the part of error concealment process.
In addition, also have following feature according to the error concealing method when utilizing the multi-screen buffering area that dynamic image is decoded of the present invention: the value of afore mentioned rules is exactly bandpass filter (Medianfilter) and the median calculated in the mobile vector value of adjacent 4 macroblocks (macro block) of the above-mentioned relatively wrong module that takes place and the utilization.
In addition, also has following feature according to the error concealing method when utilizing the multi-screen buffering area that dynamic image is decoded of the present invention, be that it also comprises the steps: promptly according to above-mentioned affirmation result, under the situation that image frame does not make a mistake before above-mentioned, utilize above-mentioned before the macroblock (macro block) of image frame step that shift action is revised.
Under the situation of MPEG-4, for the situation that dynamic image is compressed and encodes, is that unit handles with digital image data with macroblock (macro block), under the situation of (encoding) of encoding, just need promptly to comprise: shift action is inferred and revised, calculate varies with cosine (DCT) through following process, the quantization of DCT coefficient, and the process of variable-length encoded (VLC) etc.And under the situation of (decoding) of decoding, the opposite process of process when needing experience with above-mentioned the coding promptly comprises: inverse guantization (IQ), inverse DCT is revised and the process inferred etc. shift action.In general, this is more common situation.
In addition, the transmission receiving system for receiving this digital image signal detects the error situation that is produced by the network channel environment, and then, the module that sustains damage when this mistake takes place is revised again.In this case, in order to improve the image quality of recovery (reception) image, just need to utilize wrong detection and hidden method.
As mentioned above, in above-mentioned error concealing method, the error concealing method (Temporal Error Concealment) that utilizes time correlation relation be utilize from before the temporal similarity that begins of picture image, under the situation that the image of current picture makes a mistake, for the mistake that is taken place is hidden, will by utilize from before the method for information of the regulation that begins of picture image, in this case, for utilize from before the information of the regulation that begins of picture image, and need utilize macroblock (macroblock) the mobile vector value around it and current mobile vector (Motion Vector) is inferred.Then, with regard to utilize with from before the picture image corresponding macroblock of mobile vector value (macroblock) that begins to infer data and the mistake that is taken place is hidden.
But, in this case, when with from before the data of the picture image corresponding macroblock of mobile vector value (macro block) that begins to infer be under the data conditions of error module, so, above-mentioned mistake also will be propagated to current picture.Thereby the propagation that will make a mistake (error propagation) phenomenon.This is its existing problem.
Research and develop for overcoming the problems referred to above according to the error concealing method when utilizing the multi-screen buffering area that dynamic image is decoded of the present invention.Has following feature according to the error concealing method when utilizing the multi-screen buffering area that dynamic image is decoded of the present invention: when to P-frame (Predictive-frame, with reference to former image and decoded pictures) when decoding continuously, to current normal image frame (n_frame) when decoding, carrying out under the situation of reference with reference to the part of passing through error concealment process in as picture (n-1_frame) before with reference to picture (reference frame), in order to stop the generation of the phenomenon of mistake being propagated to its next picture through the macroblock (macroblock) of error concealment process, then with above-mentioned former picture with reference to picture, being about to n-2_frame stores on the picture buffer once more, then, it is utilized.
Here, above-mentioned so-called with reference to picture (reference frame), the image frame of reference when shift action is revised exactly, it is meant the image frame before the current relatively image frame.
But under the situation of foundation embodiments of the invention, for former picture (n-1_frame), it has according to each macroblock (macro block) the hiding figure (conceal map) that situation about whether making a mistake is write down.Suppose not have successional mistake on the macroblock (macro block) of reference among said n-2_frame here.
As mentioned above, for according to the error concealing method when utilizing the multi-screen buffering area that dynamic image is decoded of the present invention, it has utilized the multi-screen buffering area, part through error concealment process is being carried out under the situation of reference, just can to carry out reference with reference to the macroblock (macroblock) of the former picture of picture, like this, just can effectively stop mistake to be propagated, and finally improve the image quality of image.This is according to the advantage that error concealing method has when utilizing the multi-screen buffering area that dynamic image is decoded of the present invention.
[description of drawings]
Fig. 1 is the ordinary construction component drawings of expression according to the wrong corresponding MPEG-4 decoder of conventional art.
Fig. 2 a and Fig. 2 b represent schematic diagram that Spatial Error Concealment and TemporalError Concealment are described respectively.
Fig. 3 is expression according to traditional technology schematic diagram with reference to the error propagation phenomenon of the module that made a mistake the in the past when shift action of image is revised.
Fig. 4 is the precedence diagram of expression to describing according to the error concealing method when utilizing the multi-screen buffering area that dynamic image is decoded of the present invention.
Fig. 5 is the schematic diagram that expression overcomes the error propagation phenomenon according to the present invention.
[embodiment]
Below, with reference to the accompanying drawings to being described in more details according to desirable embodiment of the present invention.
Fig. 4 is the precedence diagram of expression to describing according to the error concealing method when utilizing the multi-screen buffering area that dynamic image is decoded of the present invention.Fig. 5 is the schematic diagram that expression overcomes the error propagation phenomenon according to the present invention.
But, here, just the data of transmitting from encoder are carried out in the process of MPEG-4 decoding, the process (error concealing process) that shift action is revised is illustrated.
That is to say, here, be that following situation is described, promptly in the process that the image frame (n_frame) of current normal input is decoded, for the correction of Inter macroblock (macrobl ock) being carried out shift action, under the situation of the module that makes a mistake in the picture (n-1_frame) before the reference, carry out the process of respective handling in order to prevent mistake from propagating (error propagation).Here, said process is described.
With reference to Fig. 4 as can be seen, at first, for the situation that the current image frame (n_frame) of current normal input is decoded, prepare step ST400 for shift action correction to the inter macroblock (MB) of above-mentioned current image frame (n_frame).
Here, have following feature according to the error concealing method when utilizing the multi-screen buffering area that dynamic image is decoded of the present invention: the image frame of above-mentioned current normal input (n_frame) is exactly P-frame (Predictive-frame), that is to say that it is with reference to former image and decoded pictures.
Then, when above-mentioned shift action is revised, whether the macroblock (macroblock) of the former image frame (n-1_frame) of institute's reference wrong situation has been taken place confirmed step ST410.
Here, before above-mentioned the effect of image frame (n-1_frame) or P-frame be exactly as above-mentioned current image frame (n_frame) with reference to picture (reference frame).
In addition, the macroblock (macro block) of above-mentioned image frame (n-1_frame) is in the past gone up the part that also comprises process error concealment process in the above-mentioned image frame (n-1_frame) in the past in the mistake that takes place.To this, can utilize hiding figure (conceal map) to confirm.
Then, according to the above-mentioned result who confirms by hiding figure, under the situation about making a mistake in the image frame before above-mentioned, then can calculate step ST420 to the value of regulation by the mobile vector value (MV) of the macroblock (macro block) around the module that makes a mistake.
As shown in Figure 5, according among first embodiment of the present invention, for the mobile vector value of 4 macroblocks (macro block) adjacent with the module that makes a mistake, bandpass filter in can using (Median filter) and calculate its median.
Then, the setting that utilizes aforementioned calculation to come out, the macroblock (macro block) that will can be used as reference before the image frame before above-mentioned (n-1_frame) the image frame (n-2_frame) reads out, then, utilize above-mentioned macroblock (macro block) and to mobile correction of above-mentioned current image frame (n_frame), step ST430.
Here, the setting that aforementioned calculation is come out is as the value of the mobile vector that calculates, and can obtain its median according to the scope of application of bandpass filter (Median filter) in above-mentioned.
In addition, have following feature according to the error concealing method when utilizing the multi-screen buffering area that dynamic image is decoded of the present invention: image frame (n-2_frame) is exactly P-frame or I-frame before above-mentioned.
But, the affirmation result who obtains by above-mentioned hiding figure, under the situation that image frame does not make a mistake before above-mentioned, then the ordinary circumstance with the foundation conventional art is identical, utilize the above-mentioned macroblock (macro block) of image frame in the past can revise step ST440 to shift action.
By above-mentioned description, the related work personnel can carry out various change and modification fully in the scope that does not depart from this invention technological thought.
Therefore, the technical scope of this invention is not limited to the content on the specification, must determine its technical scope according to interest field.
Claims (5)
1, utilize the error concealing method of multi-frame buffer in the dynamic image decoding process, comprising:
For concerning the coding/decoding method of the current image frame (n_frame) of current normal input, described error concealing method has the feature that is made of following step: to the mobile preparation process of revising of the hiding module (inter macro) of above-mentioned current image frame (n_frame);
When above-mentioned shift action was revised, the macroblock (macro block) of the image frame (n-1_frame) that reference was former was to confirm whether to have taken place wrong step;
According to above-mentioned affirmation result, image frame has taken place under the wrong situation before above-mentioned, the step of the value of stipulating being calculated by mobile vector (MV) value of the macroblock (macro block) around the above-mentioned macroblock that makes a mistake; The value of described regulation is exactly bandpass filter and the median calculated in the mobile vector value of adjacent 4 macroblocks (macro block) of the above-mentioned relatively wrong macroblock that takes place and the utilization;
The value of utilizing the regulation that aforementioned calculation comes out before above-mentioned image frame (n-1_frame) before will can be used as reference the image frame (n-2_frame) macroblock (macro block) read out, then, utilize above-mentioned macroblock (macro block) and to the mobile step of revising of above-mentioned current image frame (n_frame).
2, as the claim item 1 described error concealing method of multi-frame buffer in the dynamic image decoding process that utilize, it is characterized in that:
Above-mentioned current image frame (n_frame) and former image frame (n-1_frame) thereof are exactly P-frame (Predictive-frame).
3, as the claim item 1 described error concealing method of multi-frame buffer in the dynamic image decoding process that utilize, it is characterized in that:
The mistake that the macroblock (macro block) of image frame (n-1_frame) upward takes place before above-mentioned also comprises the middle part through error concealment process of above-mentioned image frame (n-1_frame) in the past.
4, as the claim item 1 described error concealing method of multi-frame buffer in the dynamic image decoding process that utilize, it is characterized in that:
The former image frame (n-2_frame) of image frame is exactly P-frame or I-frame before above-mentioned.
5, as the claim item 1 described error concealing method of multi-frame buffer in the dynamic image decoding process that utilize, it is characterized in that, also comprise:
Promptly according to above-mentioned affirmation result, under the situation that image frame does not make a mistake before above-mentioned, utilize above-mentioned before the macroblock (macro block) of image frame step that shift action is revised.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200510029453 CN100486338C (en) | 2005-09-07 | 2005-09-07 | Fault disguising method for dynamic image decoding by using multi-frame buffer |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200510029453 CN100486338C (en) | 2005-09-07 | 2005-09-07 | Fault disguising method for dynamic image decoding by using multi-frame buffer |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1929617A CN1929617A (en) | 2007-03-14 |
| CN100486338C true CN100486338C (en) | 2009-05-06 |
Family
ID=37859361
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200510029453 Expired - Fee Related CN100486338C (en) | 2005-09-07 | 2005-09-07 | Fault disguising method for dynamic image decoding by using multi-frame buffer |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100486338C (en) |
-
2005
- 2005-09-07 CN CN 200510029453 patent/CN100486338C/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| CN1929617A (en) | 2007-03-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5007322B2 (en) | Video encoding method | |
| US9042439B2 (en) | Reference frame ordering for multi-view coding | |
| US6081551A (en) | Image coding and decoding apparatus and methods thereof | |
| EP1902585B1 (en) | Coding dependency indication in scalable video coding | |
| RU2573257C2 (en) | Image signal decoding device, image signal decoding method, image signal encoding device, image signal encoding method and programme | |
| AU1252899A (en) | Transcoding | |
| CN100555332C (en) | Use comprises that the prediction of a plurality of macro blocks and nonanticipating picture are to picture sequence Methods for Coding and device | |
| CN1306822C (en) | Vido decoder based on software and hardware cooperative control | |
| US6795498B1 (en) | Decoding apparatus, decoding method, encoding apparatus, encoding method, image processing system, and image processing method | |
| US20060133490A1 (en) | Apparatus and method of encoding moving picture | |
| JP2007088922A (en) | Encoding apparatus and encoding method | |
| US20060133491A1 (en) | Video codec | |
| US8155186B2 (en) | Bit stream recording medium, video encoder, and video decoder | |
| US20130083858A1 (en) | Video image delivery system, video image transmission device, video image delivery method, and video image delivery program | |
| CN100486338C (en) | Fault disguising method for dynamic image decoding by using multi-frame buffer | |
| JP5472094B2 (en) | Moving image conversion apparatus, moving image distribution system, moving image conversion method, and program | |
| KR100543607B1 (en) | Method for decoding of moving picture | |
| CN1929619A (en) | Fault disguising method for dynamic image decoding | |
| US20080317121A1 (en) | Rate control methods and devices | |
| JP2001197490A (en) | Method for encoding digital animation image comprising gray shape information | |
| CN1964497A (en) | A method for error concealment in the case of motive video decoding | |
| US8358694B2 (en) | Effective error concealment in real-world transmission environment | |
| US7529413B2 (en) | Method for reducing data amount in an image memory system | |
| KR20050089453A (en) | Error concealment method in decoding motion picture by using multiple frame buffer | |
| KR20050089457A (en) | Error concealment method in decoding motion picture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20090506 Termination date: 20100907 |