CN101272494B - Video encoding/decoding method and device using synthesized reference frame - Google Patents
Video encoding/decoding method and device using synthesized reference frame Download PDFInfo
- Publication number
- CN101272494B CN101272494B CN 200810087379 CN200810087379A CN101272494B CN 101272494 B CN101272494 B CN 101272494B CN 200810087379 CN200810087379 CN 200810087379 CN 200810087379 A CN200810087379 A CN 200810087379A CN 101272494 B CN101272494 B CN 101272494B
- Authority
- CN
- China
- Prior art keywords
- frame
- reference frame
- synthesized reference
- current
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention discloses a video coding and decoding method and a device using a synthetic reference frame, which uses coded image information as the reference frame and code and decodes the image frame of a successive code. The reference frame includes the synthetic reference frame which is reset and renewed by a reconstructive frame in each frame and then renewed by the reconstructive image information of a following inter-frame prediction frame according to a macro-block or a sub block of the macro-block. The renewed synthetic reference frame is used as a reference frame for the prediction during the coding of the successive image frame and is renewed continuously. The video coding and decoding method and the device of the invention can increase the accuracy of the video sequence motion estimation in video monitoring, reducing prediction errors and improving coding efficiency without the increasing of the complexity.
Description
Technical field
The invention belongs to compression of digital video encoding and decoding technique field, the method that relates to a kind of motion compensation interframe prediction coding and decoding video, and the video coding apparatus and the video decoder that adopt this method, especially use the reference frame of a kind of new synthesized reference frame, replace or part replaces traditional reference frame as interframe prediction coding and decoding.
Background technology
Video monitoring is a core part in the safety precaution industry, because the security protection industry has lower code check to the demand of interconnecting between system, and video information there is an active demand of a large amount of storages, expansion along with the video monitoring scale, a large amount of video information transmission and storages need jumbo network service and disk, and cost is very high.And in video monitoring, the video camera of gathering video image has fixed installation, and the control of band cloud platform and camera lens is also arranged, and promptly can change the scene of monitoring, but most scene being shot is constant for a long time.So the video field signal scenic spot can be constant for a long time mostly in video monitoring, but scene will change when the positive Be Controlled of video camera.When weather changes, sail if any cloud, when sunlight changed situations such as making the shade variation, the video image scene also can be affected.The video camera that is used for video monitoring has bigger noise usually, and especially when illumination condition was bad, as indoor, night, noise was bigger, and the compression efficiency that this has just influenced video greatly causes video frequency coding rate very high.Along with the extensive use of video telephone, the video coding and decoding technology that is applied in the video telephone is also more and more crucial.The definition of image and continuity and time delay are to weigh the main aspect of video telephone quality.Often also there is bigger noise in video image in video telephone, has influenced picture quality, and video coding efficient has been brought influence.
The frequency band broad that in general video information occupies, especially live image is stored and to transmit required data volume big, under the condition of not compressing, the transmission rate of standard definition television (SDTV) is about 140Mbit/s, and high definition TV (HDTV) is then up to 1000Mbit/s.The speed of HDTV after overcompression has only 20Mbit/s, and video telephone is when transmitting on PSTN, and compressible is about 20Kbit/s.And there is the multi-user to monitor simultaneously in the video monitoring,, causes in the video monitoring data volume of transmission and video recording very huge along with increasing of video camera number.
For conserve bandwidth, more effectively utilize existing narrowband network, reduce the memory space and the complexity of digital device, must compress efficiently video information.The appearance of a series of video compression coding standards (H.261, H.263, MPEG-1, MPEG-2, MPEG-4, H.264, AVS etc.) has reflected the development trend of Video Applications to high compression rate and networking.Major function based on the prediction of motion compensation is to eliminate video sequence redundancy in time, and most of code efficiency of video coding comes from prediction module.Video coding process is exactly the process that each two field picture of video sequence is encoded.Therefore, reference frame in the forecasting process of motion compensation in occupation of important effect.
The number of patent application of inventor application is that 200710071114.5 " utilizing the method for video coding of dynamic background frames " discloses a kind of method for video coding that utilizes dynamic background frames as the reference frame, this method has preferable performance for noisy background image district, but at noisy background image district, must use with traditional reference frame to obtain overall preferably coding efficiency because of only.And the present invention provides a kind of synthetic and using method of the better reference frame of estimated performance that does not rely on Noise Estimation, has not only considered the background area image encoding, has also considered the behaviour area image encoding, thereby the inventive method is used more flexible.
Summary of the invention
At the application in the video monitoring, the noise in the video sequence image is often bigger, and the interference that these noises bring will make that inter prediction becomes inaccurate in the cataloged procedure, and it is big that the error between the match block becomes, and makes video data effectively not compressed.The present invention proposes a kind of method for video coding and device and video encoding/decoding method and device that utilizes synthesized reference frame, it can improve the accuracy of the video sequence estimation in the video monitoring, reduce predicated error, improve code efficiency, and complexity does not increase.
When MB of prediction frame is carried out encoding and decoding, traditional algorithm with the reconstruction frames of former I frame (infra-frame prediction frame) or P frame (forward predicted frame) as the reference frame, current image frame is predicted encoding and decoding, and the reference block that promptly search and current coding macro block or its sub-piece mate most on reference frame is as the prediction to current macro.The present invention's I frame and dynamic synthesized reference frame of P frame of having encoded, synthesized reference frame and conventional video code decode algorithm are corresponding as the reconstruction frames of the I frame of reference frame and P frame, the reconstruction frames that is each I frame and P frame can have corresponding synthesized reference frame, so the reconstruction frames that an available synthesized reference frame replaces former I frame or P frame is predicted encoding and decoding as the reference frame to present frame.The also available a plurality of reference frames of traditional video coding-decoding method are predicted, the present invention replaces with one or more synthesized reference frames or part replaces the reconstruction frames of former I frame or P frame as the reference frame, also being used in increases the reference frame of synthesized reference frame for adding on the former reference frame basis, thereby has improved precision of prediction.
Especially, when bi-directional predicted encoding and decoding, traditional coding and decoding video algorithm is predicted current encoding and decoding picture frame with two reference frames in front and back.Because synthesized reference frame utilizes the I frame rebuild to generate, or the reconstruction macro block that utilizes the P frame and existing synthesized reference frame or corresponding prediction piece synthesize (renewal), and promptly synthesized reference frame is with corresponding to the I frame or the P frame of its renewal.Therefore, synthesized reference frame can substitute the former bi-directional predicted reference frame (reconstruction frames of I frame or P frame) that is used for, and current encoding and decoding macro block is predicted.
The present invention proposes utilizes synthesized reference frame (synthesized reference frame, abbreviate the SR frame as) video coding-decoding method, the dynamic generation and the update method that have mainly comprised synthesized reference frame, and adopt synthesized reference frame to do technology such as inter prediction reference frame.Particularly, the method for video coding that utilizes synthesized reference frame of the present invention, it utilizes image encoded information as the reference frame follow-up image encoded frame to be carried out predictive coding, it is characterized in that described reference frame comprises synthesized reference frame, this synthesized reference frame is reset by the reconstruction frames of each intracoded frame and is upgraded, and the reconstructed image information of quilt MB of prediction frame is subsequently pressed macro block or its sub-piece upgrades, synthesized reference frame after the renewal is used as reference frame when follow-up picture frame coding predicts, and is continued to upgrade.
The present invention utilizes in the synthesized reference frame method for video coding, and a kind of process that dynamically updates by macro block of synthesized reference frame is as follows:
1) if current encoded frame is the I frame, then the synthesized reference frame content update is the coding and rebuilding frame of present frame;
2), each macro block in the synthesized reference frame is upgraded respectively in the following way if current encoded frame is the P frame:
(a) if current macro is the I macroblock coding, then replace with current reconstruction macro block with the corresponding synthesized reference frame macro block of current macro;
(b) if current macro has adopted forward prediction, will be weighted average renewal with a update coefficients respectively with its initial value and current reconstruction macro block with the corresponding synthesized reference frame macro block of current macro;
, present encoding represents that the pixel value of the existing synthesized reference frame macro block of correspondence is Sr during the coding current macro if rebuilding the pixel value of macro block with F
Pre, the pixel value that upgrades back synthesized reference frame macro block with current coding macro block is Sr, then
Sr＝(1-α)Sr
pre+αF (1)
Wherein α is different for different image-regions, should get smaller value as stagnant zone, and region of variation should be got higher value.Estimation is accurate, and residual error is hour desirable smaller value, otherwise gets higher value.
3) if present frame is a bi-directional predicted frames B frame, synthesized reference frame does not upgrade.
4) synthesized reference frame after upgrading is carried out block elimination filtering.
Another method of upgrading synthesized reference frame of the present invention is to be that unit upgrades with the sub-piece in the macro block, and the synthesized reference frame after upgrading is carried out block elimination filtering, and renewal process is as follows:
1) if present frame is the I frame, then the synthesized reference frame content update is the reconstruction frames of present frame;
2), the sub-piece difference of each macro block in the synthesized reference frame is upgraded in the following way if present frame is the P frame:
(a) if current sub-block is the subblock coding in the I macro block, or the sub-piece of intraframe coding, then replace with the sub-piece of current reconstruction with the sub-piece of the corresponding synthesized reference frame of current sub-block;
(b) if current sub-block has adopted forward prediction, according to the residual error situation after the optimum movement vector that obtains after the current sub-block estimation, used reference frame and the quantification, to upgrade with corresponding update coefficients with predicted value and the current reconstruction sub-piece of the sub-piece of the corresponding synthesized reference frame of current sub-block with current sub-block, more new formula is as follows:
Sr＝(1-α)P+αF (2)
Wherein Sr is the pixel value of the SR frame piece of current sub-block correspondence, P is the predicted value of current sub-block, F is the pixel value of the sub-piece of current reconstruction, and α is the update coefficients of regulating according to the residual error situation self adaptation after the selecting for use of current sub-block reference frame, optimum movement vector and the quantification;
(c) otherwise the sub-piece of the corresponding synthesized reference frame of current sub-block do not upgrade;
3) if present frame is a bi-directional predicted frames B frame, synthesized reference frame does not upgrade.
Because reconstructed image F can be expressed as F=P+Q (r) (3)
Wherein Q (r) is the value of residual error behind quantification and inverse quantization, and substitution (2) formula gets:
Sr＝P+α×Q(r) (4)
As seen, the renewal process of this synthesized reference frame is similar to image reconstruction process, and different is the correction of the residual error behind the inverse quantization having been carried out multiply by α.
Update coefficients α is determined by following Several Factors in the above-mentioned renewal process: the reference frame type that current macro or sub-piece are used; Whether optimum movement vector length is less than a threshold value; Whether the residual error average energy after current sub-block quantizes is less than a threshold value.
If current sub-block adopts SR frame frame for referencial use, because of the SR frame has passed through frame filter, available big weights, update coefficients α should get smaller value; Motion vector is more little, illustrates it might is stagnant zone more, and update coefficients α should get smaller value, otherwise gets higher value; The residual error average energy is more little, illustrates that prediction is correct more, and promptly current sub-block and reference picture mate more, and update coefficients α should get smaller value, otherwise gets higher value.
Update coefficients α also can be the increasing function of the residual error average energy after current sub-block quantizes in the above-mentioned renewal process, and this function is predetermined nonlinear function or piecewise linear function.
Residual error average energy described here is meant the absolute value after the current sub-block prediction residual quantizes or the mean value of square value.During to noisy video sequence coding, noise causes the matching error in the motion search process easily, and residual energy is increased, therefore, if sub-piece residual error average energy is less, update coefficients should be got less real number, makes the SR frame piece after the renewal leading by the predicted value of current sub-block.
Stagnant zone on image causes owing to change mainly by noise or video camera slight jitter, can adopt less update coefficients α in (1) and (2).Adopt the constant stagnant zone of stagnant zone detection technique detected image content, in this zone, carry out stronger smothing filtering, help reducing The noise, but influence the definition of image hardly.
If macro block is divided into a few height pieces and predicts respectively, then macroblock motion vector is meant the average of each minute block motion vector.
The code stream that utilizes the synthesized reference frame video coding to form must employing utilize the decoding of synthesized reference frame method for video coding equally.Utilize in the synthesized reference frame video encoding/decoding method, the generation of synthesized reference frame and renewal process be and utilize the synthesized reference frame method for video coding identical, and it utilizes that the picture frame of decoding and rebuilding upgrades.
The video coding apparatus that utilizes above-mentioned synthesized reference frame technique among the present invention is characterized in that it comprises the generation module of synthesized reference frame.The generation module of described synthesized reference frame comprises synthesized reference frame synthesis unit, synthesized reference frame storage and filter unit, wherein, synthesized reference frame storage and filter unit comprise de-blocking filter and frame memory SRM0 and SRM1, wherein SRM0 is used to upgrade synthesized reference frame, filter unit carries out being stored among the SRM1 after the filtering to the synthesized reference frame among the SRM0, be that what deposit among the SRM1 is synthesized reference frame after the filtering, abbreviate the filtering synthesized reference frame as, directly as the reference frame that carries out inter prediction.
When a plurality of synthesized reference frames are arranged as the reference frame, can comprise a plurality of de-blocking filters and frame memory in the generation module of synthesized reference frame, a corresponding at least frame memory and the corresponding de-blocking filter that is used to deposit synthesized reference frame after the filtering of each synthesized reference frame.If a plurality of synthesized reference frames adopt identical method to upgrade, and are corresponding different images frame, then only need, or be equivalent to adopt straight-through filter with a de-blocking filter, deposit with frame and duplicate or frame is deposited ordering realization reference frame management.
The video decoder that utilizes synthesized reference frame among the present invention is characterized in that it comprises the generation module of synthesized reference frame.The generation module of described synthesized reference frame comprises synthesized reference frame synthesis unit, synthesized reference frame storage and filter unit, wherein, synthesized reference frame storage and filter unit comprise de-blocking filter and frame memory SRM0 and SRM1, wherein SRM0 is used to upgrade synthesized reference frame, filter unit carries out being stored among the SRM1 after the filtering to the synthesized reference frame among the SRM0, be that what deposit among the SRM1 is synthesized reference frame after the filtering, abbreviate the filtering synthesized reference frame as, directly as the reference frame that carries out inter prediction.
When a plurality of synthesized reference frames are arranged as the reference frame, can comprise a plurality of de-blocking filters and frame memory in the generation module of synthesized reference frame, a corresponding at least frame memory and the corresponding de-blocking filter that is used to deposit synthesized reference frame after the filtering of each synthesized reference frame.If a plurality of synthesized reference frames adopt identical method to upgrade, and are corresponding different images frame, then only need, or be equivalent to adopt straight-through filter with a de-blocking filter, deposit with frame and duplicate or frame is deposited ordering realization reference frame management.
By video coding-decoding method of the present invention as seen, after the encoding and decoding of I frame were finished, synthesized reference frame was reset by the I frame and is upgraded, thereby does not change the random access point of code stream.Simultaneously, the synthesized reference frame of I frame correspondence is identical with the reconstruction frames of this I frame, is equivalent with this synthesized reference frame with carrying out inter prediction with the reconstruction frames of I frame therefore.
The invention has the beneficial effects as follows: in the present invention, synthesized reference frame is synthetic in video encoding-decoding process, upgrade the pixel value of synthesized reference frame owing to adopt the method for weighting, carried out the self adaptation frame filter, thereby noise had good inhibitory effect, with it as the reference frame in the encoding-decoding process, can improve the accuracy of estimation, reduce predicated error, improved code efficiency, improved subjective quality simultaneously.When running into the encoding and decoding of I macro block, I is rebuild macro block directly duplicate synthesized reference frame macro block to correspondence, can protect the picture material of big motion preferably.Therefore,, still have goodish compression effectiveness, compare complexity with multi-reference frame and significantly reduce even synthesized reference frame is used as unique reference frame.
The decipher reestablishment image that decoding output image of the present invention can be a present frame or the SR two field picture of present frame.When exporting,, the decipher reestablishment image based on the prediction of SR frame there is indirect filter action because the SR frame has the frame filter effect with decipher reestablishment image.Also can adopt the SR frame as output image, at this moment stronger filter action is arranged, can improve the image subjective effect.
In order to improve flexibility, the undated parameter α of record or transmission synthesized reference frame in can the sequence head that each random access point begins in video sequence or in image head or other syntactic element, the optimum movement vector length threshold, residual error average energy threshold value, information such as predetermined nonlinear function or piecewise linear function parameter.
Synthesized reference frame among the present invention can be rebuild in Video Decoder and generate and dynamically update, thereby do not need to transmit excessive data, can compatiblely have canonical algorithm as long as in sequence head, transmit undated parameter and dependent thresholds or the reference frame sign of macro block is done expansion.
Description of drawings
Fig. 1 is the method for video coding general flow chart that utilizes the SR frame of the present invention;
Fig. 2 is macroblock coding of the present invention and synthesized reference frame macro block renewal flow chart;
Fig. 3 is macroblock coding of the present invention and the sub-piece renewal of synthesized reference frame flow chart;
Fig. 4 is a kind of video coding apparatus structural representation that utilizes the SR frame of the present invention;
Fig. 5 utilizes the video coding apparatus structural representation of SR frame for another kind of the present invention;
Fig. 6 is a kind of video decoder structural representation that utilizes the SR frame of the present invention;
Fig. 7 is the video encoding/decoding method general flow chart that utilizes the SR frame of the present invention;
Fig. 8 is macro block decoding of the present invention and synthesized reference frame macro block renewal flow chart;
Fig. 9 utilizes the video decoder structural representation of SR frame for another kind of the present invention;
Table 1 and table 2 compare for the method for video coding of synthesized reference frame and the traditional coding method performance utilized of the present invention.
Embodiment
Embodiment 1: a kind of method for video coding and device thereof that utilizes synthesized reference frame
The method for video coding that utilizes synthesized reference frame that the present invention proposes 1,2,4 is described in detail as follows in conjunction with the accompanying drawings:
1) coding need be deposited SRM0 and SRM1 for synthesized reference frame distributes two frames at first.Frame is deposited the size that size is frame data.Wherein SRM0 is used to carry out the renewal of synthesized reference frame, deposits the synthesized reference frame without block elimination filtering, and each synthesized reference frame can a multiplexing SRM0; SRM1 is used to deposit the synthesized reference frame behind the block elimination filtering, can be directly used in inter prediction.
2) read in a two field picture, if current encoded frame is the I frame, then the synthesized reference frame frame is deposited SRM0 and is updated to the reconstruction frames of present frame without block elimination filtering.Go to step 5).
3) if current encoded frame is forward predicted frame (a P frame), read in a macro block of this P frame:
(a) if current macro is the I macroblock coding, the macro block of depositing in the SRM0 with the corresponding frame of current macro after then the current macro intraframe coding is finished is replaced with current reconstruction macro block, be Sr=F, wherein F represents current reconstruction macro block pixels value, Sr, the expression current coding macro block is upgraded back synthesized reference frame macro block pixels value.
(b) if current macro has adopted forward predictive coded, when current macro is non-skip pattern: if current macro selects for use synthesized reference frame SRM1 to carry out estimation and motion vector less than a certain threshold value TH1, the macro block of depositing in the SRM0 with the corresponding frame of current macro after then the current macro coding is finished upgrades by a kind of weighted average with its original pixel value and current reconstruction macro block, i.e. Sr=(1-α) Sr
Pre+ α F, wherein α is a update coefficients and 0＜α＜1; Otherwise another update coefficients of α.Sr wherein
PreRepresent the existing synthesized reference frame macro block pixels value corresponding with current coding macro block.
When current macro was the skip pattern: the macro block pixels value of depositing in the SRM0 with the corresponding frame of current macro after then the current macro coding is finished was not upgraded.
Encoded all macro blocks of current P frame and corresponding frame deposit macro block in the SRM0 upgrade finish after, go to step 5).
4), then utilize synthesized reference frame forward direction SRM1 and back they to be carried out predictive coding to SRM1 to the B frame if present frame is bi-directional predicted frames (a B frame).In the B intra-frame macro block cataloged procedure, the macro block that corresponding frame is deposited in the SRM0 does not upgrade.After the end-of-encode of B frame, go to step 6).
5) after by the time present frame (I frame, P frame) coding is finished, frame is deposited SRM0 carry out block elimination filtering, and deposit the content behind the SRM0 block elimination filtering in frame and deposit SRM1, be used as the reference frame of next frame.
6) repeat 2)-5) finish the coding of video sequence.
As shown in Figure 3, the video coding apparatus that utilizes synthesized reference frame of the present invention, it has estimation, and motion compensation and infra-frame prediction, transform and quantization, entropy coding, inverse transformation and inverse quantization, block elimination filtering, reconstruction frames are deposited and each unit of synthesized reference frame generation module.Described synthesized reference frame generation module comprises SR frame synthesis unit, preceding reference frame memory SRM0 and SRM1, back reference frame memory SRM0 and SRM1, and filter unit, wherein SRM0 is used to upgrade synthesized reference frame, and SRM1 is used to deposit is used to the synthesized reference frame (being the filtering synthesized reference frame) predicted behind block elimination filtering.
Embodiment 2: another kind utilizes the method for video coding and the device thereof of synthesized reference frame
A kind of method for video coding that utilizes synthesized reference frame 1,3,5 is described in detail as follows in conjunction with the accompanying drawings:
1) coding need be deposited the reference frame heap of formation for synthesized reference frame distributes n frame at first, and frame poke amount can be decided with the need, and a plurality of frames are deposited and are used to support the multi-reference frame prediction, and establish a SR frame buffer SRM0.Frame is deposited the size that size is frame data.
2) read in a two field picture,, then it is carried out traditional intraframe coding, and be the reconstruction frames of present frame without block elimination filtering with the content update among the synthesized reference frame buffer SRM0 if current encoded frame is the I frame.Go to step 5).
3) if current encoded frame is forward predicted frame (a P frame), read in a macro block of this P frame:
(a) if current macro is the I macroblock coding, after then the current macro intraframe coding is finished with the corresponding SR frame buffer of current macro SRM0 in macro block replace with current reconstruction macro block, i.e. Sr=F, wherein F represents current reconstruction macro block, Sr, macro block among the corresponding SRM0 of expression current coding macro block.
(b) if current macro has adopted forward predictive coded, when current macro is non-skip pattern, the SRM0 buffer memory upgrades by the sub-piece in the macro block: if the current sub-block motion vector is less than threshold value TH1, and current sub-block prediction residual average energy is less than threshold value TH2, then current sub-block coding is finished the interior sub-piece of back and the corresponding SRM0 of current sub-block and is used predicted value and the sub-piece of current reconstruction to current sub-block to upgrade by a kind of weighted average, i.e. Sr=(1-α
1) P+ α
1F, wherein α
1Be a update coefficients, 0＜α
1＜1; Otherwise if current sub-block prediction residual average energy less than threshold value TH3, is then used another update coefficients α
2Pressing the same manner upgrades; Otherwise with another update coefficients α
3Pressing the same manner upgrades.Wherein P represents the best predictor to the current sub-block coding.
When current macro was the skip pattern: then current macro coding was finished back and the corresponding SRM0 frame of the current macro macro block in depositing and is not upgraded.
All macro blocks of current P frame and the corresponding SRM0 frame macro block in depositing of having encoded goes to step 5) after upgrading and finishing.
4) if present frame is bi-directional predicted frames (a B frame), then adopt traditional coding method to utilize the synthesized reference frame of frame in depositing to carry out bi-directional predictive coding to present frame, the content in the synthesized reference frame frame is deposited is not upgraded.After the end-of-encode of B frame, go to step 6).
5) after present frame (I frame, P frame) coding was finished by the time, the image that frame is deposited among the SRM0 carried out block elimination filtering, and deposits the content behind the block elimination filtering in the synthesized reference frame heap, is used as the reference frame of next frame.
6) repeat 2)-5) finish the coding of video sequence.
As shown in Figure 5, a kind of video coding apparatus that utilizes synthesized reference frame of the present invention, it has estimation, motion compensation, infra-frame prediction, transform and quantization, entropy coding, inverse transformation and inverse quantization, SR frame synthetic and SRM0, block elimination filtering, reference frame is deposited and undated parameter is determined each unit.The residual error average energy that the optimum movement vector that described undated parameter determining unit obtains according to estimation, the residual computations behind the inverse quantization obtain is determined the undated parameter of current sub-block synthesized reference frame; The SR frame synthesizes and the SRM0 unit is the performance element of SR frame update process, carries out the SR frame update of pressing sub-piece with determined undated parameter, and the SR frame of the not block elimination filtering after the renewal deposits SRM0 in.Content among the SRM0 deposits the reference frame heap in behind block elimination filtering, as the reference frame of later picture frame coding.The work of other unit of video coding apparatus is identical with conventional codec.
Embodiment 3: a kind of video encoding/decoding method and device thereof that utilizes synthesized reference frame
The video encoding/decoding method that utilizes synthesized reference frame that the present invention proposes 6,7,8 is described in detail as follows in conjunction with the accompanying drawings:
It is identical that the generation of the synthesized reference frame of present embodiment and renewal process and embodiment 1 finish, and just wherein macro block reconstructed value and motion vector obtain by video decode, and be in full accord with coding side.
As shown in Figure 6, the present invention utilizes the video decoder of synthesized reference frame, and it has motion compensation and infra-frame prediction, entropy decoding, inverse transformation and inverse quantization, block elimination filtering, reconstruction frames are deposited and each unit of synthesized reference frame generation module.Described synthesized reference frame generation module comprises SR frame synthesis unit, preceding reference frame memory SRM0 and SRM1, back reference frame memory SRM0 and SRM1, and filter unit, wherein SRM0 is used to upgrade synthesized reference frame, and SRM1 is used to deposit is used to the synthesized reference frame (being the filtering synthesized reference frame) predicted behind block elimination filtering.
Embodiment 4: another kind utilizes the video encoding/decoding method and the device thereof of synthesized reference frame
A kind of method for video coding and device thereof that utilizes synthesized reference frame 9 is described as follows in conjunction with the accompanying drawings:
It is identical that the generation of the synthesized reference frame of present embodiment and renewal process and embodiment 2 finish, and just wherein macro block reconstructed value and motion vector obtain by video decode, and be in full accord with coding side.
As shown in Figure 9, the present invention utilizes the video decoder of synthesized reference frame, and it has motion compensation and infra-frame prediction, entropy decoding, inverse transformation and inverse quantization, SR frame block elimination filtering, reconstructed image block elimination filtering, reconstructed image frame are deposited, the SR frame is synthetic and SRM0, reference frame heap, undated parameter determine each unit.The residual error average energy that the described undated parameter determining unit sub-block motion vector that decoding obtains according to entropy, the residual computations behind the inverse transformation inverse quantization obtain is determined the undated parameter of current sub-block synthesized reference frame; The SR frame synthesizes and the SRM0 unit is the performance element of SR frame update process, carries out the SR frame update of pressing sub-piece with determined undated parameter, and the SR frame of the not block elimination filtering after the renewal deposits SRM0 in.Content among the SRM0 deposits the reference frame heap in behind block elimination filtering, as the reference frame of later picture frame decoding moving compensation.The work of other unit of video decoder is identical with conventional codec, and different is the reference frame that reconstructed image is not used in motion compensation.In this decoder, two kinds of image outputs can be arranged, a kind of is decipher reestablishment image, another kind is the SR two field picture.The SR two field picture is actual to be reconstructed image frame behind frame filter, can be used as actual output.If only adopt the output of SR two field picture, then reconstructed image goes piece and reconstructed image frame to deposit and can save.
Embodiment 5: a kind of video encoder that utilizes synthesized reference frame
Present embodiment is to be experiment porch with the source code rm52j_r1 at national video encoding standard AVS-P2.AVS-P2 has two reference frame ref0 and ref1 at most for MB of prediction frame.Wherein ref0 is from that nearer frame of present frame on the time domain.In the present embodiment, we remove traditional reference frame ref1, and the synthesized reference frame of introducing among the present invention replaces traditional reference frame ref1.Wherein the more new situation of synthesized reference frame in encoding-decoding process all provides in last summary of the invention and embodiment 1 in detail.Need to prove that during coding P frame, if current macro is that forward prediction selects for use synthesized reference frame to carry out predictive coding, then the macro block of the synthesized reference frame corresponding with current macro upgrades with following coefficient weighting by its original pixel value and current reconstruction macro block:
Standard supervisory sequence substation.yuv to national video encoding standard AVS encodes by the described method of present embodiment, and statistics is as shown in table 1.Experiment shows, replaces in traditional reference frame in the AVS-P2 source code on the time domain when the present frame reference frame far away with synthesized reference frame, and coding efficiency is significantly increased.Make encoder bit rate obviously descend, and decoding back reconstructed image subjective quality also have clear improvement.
Embodiment 6: another kind utilizes the video encoder test of synthesized reference frame
Present embodiment is to be experiment porch with the source code at national video encoding standard AVS-P2.AVS-P2 has two reference frame ref0 and ref1 at most for MB of prediction frame.In the present embodiment, we remove two traditional frame reference frames, introduce the dynamically synthetic synthesized reference frame among the present invention, and promptly synthesized reference frame is only reference frame in encoding-decoding process.Wherein synthesized reference frame in encoding-decoding process the superincumbent summary of the invention of more new situation and embodiment 1 in all provide in detail.Need to prove that during coding P frame, if current macro is forward prediction and is non-skip pattern, and motion vector is less than 1, then the macro block of the synthesized reference frame corresponding with current macro upgrades with following coefficient weighting by its original pixel value and current reconstruction macro block:
If current macro is a forward prediction and for non-skip pattern, and motion vector is not less than at 1 o'clock, and then the macro block of the synthesized reference frame corresponding with current macro upgrades with following coefficient weighting by its original pixel value and current reconstruction macro block:
Wherein F represents current encoding and decoding macro block value, and Sr represents that current encoding and decoding macro block upgrades back synthesized reference frame macro block pixels value, Sr
PreRepresent the original synthesized reference frame macro block pixels value corresponding with current encoding and decoding macro block.
Standard supervisory sequence substation.yuv to national video encoding standard AVS encodes with the described method of present embodiment, and statistics is as shown in table 2.Experiment shows, when replacing all traditional reference frame in the AVS-P2 source code with synthesized reference frame, coding efficiency (PSNR and code check) ratio is significantly increased when encoding with a traditional frame reference frame, performance is suitable with traditional two frame reference frames coding the time, even better than it, obviously improve with the code check subjective quality.
At last, also it is pointed out that and the invention is not restricted to above-mentioned execution mode.All distortion that those of ordinary skill in the art can directly derive or associate from content disclosed by the invention all should be thought protection scope of the present invention.Table 1: the method for video coding of synthesized reference frame and the traditional coding method performance utilized of the present invention compares
Table 2: the method for video coding of synthesized reference frame and the traditional coding method performance utilized of the present invention compares
Claims (4)
1. video coding-decoding method that utilizes synthesized reference frame, it utilizes image encoded information as the reference frame follow-up image encoded frame to be carried out predictive coding and decoding, it is characterized in that: described reference frame comprises synthesized reference frame, this synthesized reference frame is reset by the reconstruction frames of each intracoded frame and is upgraded, and the reconstructed image information of quilt MB of prediction frame is subsequently pressed macro block or its sub-piece upgrades, synthesized reference frame after the renewal is used as reference frame when follow-up picture frame coding predicts, and is continued to upgrade;
The renewal process that described synthesized reference frame is pressed macro block is as follows:
1) if current encoded frame is the I frame, then the synthesized reference frame content update is the coding and rebuilding frame of present frame;
2), each macro block in the synthesized reference frame is upgraded respectively in the following way if current encoded frame is the P frame:
(a) if current macro is the I macroblock coding, then replace with current reconstruction macro block value with the corresponding synthesized reference frame macro block of current macro;
(b) if current macro has adopted forward prediction, will be weighted average renewal with its initial value and current reconstruction macro block value with a update coefficients with the corresponding synthesized reference frame macro block of current macro;
(c) otherwise, the corresponding synthesized reference frame macro block of current macro does not upgrade;
3) if present frame is a bi-directional predicted frames B frame, synthesized reference frame does not upgrade;
4) synthesized reference frame after upgrading is carried out block elimination filtering;
Described synthesized reference frame is that the process upgraded of unit is as follows with the sub-piece in the macro block:
1) if present frame is the I frame, then the synthesized reference frame content update is the reconstruction frames of present frame;
2), the sub-piece difference of each macro block in the synthesized reference frame is upgraded in the following way if present frame is the P frame:
(a) if current sub-block is the subblock coding in the I macro block, or the sub-piece of intraframe coding, then replace with the sub-piece of current reconstruction with the sub-piece of the corresponding synthesized reference frame of current sub-block;
(b) if current sub-block has adopted forward prediction, according to the residual error situation after the optimum movement vector that obtains after the current sub-block estimation, used reference frame and the quantification, will upgrade with a update coefficients with the predicted value and the sub-piece of current reconstruction of the sub-piece of the corresponding synthesized reference frame of current sub-block with current sub-block;
(c) otherwise the sub-piece of the corresponding synthesized reference frame of current sub-block do not upgrade;
3) if present frame is a bi-directional predicted frames B frame, synthesized reference frame does not upgrade;
4) synthesized reference frame after upgrading is carried out block elimination filtering.
2. synthesized reference frame update method as claimed in claim 1 is characterized in that, update coefficients is determined by following three factors: the reference frame type that current macro or sub-piece are used; Whether optimum movement vector length is less than a threshold value; Whether the residual error average energy after current sub-block quantizes is less than a threshold value.
3. synthesized reference frame update method as claimed in claim 1 is characterized in that, update coefficients is the increasing function of the residual error average energy after current macro or sub-piece quantize, and this function is predetermined nonlinear function or piecewise linear function.
4. synthesized reference frame update method as claimed in claim 1 is characterized in that, it also carries out the stagnant zone detection, and stagnant zone is used for determining the update coefficients of synthesized reference frame.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200810087379 CN101272494B (en) | 2008-01-25 | 2008-03-21 | Video encoding/decoding method and device using synthesized reference frame |
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200810059361.8 | 2008-01-25 | ||
| CN200810059361 | 2008-01-25 | ||
| CN 200810087379 CN101272494B (en) | 2008-01-25 | 2008-03-21 | Video encoding/decoding method and device using synthesized reference frame |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101272494A CN101272494A (en) | 2008-09-24 |
| CN101272494B true CN101272494B (en) | 2011-06-08 |
Family
ID=40006141
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200810087379 Active CN101272494B (en) | 2008-01-25 | 2008-03-21 | Video encoding/decoding method and device using synthesized reference frame |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101272494B (en) |
Families Citing this family (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9854254B2 (en) * | 2009-08-04 | 2017-12-26 | Avocent Corporation | Method and system for remote viewing of static and video images |
| CN102196258B (en) * | 2010-03-11 | 2013-03-27 | 中国科学院微电子研究所 | I frame encoding method and device |
| CN102523447A (en) * | 2011-12-27 | 2012-06-27 | 李宗霖 | Transferring and exchanging method of media stream |
| US20130343459A1 (en) * | 2012-06-22 | 2013-12-26 | Nokia Corporation | Method and apparatus for video coding |
| CN108337525B (en) | 2012-09-28 | 2020-08-25 | 杜比国际公司 | Image decoding device |
| CN103905825B (en) * | 2012-12-27 | 2018-02-13 | 中国电信股份有限公司 | The motion compensation process and device of video image |
| WO2014120368A1 (en) * | 2013-01-30 | 2014-08-07 | Intel Corporation | Content adaptive entropy coding for next generation video |
| WO2015010317A1 (en) * | 2013-07-26 | 2015-01-29 | 北京大学深圳研究生院 | P frame-based multi-hypothesis motion compensation method |
| CN104717511B (en) * | 2013-12-13 | 2018-10-16 | 联发科技（新加坡）私人有限公司 | Video coding or decoded method and device |
| WO2015085575A1 (en) * | 2013-12-13 | 2015-06-18 | Mediatek Singapore Pte. Ltd. | Methods for background residual prediction |
| CN106331700B (en) | 2015-07-03 | 2019-07-19 | 华为技术有限公司 | Method, encoding device and the decoding device of reference picture coding and decoding |
| CN105898328A (en) * | 2015-12-14 | 2016-08-24 | 乐视云计算有限公司 | Self-reference coding included setting method and device for reference frame set |
| CN108259913A (en) * | 2016-12-29 | 2018-07-06 | 北京大学深圳研究生院 | A kind of intra-frame prediction method in MB of prediction frame |
| CN110235446B (en) * | 2017-06-08 | 2024-04-12 | 深圳市大疆创新科技有限公司 | Video encoding method, video decoding method and related devices |
| WO2019169640A1 (en) * | 2018-03-09 | 2019-09-12 | SZ DJI Technology Co., Ltd. | System and method for supporting video coding based on fast feedback |
| WO2020006690A1 (en) * | 2018-07-03 | 2020-01-09 | 深圳市大疆创新科技有限公司 | Video processing method and device |
| EP3796652B1 (en) | 2019-09-23 | 2022-05-18 | Axis AB | Video encoding method and method for reducing file size of encoded video |
| CN110636295B (en) * | 2019-09-27 | 2023-10-24 | 腾讯科技（深圳）有限公司 | Video encoding and decoding method and device, storage medium and electronic device |
| CN113573076A (en) * | 2020-04-29 | 2021-10-29 | 华为技术有限公司 | Method and apparatus for video encoding |
-
2008
- 2008-03-21 CN CN 200810087379 patent/CN101272494B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN101272494A (en) | 2008-09-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101272494B (en) | Video encoding/decoding method and device using synthesized reference frame | |
| CN102724498B (en) | The coding method of the scalability techniques of content-based information and equipment | |
| CN101273638B (en) | Image processing device, image processing method | |
| CN101313592B (en) | Methods and device for data alignment with time domain boundary | |
| CN100555332C (en) | Use comprises that the prediction of a plurality of macro blocks and nonanticipating picture are to picture sequence Methods for Coding and device | |
| KR101356207B1 (en) | / Method and apparatus for encoding /decoding data | |
| CN100512443C (en) | Distributive vide frequency coding method based on self adaptive Hashenhege type vector quantization | |
| CN104994396A (en) | Apparatus for decoding video | |
| KR20060109249A (en) | Method and apparatus for predicting motion vector of a macro block using motion vectors of a picture of base layer when encoding/decoding video signal | |
| KR20050053470A (en) | Method for scalable video coding and decoding, and apparatus for the same | |
| CN102986211A (en) | Rate control in video coding | |
| CN102857746A (en) | Method and device for coding and decoding loop filters | |
| CN101523915B (en) | Two pass rate control techniques for video coding using a min-max approach | |
| KR101375667B1 (en) | Method and apparatus for Video encoding and decoding | |
| CN101411200B (en) | Method of video signal coding | |
| CN100493191C (en) | Video encoder with low complexity noise reduction and video encoding method | |
| CN117480778A (en) | Residual coding and video coding methods, devices, equipment and systems | |
| KR20060070400A (en) | Method for encoding and decoding video signal | |
| JP5016561B2 (en) | Moving picture encoding apparatus, moving picture decoding apparatus, moving picture encoding method, and moving picture decoding method | |
| JP2005252609A (en) | Data processor, its method and coder | |
| CN103002284B (en) | A kind of video coding-decoding method based on model of place adaptive updates | |
| JP3149673B2 (en) | Video encoding device, video encoding method, video reproducing device, and optical disc | |
| JP3586069B2 (en) | Moving picture coding method and apparatus | |
| KR20060063604A (en) | Method for encoding and decoding video signal | |
| KR0166728B1 (en) | The image signal encoding apparatus for edge characteristics and the quantization level decision method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| ASS | Succession or assignment of patent right |
Owner name: SICHUAN HUAYAN INFORMATION INDUSTRY CO., LTD. Free format text: FORMER OWNER: ZHEJIANG UNIVERSITY Effective date: 20120703 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| COR | Change of bibliographic data |
Free format text: CORRECT: ADDRESS; FROM: 310027 HANGZHOU, ZHEJIANG PROVINCE TO: 610041 CHENGDU, SICHUAN PROVINCE |
|
| TR01 | Transfer of patent right |
Effective date of registration: 20120703 Address after: 610041. 16 floor, building 10, C District, Tianfu Software Park, Sichuan, Chengdu Patentee after: Sichuan Whayer Information Industry Co., Ltd. Address before: 310027 Department of information and electrical engineering, Zhejiang University, Yuquan campus, 38 Zhejiang Road, Zhejiang, Hangzhou Patentee before: Zhejiang University |
|
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: 610094 16 / F, building 10, Tianfu Software Park, no.219, Tianhua Second Road, hi tech Zone, Chengdu, Sichuan Province Patentee after: Huayan Intelligent Technology (Group) Co., Ltd Address before: 610041. 16 floor, building 10, C District, Tianfu Software Park, Sichuan, Chengdu Patentee before: Sichuan Whayer Information Industry Co., Ltd. |