CN101389034B - Image encoding/decoding method, apparatus and an image processing method, system - Google Patents
Image encoding/decoding method, apparatus and an image processing method, system Download PDFInfo
- Publication number
- CN101389034B CN101389034B CN 200710153927 CN200710153927A CN101389034B CN 101389034 B CN101389034 B CN 101389034B CN 200710153927 CN200710153927 CN 200710153927 CN 200710153927 A CN200710153927 A CN 200710153927A CN 101389034 B CN101389034 B CN 101389034B
- Authority
- CN
- China
- Prior art keywords
- image
- random access
- access point
- picture
- reconstructed image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
This invention discloses an image coding/decoding method and device for further improving the image coding efficiency and improving the flexibility of random access on the premises of guaranteeing the random access performance. The method comprises: hiding errors of a reference image in the front of a random access point and decoding a bidirectional prediction coding image behind the random access point so as to obtain a re-constructed image, wherein the reference image is the direct or indirect reference image of the bidirectional prediction coding image; or hiding errors of the bidirectional prediction coding image behind the random access point to obtain the re-constructed image; referencing a first intra-frame coding image behind the random access point and the re-constructed image; coding at least one forward prediction coding image behind the random access point. This invention further provides an image processing method and system. This invention is used for coding and decoding.
Description
Technical field
The present invention relates to technical field of image processing, relate in particular to a kind of image coding/decoding method, device and a kind of image processing method, system.
Background technology
Video image compression coding is a current very active research field.In the middle of in the past nearly 20 years, technology of video compressing encoding is constantly developed, and new video compression coding standard also continues to bring out.The Moving Picture Experts Group-1 organized to set up of MPEG (Moving Picture Experts Group) in 1991 is used towards the VCD stored CD, has obtained great success on market; MPEG in 1994 and international telegraph union (ITU, International Telegraph Union) are united the Moving Picture Experts Group-2 of formulation, towards the application of digital television broadcasting and DVD videodisc.This standard be most widely used in digital video broadcasting and videodisc field at present, the most ripe, video compression standard that influence is the most far-reaching.MPEG has released OO video compression coding standard MPEG-4 of new generation afterwards, ITU released towards the standard of video conference, video communication H.263 and its later release H.263+, H.263++, H.263L.Up-to-date video compression coding standard mainly contains the H.264/AVC standard that ITU/MPEG unites formulation at present, and VC-1 standard, the former is international standard in March, 2005 by the promulgation of ISO/IEC/ITU normal structure, and the latter was issued by the SMPTE normal structure in April, 2006.The development trend of technology of video compressing encoding is: pursue higher encoding compression efficient, better network compatibility, better user experience and application widely.
In technology of video compressing encoding, relate to in-frame encoding picture, inter coded images and figure group notions such as (GOP, Group of Picture).In-frame encoding picture can be finished coding by image itself, does not need other images for referencial use.In-frame encoding picture can utilize infra-prediction techniques to encode.For example, I image (I frame) is exactly a kind of in-frame encoding picture.Inter coded images is to utilize the inter prediction technology to carry out image encoded, need carry out predictive coding to this image according to reference picture.Inter coded images has two types: forward predictive coded image and bidirectionally predictive coded picture.The forward predictive coded image, for example P image (P frame) can only carry out predictive coding with reference to the image that occurs previously; Bidirectionally predictive coded picture, for example B image (B frame) can carry out predictive coding with reference to forward direction and the image that the back occurs on both direction, and under special circumstances, bidirectionally predictive coded picture also can only carry out predictive coding with reference to the back to image.Be called reference picture by inter coded images with image for referencial use.Inter coded images needs reference picture just can carry out inter prediction encoding, equally in decoding end the decoding of inter coded images is also needed reference picture.The decoded picture of predictive-coded picture (P image) can be used as reference picture between the decoded picture of intraframe predictive coding image (I image) and forward frame, but, the decoded picture of bidirectionally predictive coded picture (B image) cannot be used as reference picture, and promptly bidirectionally predictive coded picture is non-reference picture.It also can be a plurality of that the frame number of reference picture can be one.Utilize multiple image to be the multi-reference frame Predicting Technique as the image coding technique of reference image.
Figure group (GOP, Group Of Pictures) is the combination of one or more coded images, is made up of an in-frame encoding picture and a plurality of inter coded images of following after this image.
In technology of video compressing encoding, relate to the problem that puts in order of various coded images, comprise coded sequence and DISPLAY ORDER.If there is not the B frame in the video sequence, then coded sequence is identical with DISPLAY ORDER, because decoding order is identical with coded sequence.If comprise the B frame in the video sequence, then coded sequence is different with DISPLAY ORDER, should carry out image before decoded picture output shows and reorder.
Illustrating image below reorders:
The DISPLAY ORDER of image is referring to table one:
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 |
| I | B | B | P | B | B | P | B | B | I | B | B | P |
From DISPLAY ORDER, two B frames are arranged between I frame and the P frame, between two continuous P frames two B frames are arranged also.In when coding, with image 1I predicted picture 4P, with image 4P and 1I predicted picture 2B and 3B, coded sequence is referring to table two:
| 1 | 4 | 2 | 3 | 7 | 5 | 6 | 10 | 8 | 9 | 13 | 11 | 12 |
| I | P | B | B | P | B | B | I | B | B | P | B | B |
Equally, decoding order also as shown in Table 2, so, in when output decoding, need be adjusted into the DISPLAY ORDER shown in the table one.
Carry out program at needs and change under the situation such as random position, fast forwarding and fast rewinding of platform, code stream switching, editor and splicing, programme replay, relate to random access.Random access is meant a certain random access point (RAP, the Random Access Point) beginning outside the video bit stream starting point, to bit stream decoding and recover decoded picture.Random access can be divided into two kinds, and a kind of is random access immediately, begins just can be correctly decoded from RAP; Another kind is a random access gradually, begins to need a process to being correctly decoded from RAP.Different business is to the requirement difference of random access performance, such as, for broadcasting service, European digital television standard (DVB, Digital Video Broadcasting) regulation a RAP will occur every 0.5s; Video communication, video conference, PPV (Pay Per View) etc. are professional lower to the requirement of random access performance.The random access performance is directly relevant with user's experience.
Under the situation that adopts the multi-reference frame Predicting Technique, because the P frame can stride across the I frame with reference to the image before the I frame, like this, when random access takes place in I frame (RAp), because the image non-availability before the I frame, and the P frame will be with reference to the image before the I frame, therefore, and the problem that will cause the P frame after the I frame to decode.
For addressing the above problem, existing way all is the prediction reference characteristic and the reference frame number of the inter prediction encoding image behind the restriction RAP.This is a cost to sacrifice code efficiency.
In standard H.264,, introduce instantaneous decoding refresh (IDR, Instantaneous Decoding Refresh) image for solving the random access problem under the multi-reference frame situation.The IDR image is a kind of new image type, is in-frame encoding picture, but is different from common in-frame encoding picture (I image).H.264 stipulate in the standard that the image after the IDR image does not use the image before the IDR for referencial use, therefore, from the IDR image, successive image all can be correctly decoded.First image of video sequence should be the IDR image, and the IDR image can be used as the random access point of a compressed bit stream.
But, in realizing process of the present invention, the inventor finds that there are the following problems at least in the prior art: because the image after the IDR image does not use IDR image before for referencial use, that is to say, image after the IDR image can't make full use of the multi-reference frame technology, therefore will cause code efficiency to reduce.Random access can only occur in the IDR border, and the IDR image is a kind of special I frame, that is to say, random access can not be with each I frame as cutting point, and promptly random access can not occur in the border of figure group.Therefore, this method is having bigger restriction aspect the flexibility of random access.
In digital audio/video encoding and decoding technique standard (AVS), also adopted the multi-reference frame technology, allow forward direction with reference to two frames.For solving the random access problem, give more implication to sequence head (Sequence header) in the standard.Stipulate in the standard that first P frame behind the sequence head behind first I frame is reference sequences head image before not.So, when the I frame that has sequence head was cut at random, successive image did not lack reference picture, all can be correctly decoded, thereby support random access.
In realizing process of the present invention, the inventor finds that there are the following problems at least in the prior art: identical with the effect of IDR image H.264, shortcoming is also identical substantially.Because the prediction reference characteristic of first P frame can't make full use of the multi-reference frame technology behind the I frame behind the restriction sequence head, has caused the reduction of code efficiency.The random access flexibility is not high, can only occur in the sequence head border, can not occur in the GOP border.In sequence inside, allow the P frame to stride across the frame of I frame with reference to the front, therefore, can't carry out random access.
In sum, random access and code efficiency are the relations of a pair of contradiction, and good random access performance is a cost to sacrifice code efficiency.Prior art is under the prerequisite that guarantees the random access performance, and it is not high to have reduced image coding efficiency, and makes the very flexible of random access.
Summary of the invention
The embodiment of the invention provides a kind of image coding/decoding method, device and a kind of image processing method, system, in order under the prerequisite that guarantees the random access performance, further improves image coding efficiency, and improves the flexibility of random access.
A kind of method for encoding images that the embodiment of the invention provides comprises:
Reference picture before the random access point is carried out error concealment to be handled, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image;
With reference to first in-frame encoding picture after the described random access point, and described reconstructed image, at least one the forward predictive coded image after the described random access point is encoded.
A kind of picture decoding method that the embodiment of the invention provides comprises:
Reference picture before the random access point is carried out error concealment to be handled, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image;
With reference to first in-frame encoding picture after the described random access point, and described reconstructed image, at least one the forward predictive coded image after the described random access point is decoded.
The image processing method that the embodiment of the invention provides comprises:
At coding side, reference picture before the random access point is carried out error concealment handle, and the bidirectionally predictive coded picture after the described random access point is carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image; With reference to first in-frame encoding picture after the described random access point, and described reconstructed image, at least one the forward predictive coded image after the described random access point is encoded;
In decoding end, reference picture before the described random access point is carried out error concealment to be handled, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image; With reference to described reconstructed image, and described in-frame encoding picture, described forward predictive coded image is decoded.
A kind of picture coding device that the embodiment of the invention provides comprises:
Coding refresh process unit, being used for that the reference picture before the random access point is carried out error concealment handles, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image;
Coding unit is used for reference to first in-frame encoding picture after the described random access point, and described reconstructed image, and at least one the forward predictive coded image after the described random access point is encoded.
A kind of picture decoding apparatus that the embodiment of the invention provides comprises:
The decoding refresh processing unit, being used for that the reference picture before the random access point is carried out error concealment handles, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image;
Decoding unit is used for reference to first in-frame encoding picture after the described random access point, and described reconstructed image, and at least one the forward predictive coded image after the described random access point is decoded.
A kind of image processing system that the embodiment of the invention provides comprises:
Picture coding device, being used for that the reference picture before the random access point is carried out error concealment handles, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image; With reference to first in-frame encoding picture after the described random access point, and described reconstructed image, at least one the forward predictive coded image after the described random access point is encoded;
Picture decoding apparatus, being used for that the reference picture before the described random access point is carried out error concealment handles, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image; With described reconstructed image, and described in-frame encoding picture, described forward predictive coded image is decoded.
The embodiment of the invention, reference picture before the random access point is carried out error concealment to be handled, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image; With reference to first in-frame encoding picture after the described random access point, and described reconstructed image, at least one forward predictive coded image after the described random access point is encoded, make when when in-frame encoding picture begins random access takes place, forward predictive coded image behind the RAP still can adopt the multi-reference frame Predicting Technique, therefore, the image coding technique scheme that the embodiment of the invention provides, under the prerequisite that guarantees the random access performance, improved code efficiency, and, make random access can also occur in the border of GOP, improved the flexibility of random access.
Description of drawings
Figure 1A is two frames for the frame number when reference picture, the schematic diagram that obtains reconstructed image that the embodiment of the invention provides, and this moment reconstructed image and RAP1 image before is irrelevant, but image-related with the GOP1 before the RAP2;
Figure 1B is two frames for the frame number when reference picture, the schematic diagram that obtains reconstructed image that the embodiment of the invention provides, and this moment reconstructed image and RAP2 image before is irrelevant;
Fig. 2 is when the frame number of reference picture is two frames, the coding schematic flow sheet to the P image that the embodiment of the invention provides;
Fig. 3 is when the frame number of reference picture is two frames, the decoding process schematic diagram to the P image that the embodiment of the invention provides;
Fig. 4 is when the frame number of reference picture is three frames, the gop structure schematic diagram that the embodiment of the invention provides;
Fig. 5 is when the frame number of reference picture is three frames, the coding schematic flow sheet to the P image that the embodiment of the invention provides;
Fig. 6 is when the frame number of reference picture is three frames, the decoding process schematic diagram to the P image that the embodiment of the invention provides;
The schematic flow sheet of the image processing method that Fig. 7 provides for the embodiment of the invention;
The process flow schematic diagram when random access takes place that Fig. 8 provides for the embodiment of the invention;
The structural representation of the picture coding device that Fig. 9 provides for the embodiment of the invention;
The concrete structure schematic diagram of the picture coding device that Figure 10 provides for the embodiment of the invention;
The structural representation of the picture decoding apparatus that Figure 11 provides for the embodiment of the invention
The concrete structure schematic diagram of the picture decoding apparatus that Figure 12 provides for the embodiment of the invention;
The image processing system schematic diagram that Figure 13 provides for the embodiment of the invention.
Embodiment
The embodiment of the invention provides a kind of image coding/decoding method and device, in order under the prerequisite that guarantees the random access performance, further improves the code efficiency of image, and improves the flexibility of random access.
The first frame coded image behind the RAP is the I image, P image in the prior art behind the RAP is only with reference to the I image, and in the embodiment of the invention, the P image behind the RAP can also be with reference to the reconstructed image that obtains by the nearest B picture decoding of the RAP that adjusts the distance not only with reference to the I image.The nearest B image of the distance R AP that mentions in the embodiment of the invention all refers to the nearest B image according to DISPLAY ORDER distance R AP.
Same as the prior art in the embodiment of the invention to not needing behind I image, B image and the RAP with reference to the Code And Decode process of the P image of the image before the RAP, therefore, introduce the process of Code And Decode of the P image of needs behind the RAP image before with reference to RAP in the embodiment of the invention.
The embodiment of the invention is the situation of two frames and three frames with the frame number of reference picture, describes respectively.
The frame number of at first introducing reference picture is the situation of two frames.
The gop structure that the embodiment of the invention provides is shown in Figure 1A or 1B, and putting in order of the coded image shown in the figure is DISPLAY ORDER.The reference picture of the P4 image behind the RAP2 comprises I3 image and B1 ' image, and B1 ' image is for to carry out the reconstructed image that error concealment obtains to the B1 image; Perhaps, the image before the RAP2 is carried out error concealment handle, and described B1 image is carried out the reconstructed image that decoding processing obtains.Wherein, will obtain the processing that reconstructed image is done in the embodiment of the invention, and be called decoding refresh and handle.The B1 image is the nearest B image (DISPLAY ORDER) of distance R AP2.The reference picture of B1 image when coding comprises I3 image and P1 image.
With the P4 image behind the RAP2 is example, referring to Fig. 2, the coding method of P4 image is comprised:
S201, the B1 image is carried out decoding refresh handle, obtain B1 ' image.
S202, with B1 ' image and I3 image as the reference image, the P4 image is encoded.
Wherein, can adopt following dual mode to obtain B1 ' image:
Mode one: begin to carry out decoding refresh from RAP1 and handle.Shown in Figure 1A, this moment, B1 ' was irrelevant with the image before the RAP1, but image-related with the GOP1 before the RAP2.That is, the image before B1 ' image and the I0 image is irrelevant, and B1 ' image obtains by reference P2 ' image, and P2 ' image obtains by reference P1 ' image and I0 image, and P1 ' image obtains by reference I0 image.P2 ' image wherein and P1 ' image are respectively the reconstructed images of P2 image and P1 image.The P1 image not only can be with reference to the I0 image in when coding, can also be further with reference to the image before the RAP1.So, be divided into two kinds of situations this moment.
Situation one: if the P1 image when coding only with reference to the I0 image, then P1 ' image can correctly obtain by reference I0 image.
Situation two: if the P1 image when coding not only with reference to the I0 image, also with reference to RAP1 image before, then P1 ' image will adopt the method for error concealment, only obtains with reference to the I0 image.
The B1 ' that employing mode one generates is irrelevant with the image before the RAP1, but image-related with the P1 among the RAP2 GOP1 before.Because B1 ' image has passed through the decoding refresh of I0 image and two I images of I3, thus the better image quality can have been guaranteed, thus bigger to the contribution of code efficiency.
Mode two: begin to carry out decoding refresh from RAP2 and handle.Shown in Figure 1B, this moment, B1 ' was irrelevant with the image before the RAP2, and promptly B1 ' image only obtains with reference to the I3 image.Adopt the method for error concealment,, generate B1 ' image only with reference to the I3 image.When the P4 image is encoded, with reference to I3 image and B1 ' image.In decoding end, when at the RAP2 place random access taking place, the image before B1 ' image and the RAP2 is irrelevant, the P4 image can be correctly decoded, so, adopt this mode, when random access takes place, can decode correct images at once, the transient process that does not exist recovery to be correctly decoded.
If with reference to the image before the RAP2, promptly the B1 image does not lack reference picture to the B1 image, in fact the B1 ' that generates this moment is exactly the normal decoder image of B1 so, also handles with regard to not needing to carry out error concealment.
Be example still with the P4 image behind the RAP2, corresponding, referring to Fig. 3, the coding/decoding method of P4 image is comprised:
S301, the B1 image is carried out decoding refresh handle, obtain B1 ' image.
S302, with B1 ' image and I3 image as the reference image, the P4 image is decoded.
In the embodiment of the invention, coding side is identical with the method that decoding end obtains reconstructed image.
When random access took place, as at RAP2 random access taking place, can the P4 image behind the RAP2 be correctly decoded immediately, depends on the building method as the reconstructed image B1 ' of additional reference frame.
If the image before B1 ' image and the RAP2 is irrelevant, then B1 ' image can directly obtain in decoding end, and this P4 image will can not lack reference picture this moment so, can be correctly decoded.Successive image also can be correctly decoded.Thereby realized random access.And, the process that do not exist recovery to be correctly decoded this moment.
If image-related before B1 ' image and the RAP2, then B1 ' image is in the decoding end non-availability, this P4 image will can't be correctly decoded owing to lacking reference picture so, can adopt the method for error concealment to handle this moment, promptly this P4 image serves as to handle with reference to carrying out error concealment with the I3 image among the current GOP2 only, obtain the reconstructed image of P4 image, the reference when being used to show with other images of decode successive.Successive image is proceeded decoding, carry out decoding refresh up to the nearest B image of next RAP (RAP3) (B2 image) of adjusting the distance and handle to generate reconstructed image, the reference picture when this reconstructed image is used for first P image (P5 image) behind the RAP3 decoded.From RAP3, first P image behind the RAP3 does not lack reference picture, thereby can be correctly decoded, and other successive images also can be correctly decoded.As seen the process that has a recovery to be correctly decoded here: promptly random access is cut from current RAP, and recovers fully to be correctly decoded from next RAP.To the decode procedure that is carried out the next RAP, comprise error concealment and decoding refresh process from current RAP, the present invention is referred to as refreshing process to such process.So the technical scheme that provides by the embodiment of the invention makes because the problem that reference frame that random access brings is lost takes place, and can not cause the lasting diffusion of error, at most only can influence current RAP and arrive image between the next RAP.
The frame number of introducing reference picture below is the situation of three frames.
The gop structure that the embodiment of the invention provides as shown in Figure 4, this moment putting in order of coded image shown in Figure 4 be coded sequence.The reference picture of the P4 image behind the RAP2 comprises I3 image, B1 ' image and B2 ' image.The reconstructed image of B1 ' image for the B1 picture decoding is obtained; The reconstructed image of B2 ' image for the B2 picture decoding is obtained.The B2 image is the nearest B image (DISPLAY ORDER) of distance R AP2.B1 image and the B2 image reference picture when coding includes P1 image, P2 image and I3 image.
With the P4 image behind the RAP2 is example, referring to Fig. 5, the coding method of P4 image is comprised:
S501, the B1 image is carried out decoding refresh handle, obtain B1 ' image.
S502, the B2 image is carried out decoding refresh handle, obtain B2 ' image.
S503, with B1 ' image, B2 ' image and I3 image as the reference image, the P4 image is encoded.
Wherein, the method that obtains obtaining among the method for B1 ' image and B2 ' image and the above-mentioned embodiment when reference frame is two frames reconstructed image in like manner.Comprise following dual mode:
Begin to carry out decoding refresh from RAP1 and handle, carry out decoding refresh until nearest two the continuous B images (B1 image and B2 image) of the RAP2 that adjusts the distance and handle, generate 2 reconstructed images (B1 ' image and B2 ' image).This moment these two reconstructed images and RAP1 image before is irrelevant, but image-related with the GOP1 before the RAP2, therefore can cause having the transient process that recovery is correctly decoded when random access takes place.
Begin directly to adopt the method for error concealment nearest two the continuous B images (B1 image and B2 image) of RAP2 of adjusting the distance to handle from RAP2, generate 2 reconstructed images (B1 ' image and B2 ' image).This moment these two reconstructed images and RAP2 image before is irrelevant.Adopt this mode, when random access took place, the P image of RAP2 back can decode correct images at once, the transient process that does not exist recovery to be correctly decoded.A kind of special circumstances are arranged, if nearest two the continuous B images of distance R AP2 are not with reference to the image before the RAP2, be that these B images do not lack reference picture and can be correctly decoded, in fact Ci Shi reconstructed image is exactly the normal decoder image of these B images so, at this moment, not needing to carry out error concealment handles.
Be example also with the P4 image behind the RAP2, corresponding, referring to Fig. 6, the coding/decoding method of P4 image is comprised:
S601, the B1 image is carried out decoding refresh handle, obtain B1 ' image.
S602, the B2 image is carried out decoding refresh handle, obtain B2 ' image.
S603, with B1 ' image, B2 ' image and I3 image as the reference image, the P4 image is decoded.
When random access took place, as at RAP2 random access taking place, can first P image (P4 image) and second P image (P5 image) behind the RAP2 be correctly decoded immediately, depend on as the reconstructed image B1 ' of additional reference frame and the building method of B2 '.If promptly the image before reconstructed image B1 ' and B2 ' and the RAP2 is irrelevant, then can be correctly decoded immediately; If image-related before reconstructed image B1 ' and B2 ' and the RAP2, the process that then needs a recovery to be correctly decoded.
In sum, referring to Fig. 7, the image processing method that the embodiment of the invention provides comprises:
S701, at coding side, the P image behind the RAP is adopted the multi-reference frame predictive coding, the B image is carried out decoding refresh handles, obtain reconstructed image B '; And with described reconstructed image, and the I image is encoded to the P image as the reference image.
Wherein, the mode that obtains described reconstructed image B ' comprises:
Reference picture before the RAP is carried out error concealment to be handled, and the B image behind the described RAP carried out decoding processing, obtain reconstructed image B ', wherein, described reference picture and described B are image-related, and promptly described reference picture is the direct reference picture of described B image or indirect reference picture.
Mode two: the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image.
S702, in decoding end, described B image is carried out decoding refresh handles, obtain described reconstructed image B '; And with described reconstructed image, and described I image is decoded to described P image as the reference image.
When random access takes place described RAP,, comprise step referring to Fig. 8:
S801, to being positioned at the P image behind the described RAP, the reference picture B ' that judges the P image whether with described RAP before image-related, promptly whether the reference picture of B image can get, if, then carry out step S802, otherwise, carry out step S803.
S802, to described P picture decoding the time, adopt the method for error concealment to handle, obtain the decoded picture of P image.
Image before the reference picture B ' of S803, P image and the described RAP is irrelevant, then can be correctly decoded at once, obtains the decoded picture of P image.
One of ordinary skill in the art will appreciate that all or part of step that realizes in the coding/decoding method that the foregoing description provides is to instruct relevant hardware to finish by program, described program can be stored in the computer read/write memory medium, this program comprises the coding step of image when carrying out:
Reference picture before the random access point is carried out error concealment to be handled, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image;
With reference to first in-frame encoding picture after the described random access point, and described reconstructed image, at least one the forward predictive coded image after the described random access point is encoded.
Correspondingly, described program comprises the decoding step of image when carrying out:
Reference picture before the random access point is carried out error concealment to be handled, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image;
With reference to first in-frame encoding picture after the described random access point, and described reconstructed image, at least one the forward predictive coded image after the described random access point is decoded.
Described storage medium, as: ROM/RAM, magnetic disc, CD etc.
Below introduce the device that the embodiment of the invention provides.
Referring to Fig. 9, a kind of picture coding device that the embodiment of the invention provides comprises: coding refresh process unit 901 and coding unit 902.
Described coding refresh process unit 901, the nearest B image of RAP (perhaps nearest continuous a plurality of B images) that is used to adjust the distance carry out decoding refresh to be handled, and obtains reconstructed image.
Described coding unit 902 is used for described reconstructed image, and first I image behind the described RAP is encoded to first P image behind the described RAP (can also comprise n P image, n is a natural number) as the reference image.
Figure 10 shows the concrete structure of the picture coding device that the embodiment of the invention provides, and as seen from Figure 10, in traditional coding framework, has increased a coding refresh process unit, is specifically designed to and generates extra reference picture.
In the coding refresh process unit, comprise an error concealment processing unit and a codec processing unit (during practical application, this codec processing unit can be reused with the codec processing unit in the conventional codec).In coding refresh process unit shown in Figure 10, the unit except the error concealment processing unit all belongs to described codec processing unit.As seen from Figure 10, after the inverse transformation unit, codec processing unit in the conventional codec and the parallel decoding processing of carrying out of coding refresh process units synchronization.Codec processing unit in the conventional codec is output as the reconstructed image of normal decoder, and coding refresh process unit is output as the reconstructed image of decoding refresh.Illustrate the handling process of described coding refresh process unit below.
When reference frame was two frames, the handling process of coding refresh process unit was as follows:
1) to first P frame behind the RAP, adopt the error concealment technology to handle, generate reconstructed image, and as the decoding reference of follow-up other images.
2) follow-up other images are carried out decoding refresh and handle, be actually and carry out the predictive compensation decoding.
3) the nearest B image of next RAP of adjusting the distance carries out decoding refresh and handles, and obtains reconstructed image B '.Reference when this B ' image is used for first P image encoding of next RAP back.
4) repeat said process.
When reference frame was three frames, the handling process of coding refresh process unit was as follows:
1) to first P frame behind the RAP and second P frame, adopt the error concealment technical finesse, generate reconstructed image, and as the decoding reference of follow-up other images.
2), be actually and carry out the predictive compensation decoding to follow-up other picture decoding refresh process.
3) 2 the nearest continuous B images of next RAP of adjusting the distance carry out decoding refresh and handle, and obtain 2 reconstructed images.Reference picture when these 2 reconstructed images are used for the coding of first P frame behind the next RAP and second P frame.
4) repeat said process.
Correspondingly, referring to Figure 11, a kind of picture decoding apparatus that the embodiment of the invention provides comprises: decoding refresh processing unit 1101 and decoding unit 1102.
Described decoding refresh processing unit 1101, the nearest B image of RAP (perhaps nearest continuous a plurality of B images) that is used to adjust the distance carry out decoding refresh to be handled, and obtains reconstructed image.
Described decoding unit 1102 is used for described reconstructed image, and first I image behind the described RAP is encoded to first P image behind the described RAP (can also comprise n P image, n is a natural number) as the reference image.
Figure 12 shows the concrete structure of the picture decoding apparatus that the embodiment of the invention provides, and as seen from Figure 12, in traditional coding framework, has increased a decoding refresh processing unit, is specifically designed to generate extra reference picture.
The handling process of the coding refresh process unit in the handling process of the decoding refresh processing unit in the picture decoding apparatus that the embodiment of the invention provides and the above-mentioned picture coding device is corresponding, and difference is that the reference picture that is generated is used to decode.
Referring to Figure 13, the image processing system that the embodiment of the invention provides comprises: picture coding device 1301 and picture decoding apparatus 1302.
Described picture coding device 1301, being used for that the reference picture before the random access point is carried out error concealment handles, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the reference picture of described bidirectionally predictive coded picture; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image; With first in-frame encoding picture after the described random access point, and described reconstructed image is encoded at least one the forward predictive coded image after the described random access point as the reference image.
Described picture decoding apparatus 1302, being used for that the reference picture before the described random access point is carried out error concealment handles, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the reference picture of described bidirectionally predictive coded picture; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image; With described reconstructed image, and described in-frame encoding picture is decoded to described forward predictive coded image as the reference image.
In sum, the embodiment of the invention provides a kind of and non-reference frame is carried out decoding refresh handles technical scheme with the code efficiency of improving the P image behind the RAP.Non-reference frame is carried out decoding refresh handle, construct the nearest reconstructed image of distance R AP, and as the additional reference image of the P image behind the RAP, thereby coding gain improved.And the embodiment of the invention is supported random access, and random access can occur in the border of figure group, and the flexibility of random access, has been improved in promptly any I frame place therefore.
Technical scheme provided by the invention, have lost frame recovering, quick synchronous again ability, therefore, can be applicable to editor/splicing, the video request program (VOD of digital video broadcasting, Streaming Media multicast, compressed bit stream, Video On Demand)/individual digital video recording (PVR, application scenarios such as fast forwarding and fast rewinding when Personal Digital Video Recorder) playing, can also be applied to look between looking in the coding (multiview) more and switch, and the switching of the interlayer in the scalable coding (SVC), to improve the code efficiency at switching point place.
Obviously, those skilled in the art can carry out various changes and modification to the present invention and not break away from the spirit and scope of the present invention.Like this, if of the present invention these are revised and modification belongs within the scope of claim of the present invention and equivalent technologies thereof, then the present invention also is intended to comprise these changes and modification interior.
Claims (11)
1. a method for encoding images is characterized in that, this method comprises:
Reference picture before the random access point is carried out error concealment to be handled, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture, and the image before the particular frame intra-coded picture before this reconstructed image and the described random access point is irrelevant;
Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image, wherein, the image before this reconstructed image and the described random access point is irrelevant;
With reference to first in-frame encoding picture after the described random access point, and described reconstructed image, at least one the forward predictive coded image after the described random access point is encoded.
2. method according to claim 1 is characterized in that, according to coded sequence, described bidirectionally predictive coded picture is positioned at after the described in-frame encoding picture; According to DISPLAY ORDER, described bidirectionally predictive coded picture is positioned at before the described in-frame encoding picture.
3. method according to claim 1 is characterized in that, when the image before the particular frame intra-coded picture before described reconstructed image and the described random access point was irrelevant, the step that obtains described reconstructed image comprised:
Particular frame intra-coded picture with reference to before the described random access point by error concealment, obtains first reconstructed image;
With reference to described first reconstructed image, subsequent reference images is decoded, and obtain second reconstructed image;
With described first reconstructed image and/or described second reconstructed image, and first in-frame encoding picture after the described random access point is as the reference image, bidirectionally predictive coded picture after the described random access point is carried out decoding processing, obtain the reconstructed image of described bidirectionally predictive coded picture.
4. a picture decoding method is characterized in that, this method comprises:
Reference picture before the random access point is carried out error concealment to be handled, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture, and the image before the particular frame intra-coded picture before this reconstructed image and the described random access point is irrelevant;
Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image, wherein, the image before this reconstructed image and the described random access point is irrelevant;
With reference to first in-frame encoding picture after the described random access point, and described reconstructed image, at least one the forward predictive coded image after the described random access point is decoded.
5. method according to claim 4 is characterized in that, according to DISPLAY ORDER, described bidirectionally predictive coded picture is positioned at before the described in-frame encoding picture; According to coded sequence, described bidirectionally predictive coded picture is positioned at after the described in-frame encoding picture.
6. method according to claim 4 is characterized in that, when the image before the particular frame intra-coded picture before described reconstructed image and the described random access point was irrelevant, the step that obtains described reconstructed image comprised:
Particular frame intra-coded picture with reference to before the described random access point by error concealment, obtains first reconstructed image;
With reference to described first reconstructed image, subsequent reference images is decoded, and obtain second reconstructed image;
With described first reconstructed image and/or described second reconstructed image, and first in-frame encoding picture after the described random access point is as the reference image, bidirectionally predictive coded picture after the described random access point is carried out decoding processing, obtain the reconstructed image of described bidirectionally predictive coded picture.
7. a picture decoding method is characterized in that, this method comprises: when at random access point generation random access,
If the bidirectionally predictive coded picture after the described random access point, when coding with reference to the image before the described random access point, then with reference to first intraframe predictive coding image after the described random access point, by error concealment, at least one the forward predictive coded image after the described random access point is decoded;
If the bidirectionally predictive coded picture after the described random access point only with reference to the image after the described random access point, then is correctly decoded at least one the forward predictive coded image after the described random access point when coding.
8. an image processing method is characterized in that, this method comprises:
At coding side, reference picture before the random access point is carried out error concealment to be handled, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, and the image before the particular frame intra-coded picture before this reconstructed image and the described random access point is irrelevant, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image, wherein, the image before this reconstructed image and the described random access point is irrelevant; With reference to first in-frame encoding picture after the described random access point, and described reconstructed image, at least one the forward predictive coded image after the described random access point is encoded;
In decoding end, reference picture before the described random access point is carried out error concealment to be handled, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, and the image before the particular frame intra-coded picture before this reconstructed image and the described random access point is irrelevant, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image, wherein, the image before this reconstructed image and the described random access point is irrelevant; With reference to described reconstructed image, and described in-frame encoding picture, described forward predictive coded image is decoded.
9. a picture coding device is characterized in that, this device comprises:
Coding refresh process unit, being used for that the reference picture before the random access point is carried out error concealment handles, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture, and the image before the particular frame intra-coded picture before this reconstructed image and the described random access point is irrelevant; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image, wherein, the image before this reconstructed image and the described random access point is irrelevant;
Coding unit is used for reference to first in-frame encoding picture after the described random access point, and described reconstructed image, and at least one the forward predictive coded image after the described random access point is encoded.
10. a picture decoding apparatus is characterized in that, this device comprises:
The decoding refresh processing unit, being used for that the reference picture before the random access point is carried out error concealment handles, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture, and the image before the particular frame intra-coded picture before this reconstructed image and the described random access point is irrelevant; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image, wherein, the image before this reconstructed image and the described random access point is irrelevant;
Decoding unit is used for reference to first in-frame encoding picture after the described random access point, and described reconstructed image, and at least one the forward predictive coded image after the described random access point is decoded.
11. an image processing system is characterized in that, this system comprises:
Picture coding device, being used for that the reference picture before the random access point is carried out error concealment handles, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture, and the image before the particular frame intra-coded picture before this reconstructed image and the described random access point is irrelevant; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image, wherein, the image before this reconstructed image and the described random access point is irrelevant; With reference to first in-frame encoding picture after the described random access point, and described reconstructed image, at least one the forward predictive coded image after the described random access point is encoded;
Picture decoding apparatus, being used for that the reference picture before the described random access point is carried out error concealment handles, and the bidirectionally predictive coded picture after the described random access point carried out decoding processing, obtain reconstructed image, wherein, described reference picture is the direct or indirect reference picture of described bidirectionally predictive coded picture, and the image before the particular frame intra-coded picture before this reconstructed image and the described random access point is irrelevant; Perhaps, the bidirectionally predictive coded picture after the described random access point is carried out error concealment handle, obtain reconstructed image, wherein, the image before this reconstructed image and the described random access point is irrelevant; With described reconstructed image, and described in-frame encoding picture, described forward predictive coded image is decoded.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200710153927 CN101389034B (en) | 2007-09-14 | 2007-09-14 | Image encoding/decoding method, apparatus and an image processing method, system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200710153927 CN101389034B (en) | 2007-09-14 | 2007-09-14 | Image encoding/decoding method, apparatus and an image processing method, system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101389034A CN101389034A (en) | 2009-03-18 |
| CN101389034B true CN101389034B (en) | 2010-06-09 |
Family
ID=40478160
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200710153927 Expired - Fee Related CN101389034B (en) | 2007-09-14 | 2007-09-14 | Image encoding/decoding method, apparatus and an image processing method, system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101389034B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101750495B1 (en) | 2010-04-16 | 2017-06-27 | 에스케이텔레콤 주식회사 | Inter Prediction method and Decoding/Encoding Method using Inter Prediction |
| US9491483B2 (en) | 2010-04-16 | 2016-11-08 | Sk Telecom Co., Ltd. | Inter-prediction method and video encoding/decoding method using the inter-prediction method |
| JP6045222B2 (en) * | 2012-06-28 | 2016-12-14 | 株式会社Ｎｔｔドコモ | Moving picture predictive decoding apparatus, method and program |
| US9591303B2 (en) * | 2012-06-28 | 2017-03-07 | Qualcomm Incorporated | Random access and signaling of long-term reference pictures in video coding |
| CN105812798A (en) * | 2014-12-31 | 2016-07-27 | 深圳中兴力维技术有限公司 | Image encoding and decoding method and device thereof |
| CN104768011B (en) | 2015-03-31 | 2018-03-06 | 浙江大学 | Image coding/decoding method and relevant apparatus |
| EP3160142A1 (en) * | 2015-10-21 | 2017-04-26 | Thomson Licensing | Method for encoding and method for decoding a light field based image and corresponding devices |
| WO2017075804A1 (en) * | 2015-11-06 | 2017-05-11 | Microsoft Technology Licensing, Llc | Flexible reference picture management for video encoding and decoding |
| CN108243339B (en) * | 2016-12-27 | 2021-02-26 | 浙江大学 | Image coding and decoding method and device |
| CN113824965A (en) * | 2021-09-22 | 2021-12-21 | 北京百度网讯科技有限公司 | Video coding method and device and electronic equipment |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1913640A (en) * | 2006-08-11 | 2007-02-14 | 宁波大学 | Multi-mode multi-viewpoint video signal code compression method |
-
2007
- 2007-09-14 CN CN 200710153927 patent/CN101389034B/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1913640A (en) * | 2006-08-11 | 2007-02-14 | 宁波大学 | Multi-mode multi-viewpoint video signal code compression method |
Non-Patent Citations (2)
| Title |
|---|
| JP特开2003-224826A 2003.08.08 |
| JP特开2006-246277A 2006.09.14 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101389034A (en) | 2009-03-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101389034B (en) | Image encoding/decoding method, apparatus and an image processing method, system | |
| US6980594B2 (en) | Generation of MPEG slow motion playout | |
| JP3923898B2 (en) | Image coding method and apparatus | |
| EP1323314B1 (en) | Transcoding progressive i-slice refreshed mpeg data streams to enable trick play | |
| CN101321284B (en) | Encoding/decoding method, equipment and system | |
| US7961788B2 (en) | Method and apparatus for video encoding and decoding, and recording medium having recorded thereon a program for implementing the method | |
| US20060233235A1 (en) | Video encoding/decoding apparatus and method capable of minimizing random access delay | |
| US8811483B2 (en) | Video processing apparatus and method | |
| CN102761773A (en) | Method for improving user experience of IPTV (Internet Protocol Television) service by using I frames | |
| JP2007221411A (en) | Image encoder and image decoder, image encoding and decoding method, and program thereof | |
| CN101459840B (en) | Encoding and decoding method, apparatus and system for video image | |
| CN101489101B (en) | Method, apparatus and system for IP television channel switching processing | |
| CN101389018B (en) | Image encoding/decoding method, apparatus and a image processing method, system | |
| US8155186B2 (en) | Bit stream recording medium, video encoder, and video decoder | |
| JP5189618B2 (en) | Video encoding method, video encoding apparatus, and video encoding program | |
| JP2020108032A (en) | Video code stream editing device and program | |
| US9219930B1 (en) | Method and system for timing media stream modifications | |
| JP4043406B2 (en) | Image decoding method and apparatus, and television receiver capable of using them | |
| US20110299591A1 (en) | Video processing apparatus and method | |
| JP5240230B2 (en) | Transcoder, recording apparatus, and transcoding method | |
| JP2002218470A (en) | Method for converting image encoded data rate and device for converting image encoding rate | |
| US20100166383A1 (en) | System and method for providing trick modes | |
| JP2005159832A (en) | Signal processor | |
| JP4651344B2 (en) | MPEG-2 stream wipe switching method | |
| JP2011250358A (en) | Image processing apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20100609 Termination date: 20200914 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |