JP2015133742A - Moving image prediction encoding device, moving image prediction encoding method, moving image prediction encoding program, moving image prediction decoding device, moving image prediction decoding method, and moving image prediction decoding program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve trouble (malfunction etc.) related to a weak point of the prior art by efficient compression encoding of images before or after an image which becomes a spot of random access.SOLUTION: A moving image prediction encoding device includes: encoding means for encoding input images to generate compressed image data including a random access image, and encoding data related to display order information on the images; restoring means for restoring reproduction images by decoding the compressed image data; and image storing means for storing the reproduction images as reference images. As the reference image to be used when encoding an image whose display order information is larger than that of the random access image, the restoring means decodes the compressed image data including such reference image information that does not include a reference image preceding the random access image in encoding processing order or display order information.

Description

  The present invention relates to a moving picture predictive coding apparatus, method, and program, and a moving picture predictive decoding apparatus, method, and program.

  In order to efficiently transmit and store moving image data, a compression encoding technique is used. In the case of moving images, MPEG1-4 and H.264 are used. 261-H. H.264 is widely used.

  In these encoding methods, an image to be encoded is divided into a plurality of blocks, and encoding / decoding processing is performed on each block. In order to increase the encoding efficiency, the following predictive encoding method is used. In predictive coding within a screen, a prediction signal is generated using an adjacent reproduced image signal (reconstructed image data encoded in the past) in the same screen as the target block, and the predicted signal is used as the target block. The difference signal obtained by subtracting from the signal is encoded. In predictive encoding between screens, a displacement of a signal is searched with reference to a previously reproduced image signal in a screen different from the target block, a predicted signal is generated by compensating for the displacement, and the predicted signal is converted to the target block. The differential signal obtained by subtraction from the signal is encoded. At this time, the already-reproduced image referred to for motion search / compensation is referred to as a reference image.

  Further, in the bidirectional inter-screen prediction, not only the past image displayed before the target image in the display time order but also a future image displayed after the target image may be referred to (however, The future image must be encoded before the target image and reproduced in advance). And by averaging both the prediction signal acquired from the past image and the prediction signal acquired from the future image, it is effective in predicting the signal of the hidden and newly appearing object. There is an effect of reducing noise contained in the prediction signal.

  Further, H.C. In the H.264 inter-frame prediction encoding, the prediction signal for the target block refers to a plurality of reference images encoded and reproduced in the past, and an image signal with the least error is selected as an optimal prediction signal while performing motion search. . Then, a difference between the pixel signal of the target block and the optimal prediction signal is obtained, and the difference is subjected to discrete cosine transform and quantized, and then entropy-coded. At the same time, information relating to which reference image the optimum prediction signal for the target block is acquired from (reference index) and information relating to which region in the reference image the optimum prediction signal is obtained (motion vector) are also encoded. To do. H. In H.264, 4 to 5 reproduced images are stored in the frame memory as reference images. In this specification, the frame memory includes a so-called decoded picture buffer.

  Inter-screen predictive coding can efficiently compress and encode images by utilizing the correlation between images, but in order to be able to view a video program from the middle by switching TV channels, the inter-screen dependency It is necessary to cut off. A portion having no dependency between screens in a compressed bit stream of a moving image is hereinafter referred to as a “random access point”. In addition to channel switching, a random access point is also required when editing moving images and connecting compressed data of different moving images. H. In H.264, an IDR picture is designated, and the designated IDR picture is encoded by the above-described intra prediction encoding method, and at the same time, the reproduction image stored in the frame memory is set to be unnecessary, and the reproduction image is referred to. By disabling it, the frame memory is substantially released (refreshed). Such a process is called “memory refresh” and is sometimes called “frame memory refresh” or “buffer refresh”.

  FIG. 11A is a schematic diagram illustrating a prediction structure of a moving image including an IDR picture. A plurality of images 901, 902,..., 909 shown in FIG. 11A are part of an image group constituting a moving image, and each image is also referred to as “picture” or “frame”. Each arrow indicates the direction of prediction. For example, the image 902 acquires a prediction signal using the images 903 and 905 that are the starting points of two arrows toward the image 902 as reference images. Note that the image 901 in FIG. 11A is encoded with reference to a past image not shown in FIG. Next, the images 902, 903, and 904 are encoded. At this time, the above-described bidirectional predictive encoding method is used to increase the compression rate. That is, the image 905 is first encoded and reproduced, and then the image 903 is encoded with reference to the images 901 and 905 that have already been reproduced (in FIG. 11A, the arrow from the image 901 is omitted). ing). Each of the images 902 and 904 is encoded using the three reproduced images 901, 905, and 903 as reference images (in FIG. 11A, arrows from the image 901 are omitted). Similarly, images 906, 907, and 908 are encoded with reference to images 905 and 909. The compressed data of each image encoded (compressed) in this way is transmitted or stored in the order as shown in FIG. The compressed data in FIG. 11 (B) and the image in FIG. 11 (A) show a correspondence relationship by common codes such as P1, IDR5, and B3. For example, the compressed data 910 is compressed data of the image 901 with the same code P1, and the compressed data 911 is compressed data of the image 905 with the same code IDR5.

  Now, considering random access, consider the case where the image 905 is designated as an IDR picture and intra prediction encoding is performed. In this case, H.C. According to the IDR rule in H.264, immediately after the image 905 is reproduced by decoding the compressed data 911 (or immediately before the decoding of the compressed data 911 is started), all the references stored in the frame memory are stored. An image (that is, a past reproduction image including the image 901) must be set as unnecessary and cannot be referred to. As a result, the image 901 in FIG. 11A cannot be referred to, and the reference from the image 901 cannot be performed when the images 902, 903, and 904 are encoded. Such processing relating to an IDR picture is described, for example, in Non-Patent Document 1 below.

International Publication WO2005 / 006763A1

Iain E.G.Richardson, "H.264 and MPEG-4 Video Compression", John Wiley & Sons, 2003, section 6.4.2.

  Since the reference image used for the above prediction is limited by the introduction of the IDR picture, the images (images 902, 903, and 904 in FIG. 11A) that precede the IDR picture in the image display order are efficiently displayed. It becomes impossible to encode. In order to solve this point, the above-mentioned Patent Document 1 describes the frame memory refresh timing (that is, the timing at which the reference image in the frame memory is set to be unnecessary) as the code of the image encoded after the IDR picture. A method of delaying until the execution time is disclosed. By delaying the refresh timing of the frame memory, since the image 901 remains in the frame memory when the images 902, 903, and 904 in FIG. 11A are encoded, the images 902, 903, and 904 are encoded. The image 901 can be referred to and can be encoded efficiently.

According to Patent Document 1, the following method is disclosed as a method of delaying the memory refresh timing.
Method 1: Information regarding the number of images to be delayed is added to the IDR picture.
Method 2: A signal (flag) instructing execution of the memory refresh is added to the compressed image data corresponding to the timing at which the memory refresh is performed.
Method 3: The first P picture (unidirectional prediction image) appearing after the IDR picture is set as the refresh timing.

However, the above method has the following drawbacks.
Disadvantage 1: In Method 1 described above, when editing a moving image, some of the plurality of images are discarded and another image is connected, or another image is inserted. There is an inconvenience that “information on the number of images to be delayed” added to the IDR picture is not appropriate and causes a malfunction.
Disadvantage 2: In Method 2 above, even when a flag is used, if the compressed data of the corresponding image is deleted by editing the moving image, the flag added to the deleted compressed data disappears, causing a malfunction. Inconvenience.
Disadvantage 3: In method 3 above, the memory refresh signal (timing) is limited to the P picture, so that there is a disadvantage that it cannot be encoded by other methods. For example, it cannot be encoded as intra prediction (I picture) at a scene change.

  Note that the “malfunction” here is due to the fact that the memory refresh is not performed at an appropriate timing, and the reference image necessary for decoding subsequent data is not stored in the frame memory. As a result, it means that subsequent images cannot be reproduced correctly.

  SUMMARY OF THE INVENTION An object of the present invention is to solve the above-described problems and efficiently compress and encode images before and after an image to be a random access location, and at the same time eliminate the disadvantages associated with the disadvantages of the prior art.

  In order to achieve the above object, a moving image predictive encoding apparatus according to the present invention includes an input unit that inputs a plurality of images constituting a moving image, and the input image is either intra prediction or inter prediction. By encoding using this method, compressed image data including a random access image that is a random access image is generated, and encoding means for encoding data related to the display order information of the image is generated. A restoration unit that restores a reproduced image by decoding the compressed image data; and an image storage unit that stores the restored reproduction image as a reference image used to encode a subsequent image. The restoration means uses an encoding process as a reference image used when encoding an image whose display order information is larger than the display order information of a random access image. Or wherein decoding the compressed image data including the reference image information that does not contain a reference image preceding the random access image in the display order information, characterized in that. In other words, the restoration means uses the encoding process order and display order information as the reference image used when encoding compressed image data of an image whose display order information is larger than the display order information of the random access image. The compressed image data including the reference image information including only the reference image larger than or the same as the image encoding processing order and the display order information is decoded.

  In order to achieve the above object, the moving picture predictive decoding apparatus according to the present invention is obtained by encoding a plurality of images constituting a moving picture by either intra prediction or inter prediction. Input means for inputting compressed image data including a random access image that is a random access image, and display order encoded data obtained by encoding data relating to display order information of the image, and the compressed image Reconstructing the reproduced image by decoding the data, restoring means for restoring the display order information by decoding the display order encoded data, and decoding the restored reproduced image to the subsequent image An image storage means for storing as a reference image to be used, wherein the restoration means is an image whose display order information is larger than the display order information of the random access image. As the reference image used when decoding the compressed image data, the compressed image data including reference image information that does not include the reference image preceding the random access image in the decoding processing order or display order information is decoded. It is characterized by that. That is, the restoration means uses a decoding process order and display order information of the random access image as a reference image used when decoding compressed image data of an image whose display order information is larger than the display order information of the random access image. The compressed image data including the reference image information including only the reference image larger than or the same as the decoding processing order and the display order information is decoded.

  In addition, the video predictive encoding method according to the present invention is a video that is executed by a video predictive encoding device that includes an image storage means for storing a reference image used for encoding a subsequent image. Random access by a predictive encoding method in which a plurality of images constituting a moving image are input, and the input image is encoded by either intra prediction or inter prediction. A compressed image data including a random access image that is an image to be reproduced, an encoding step for encoding data relating to display order information of the image, and a reproduced image by decoding the generated compressed image data A restoration step for restoring the image, and the restored reproduction image is stored in the image storage means as a reference image used for encoding a subsequent image. An image storage step, and in the restoration step, the moving image predictive encoding device uses a code as a reference image used when encoding an image whose display order information is larger than the display order information of a random access image. The compressed image data including reference image information that does not include a reference image preceding the random access image in the conversion processing order or display order information is decoded. That is, in the restoration step, the moving image predictive encoding device uses the encoding processing order as a reference image used when encoding compressed image data of an image whose display order information is larger than the display order information of a random access image. And the compressed image data including the reference image information including only the reference image whose display order information is larger than or the same as the encoding process order and the display order information of the random access image.

  In addition, the moving picture predictive decoding method according to the present invention is a moving picture predictive decoding method executed by a moving picture predictive decoding apparatus including an image storage means for storing a reference image used for decoding subsequent images. The compressed image data including a random access image, which is a random access image, obtained by encoding a plurality of images constituting a moving image by a method of intra prediction or inter prediction. And an input step of inputting display order encoded data obtained by encoding the data related to the display order information of the image, and reconstructing the reproduced image by decoding the compressed image data, and the display A restoration step for restoring display order information by decoding the order encoded data and the restored reproduced image are used for decoding subsequent images. An image storage step of storing in the image storage means as a reference image, and in the restoration step, the moving picture predictive decoding device is a compressed image of an image whose display order information is larger than the display order information of a random access image. Decoding the compressed image data including reference image information that does not include the reference image preceding the random access image in the decoding processing order or display order information as a reference image used when decoding data; Features. That is, in the restoration step, the moving image predictive decoding device uses the decoding processing order and the display order as reference images used when decoding compressed image data of an image whose display order information is larger than the display order information of a random access image. The compressed image data including reference image information including only reference images whose information is greater than or equal to the decoding processing order and display order information of the random access image is decoded.

  The moving image predictive encoding program according to the present invention includes a computer, an input means for inputting a plurality of images constituting a moving image, and a method for either intra prediction or inter prediction of the input image. And encoding means for encoding the data related to the display order information of the image, and generating the compressed image data including a random access image that is an image that becomes a random access, and the generated compression The restoration means that restores the reproduced image by decoding the image data, and the image storage means that stores the restored reproduction image as a reference image used to encode the subsequent image, and the restoration As a reference image used when encoding an image whose display order information is larger than the display order information of the random access image, Decoding the compressed image data including the reference image information that does not contain a reference image preceding the random access image in the processing order or display order information, characterized in that. In other words, the restoration means uses the encoding process order and display order information as the reference image used when encoding compressed image data of an image whose display order information is larger than the display order information of the random access image. The compressed image data including the reference image information including only the reference image larger than or the same as the image encoding processing order and the display order information is decoded.

  In addition, the moving picture predictive decoding program according to the present invention includes a random access obtained by encoding a plurality of images constituting a moving picture by a method of intra prediction or inter prediction. Input means for inputting compressed image data including a random access image, which is an image, and display order encoded data obtained by encoding data relating to display order information of the image; and decoding the compressed image data And a restoration means for restoring the display order information by decoding the display order encoded data and a reference used for decoding the restored picture and the subsequent playback picture. Operating as image storage means for storing as an image, wherein the restoration means has display order information larger than display order information of a random access image. As the reference image used when decoding the compressed image data of the image, the compressed image data including the reference image information that does not include the reference image preceding the random access image in the decoding processing order or display order information is decoded. It is characterized by. That is, the restoration means uses a decoding process order and display order information of the random access image as a reference image used when decoding compressed image data of an image whose display order information is larger than the display order information of the random access image. The compressed image data including the reference image information including only the reference image larger than or the same as the decoding processing order and the display order information is decoded.

  In order to achieve the above object, a moving image predictive encoding apparatus according to the present invention includes an input unit that inputs a plurality of images constituting a moving image, and the input image is either intra prediction or inter prediction. By encoding using this method, compressed image data including a random access image that is a random access image is generated, and encoding means for encoding data related to the display order information of the image is generated. A restoring means for restoring a reproduced image by decoding the compressed image data, an image storage means for storing the restored reproduced image as a reference image used for encoding a subsequent image, and the image Memory management means for controlling the storage means, the memory management means first after the encoding process to generate the random access image is completed, Immediately before or immediately after encoding an image whose order information is larger than the display order information of the random access image, out of the reference images stored in the image storage means, the reference image excluding the random access image is set as unnecessary. The image storage means is refreshed.

  The encoding means includes one or more images including display order information larger than display order information of the random access image and an image to be encoded first after completion of the encoding process for generating the random access image. It is desirable to encode a difference value between the display order information of the encoding target image and the display order information of the random access image as data related to the display order information of the encoding target image.

  In addition, the above encoding means completes the encoding process in which the display order information is larger than the display order information of the random access image and the random access image is generated from the image to be encoded next to the random access image. For each image up to the first image to be encoded later, the difference value between the display order information of each image and the display order information of the random access image is encoded as data related to the display order information of each image. It is desirable to do.

  The moving picture predictive decoding apparatus according to the present invention is a random access image obtained by encoding a plurality of pictures constituting a moving picture by a method of intra prediction or inter prediction. Input means for inputting compressed image data including an access image and display order encoded data obtained by encoding data related to display order information of the image, and a reproduced image by decoding the compressed image data And a restoration means for restoring display order information by decoding the display order encoded data, and an image for storing the restored reproduced image as a reference image used for decoding subsequent images Storage means, and memory management means for controlling the image storage means, wherein the memory management means decodes the random access image. First, immediately after decoding an image whose display order information is larger than the display order information of the random access image, the random access image is excluded from the reference images stored in the image storage means. The image storage means is refreshed by setting the reference image as unnecessary.

  The restoration means includes at least one decoding target image including display order information larger than the display order information of the random access image and an image to be decoded first after completion of the decoding process for decoding the random access image. Display order information, obtained by decoding the display order encoded data of the decoding target image, the difference value between the display order information of the decoding target image and the display order information of the random access image, and the random order It is desirable to restore the display order information of the decoding target image by adding the display order information of the access image.

  In addition, the restoration unit may be configured to first display from the image to be decoded next to the random access image, after the completion of the decoding process in which the display order information is larger than the display order information of the random access image and generate the random access image. For the display order information for each image up to the image to be decoded, the display order information for each image and the display order for the random access image obtained by decoding the display order encoded data for each image. It is desirable to restore the display order information of each image by adding the difference value with the information and the display order information of the random access image.

  A moving picture predictive coding method according to the present invention includes a moving picture predictive coding apparatus provided with a picture storing means for storing a reference picture used for coding a subsequent picture. Random access is achieved by an input step of inputting a plurality of images constituting a moving image and encoding the input image by either of intra-screen prediction or inter-screen prediction. A compressed image data including a random access image that is an image is generated, an encoding step for encoding data related to display order information of the image, and a reproduced image is restored by decoding the generated compressed image data A restoration step, and an image case in which the restored reproduction image is stored in the image storage means as a reference image used for encoding a subsequent image. And a memory management step for controlling the image storage means. In the memory management step, the moving image predictive encoding device firstly executes the encoding process for generating the random access image after the completion of the encoding process. Immediately before or immediately after encoding an image whose display order information is larger than the display order information of the random access image, among the reference images stored in the image storage means, reference images other than the random access image are set as unnecessary. Thus, the image storage means is refreshed.

  In the encoding step, the moving image predictive encoding device is the first encoding target after the encoding process for generating the random access image is completed and the display order information is larger than the display order information of the random access image. It is desirable to encode a difference value between the display order information of the encoding target image and the display order information of the random access image as data relating to display order information of one or more encoding target images including an image.

  Further, in the encoding step, the video predictive encoding device has a display order information larger than a display order information of the random access image and an image to be encoded next to the random access image and the random access image. For each image up to the first image to be encoded after completion of the encoding process for generating the image, the display order information of each image and the display order of the random access image are used as data related to the display order information of each image. It is desirable to encode the difference value with the information.

  A moving picture predictive decoding method according to the present invention is a moving picture predictive decoding method executed by a moving picture predictive decoding apparatus including an image storage unit for storing a reference image used for decoding a subsequent image. Compressed image data including a random access image which is a random access image obtained by encoding a plurality of images constituting a moving image by either intra prediction or inter prediction method, and An input step of inputting display order encoded data obtained by encoding data relating to the display order information of the image, and a reproduction image is restored by decoding the compressed image data, and the display order code A restoration step for restoring display order information by decoding the digitized data, and the restored reproduced image is used for decoding subsequent images. An image storage step for storing the image in the image storage unit; and a memory management step for controlling the image storage unit. In the memory management step, the video predictive decoding device decodes the random access image. After the decoding process is completed, first, the random access image among the reference images stored in the image storage means immediately before or immediately after decoding an image whose display order information is larger than the display order information of the random access image. The image storage means is refreshed by setting the reference images other than the reference image to be unnecessary.

  In the restoration step, the video predictive decoding apparatus includes an image that is first displayed after decoding processing is completed, in which display order information is larger than display order information of the random access image and the random access image is decoded. For the display order information of one or more decoding target images, the display order information of the decoding target image and the display order information of the random access image obtained by decoding the display order encoded data of the decoding target image, It is desirable to restore the display order information of the decoding target image by adding the difference value of the image and the display order information of the random access image.

  In the restoration step, the video predictive decoding apparatus generates the random access image having display order information larger than the display order information of the random access image, from an image to be decoded next to the random access image. For the display order information for each image up to the first image to be decoded after the completion of the decoding process, the display order information for each image obtained by decoding the display order encoded data for each image, and It is desirable to restore the display order information of each image by adding the difference value with the display order information of the random access image and the display order information of the random access image.

  The moving image predictive encoding program according to the present invention encodes a computer by an input means for inputting a plurality of images constituting a moving image, and the input image by any one of intra prediction or inter prediction. Generating compressed image data including a random access image that is an image to be random access, encoding means for encoding data relating to display order information of the image, and the generated compressed image data A restoration unit that restores a reproduced image by decoding the image, an image storage unit that stores the restored reproduction image as a reference image used to encode a subsequent image, and controls the image storage unit The memory management unit operates as a memory management unit, and after the encoding process for generating the random access image is completed, Immediately before or immediately after encoding an image whose order information is larger than the display order information of the random access image, out of the reference images stored in the image storage means, the reference image excluding the random access image is set as unnecessary. The image storage means is refreshed.

  The moving image predictive decoding program according to the present invention is a random access image obtained by encoding a plurality of images constituting a moving image by a method of intra prediction or inter prediction. Input means for inputting compressed image data including a random access image and display order encoded data obtained by encoding data relating to display order information of the image, and decoding the compressed image data And a restoration means for restoring display order information by decoding the display order encoded data, and the restored playback picture as a reference picture used for decoding subsequent pictures. An image storage means for storing and a memory management means for controlling the image storage means are operated, and the memory management means After the decoding process for decoding the access image is completed, first of all, the reference image stored in the image storage means immediately before or immediately after decoding the image whose display order information is larger than the display order information of the random access image The image storage means is refreshed by setting the reference image excluding the random access image to be unnecessary.

  The present invention as described above efficiently compresses and encodes images before and after an image that becomes a random access location, and at the same time, eliminates the disadvantages associated with the disadvantages of the prior art.

  In the present invention, information indicating a display order attached to each image constituting a moving image or compression-encoded image data (hereinafter referred to as “display order information” (display time, time reference information, temporal reference, etc. in the prior art) By setting the timing of memory refresh that is performed after the predicted image (intra frame) in the screen, which is the location of random access, by using a plurality of random access images before and after the random access image At the same time that the image is efficiently compression-encoded, the disadvantages associated with the drawbacks of the prior art can be eliminated as follows.

  That is, since the display order information is always attached to each image, it is not necessary to send new information (flag), and the disadvantage 2 of the prior art is solved.

  In addition, even when editing a moving image (for example, discarding some images or connecting another image), the display order information of each image constituting the moving image is set appropriately, which may cause a malfunction. There is no disadvantage of the prior art.

  Furthermore, the memory refresh timing according to the present invention is not limited to P pictures, and does not depend on the picture coding type (I picture, P picture, B picture), so coding is performed regardless of whether or not memory refresh is required. It is possible to process with the most efficient coding type, and the disadvantage 3 of the prior art is eliminated.

It is a functional block diagram which shows the structure of the moving image predictive coding apparatus which concerns on embodiment of this invention. It is a functional block diagram which shows the structure of the moving image predictive decoding apparatus which concerns on embodiment of this invention. It is a flowchart which shows the moving image predictive encoding / decoding method which concerns on embodiment of this invention. It is a schematic diagram for demonstrating the moving image predictive encoding / decoding method which concerns on embodiment of this invention. It is a flowchart which shows the moving image predictive encoding / decoding method which concerns on the modification of embodiment of this invention. It is a schematic diagram for demonstrating the moving image predictive encoding / decoding method which concerns on the modification of embodiment of this invention. It is a figure which shows the hardware constitutions of the computer for performing the program recorded on the recording medium. It is a general-view figure of the computer for performing the program memorize | stored in the recording medium. It is a block diagram which shows the structural example of a moving image predictive encoding program. It is a block diagram which shows the structural example of a moving image prediction decoding program. It is a schematic diagram which shows the prediction structure of the conventional moving image predictive encoding / decoding method.

  Hereinafter, embodiments of the present invention will be described with reference to FIGS.

[About video predictive coding apparatus]
FIG. 1 is a functional block diagram showing a configuration of a video predictive coding apparatus 100 according to an embodiment of the present invention. As illustrated in FIG. 1, the moving image predictive encoding device 100 includes an input terminal 101, a block divider 102, a prediction signal generator 103, a frame memory 104, a subtractor 105, a converter 106, a quantum, as a functional configuration. And a demultiplexer 107, an inverse quantizer 108, an inverse transformer 109, an adder 110, an entropy encoder 111, an output terminal 112, an input terminal 113, and a frame memory manager 114. The operation of each functional block will be described in the operation of the moving picture predictive encoding device 100 described later.

  Hereinafter, the operation of the video predictive coding apparatus 100 will be described. A moving image signal composed of a plurality of images to be encoded is input to an input terminal 101, and each image is divided into a plurality of regions by a block divider 102. In this embodiment, each image is divided into a plurality of blocks composed of 8 × 8 pixels, but may be divided into other block sizes or block shapes. Next, a prediction signal is generated by a prediction method to be described later for a block to be encoded (hereinafter referred to as “target block”). In the present embodiment, two types of prediction methods, inter-screen prediction and intra-screen prediction, can be used as the prediction method. In the inter-screen prediction, the bidirectional inter-screen prediction described in the background art can also be used. Yes. The basic operations of inter-screen prediction and intra-screen prediction are outlined below.

  In inter-screen prediction, a reproduced image that has been encoded in the past is used as a reference image, and motion information (for example, a motion vector) that gives a prediction signal with the smallest error with respect to the target block is obtained from this reference image. This process is called “motion detection”. Further, according to circumstances, the target block may be subdivided, and the inter-screen prediction method may be determined for the subdivided small area. In this case, among the various division methods, the division method of the small area and the motion information of each small area that are most efficient for the entire target block are determined. In the present embodiment, inter-screen prediction is performed by the prediction signal generator 103, and the target block is input to the prediction signal generator 103 via the line L102, and the reference image is input to the prediction signal generator 103 via the line L104. As the reference image, a plurality of images encoded and restored in the past are used as the reference image. The details thereof are MPEG-2, MPEG-4, H.264, which are conventional techniques. It is the same as any of the H.264 methods. The determined subregion division method information and the motion information of each subregion are sent from the prediction signal generator 103 to the entropy encoder 111 via the line L112 and encoded by the entropy encoder 111. The encoded data is sent from the output terminal 112 via the line L111. In addition, information (reference index) on which reference image a prediction signal is acquired from among a plurality of reference images is also sent from the prediction signal generator 103 to the entropy encoder 111 via the line L112, and is subjected to entropy encoding. The encoded data is sent from the output terminal 112 via the line L111. In the present embodiment, as an example, four to five reproduced images are stored in the frame memory 104 and used as reference images. The prediction signal generator 103 acquires a reference image from the frame memory 104 based on the small region dividing method, the reference image and motion information for each small region, and generates a prediction signal (inter-screen prediction) from the reference image and the motion information. Is generated in the sense of the prediction signal obtained by the above. The inter-screen prediction signal generated in this way is sent to the subtracter 105 and the adder 110 related to the processing described later via the line L103.

  On the other hand, in the intra prediction, an intra prediction signal is generated using already reproduced pixel values spatially adjacent to the target block. Specifically, the prediction signal generator 103 obtains a previously reproduced pixel signal in the same screen from the frame memory 104 and extrapolates the already reproduced pixel signal to obtain a prediction signal (obtained by intra prediction). In the meaning of the predicted signal). The generated intra-screen prediction signal is sent from the prediction signal generator 103 to the subtractor 105 via the line L103. A method for generating an intra-screen prediction signal in the prediction signal generator 103 is the conventional technique of H.264. This is the same as the H.264 method. Information relating to the extrapolation method in the intra prediction is sent from the prediction signal generator 103 to the entropy encoder 111 via the line L112, encoded by the entropy encoder 111, and then encoded data is It is sent out from the output terminal 112.

  The basic operations of inter-screen prediction and intra-screen prediction have been outlined above. Actually, for each target block, the one with the smallest error is selected from the inter-screen prediction signal and the intra-screen prediction signal obtained as described above, and the subtractor 105 is selected from the prediction signal generator 103 via the line L103. Sent to.

  By the way, since there is no image before the first image to be encoded, all target blocks in the first image are processed by intra prediction. Further, in preparation for switching of TV channels, all target blocks in a certain image are periodically processed by intra prediction as random access points. Such an image is called an intra frame. In H.264, this is called an IDR picture.

  The subtractor 105 generates a residual signal by subtracting the prediction signal received via the line L103 from the signal of the target block received via the line L102. This residual signal is subjected to discrete cosine transform by a transformer 106, and each transform coefficient is quantized by a quantizer 107. Finally, the quantized transform coefficient is encoded by the entropy encoder 111, and the obtained encoded data is sent from the output terminal 112 via the line L111 together with information on the prediction method.

  On the other hand, in order to perform intra prediction or inter prediction for the subsequent target block, the quantized transform coefficient (the encoded data of the target block) is dequantized by the inverse quantizer 108. Later, the inverse transformer 109 performs inverse discrete cosine transform, thereby restoring the residual signal. Then, the adder 110 adds the restored residual signal and the prediction signal sent from the line L103, thereby reproducing the signal of the target block, and the obtained reproduction signal is stored in the frame memory 104. . In this embodiment, the converter 106 and the inverse converter 109 are used. However, other conversion processes instead of these may be used. In some cases, the converter 106 and the inverse converter 109 may not be provided.

  Incidentally, the frame memory 104 is finite, and it is actually impossible to store all the reproduced images. Therefore, only the reproduced image used for encoding the subsequent image is stored in the frame memory 104. The frame memory manager 114 controls the frame memory 104. The frame memory manager 114 deletes the oldest reproduced image from N (for example, N = 4) reproduced images stored in the frame memory 104, thereby obtaining the latest reproduced image used as the reference image. The frame memory 104 is controlled so that it can be stored. Actually, display order information of each image and type information for encoding the image (intra-screen predictive coding, inter-screen predictive coding, bi-directional predictive coding) are input to the frame memory manager 114 from the input terminal 113. Based on these pieces of information, the frame memory manager 114 operates. At this time, the display order information of each image is sent from the frame memory manager 114 to the entropy encoder 111 via the line L114, encoded by the entropy encoder 111, and the encoded display order information is Along with the encoded image data, it is sent out from the output terminal 112 via the line L111. The display order information is attached to each image, and may be information indicating the order of the images or information indicating the time when the images are displayed (for example, display reference time (temporal reference) of the images). In the present embodiment, for example, the display order information is directly encoded by binary encoding. A control method by the frame memory manager 114 will be described later.

[About video predictive decoding apparatus]
Next, the video predictive decoding apparatus according to the present invention will be described. FIG. 2 is a functional block diagram showing the configuration of the video predictive decoding apparatus 200 according to the embodiment of the present invention. As shown in FIG. 2, the moving picture predictive decoding apparatus 200 has an input terminal 201, a data analyzer 202, an inverse quantizer 203, an inverse transformer 204, an adder 205, and a prediction signal generator 208 as functional configurations. A frame memory 207, an output terminal 206, and a frame memory manager 209. The operation of each functional block will be described in the operation of the moving picture predictive decoding apparatus 200 described later. Note that the decoding means is not limited to the inverse quantizer 203 and the inverse transformer 204, and other means may be used. Further, the means for decoding may be configured only by the inverse quantizer 203 without the inverse transformer 204.

  Hereinafter, the operation of the moving picture predictive decoding apparatus 200 will be described. The compressed data obtained by the encoding method described above is input from the input terminal 201. This compressed data includes the residual signal of the target block, information related to the generation of the prediction signal, quantization parameters, image display order information, and information related to the image encoding type. Among these, information relating to the generation of the prediction signal includes, for example, in the case of inter-screen prediction, information relating to block division (subregion division method information (eg, block size)), motion information of each small region, and reference index In the case of intra prediction, information regarding the extrapolation method is included.

  The data analyzer 202 extracts, from the input compressed data, information on the residual signal of the target block, information related to generation of a prediction signal, quantization parameters, image display order information, and image coding type. . Among these, the residual signal and quantization parameter of the target block are sent to the inverse quantizer 203 via the line L202, and the inverse quantizer 203 inverts the residual signal of the target block based on the quantization parameter. In addition, the inverse transformer 204 performs inverse discrete cosine transform on the result of the inverse quantization. The residual signal restored in this way is sent to the adder 205 via the line L204.

  On the other hand, information relating to the generation of the extracted prediction signal is sent to the prediction signal generator 208 via the line L206b. The prediction signal generator 208 acquires an appropriate reference image from a plurality of reference images in the frame memory 207 based on information related to the generation of the prediction signal, and generates a prediction signal based on the appropriate reference image. . The generated prediction signal is sent to the adder 205 via the line L208, and is added to the restored residual signal by the adder 205. As a result, the signal of the target block is reproduced. The reproduced signal of the target block is output from the output terminal 206 via the line L205 and simultaneously stored in the frame memory 207 as a reproduced image.

  The frame memory 207 stores a playback image used for decoding and playback of subsequent images. The frame memory manager 209 deletes the oldest reproduced image from N reproduced images stored in the frame memory 207 (here, N = 4 as an example, but may be a predetermined integer). Then, control is performed so that the latest reproduced image used as the reference image can be stored in the frame memory 207. The frame memory manager 209 operates based on the display order information of the target image input via the line L206a and information related to the image encoding type. A control method by the frame memory manager 209 will be described later.

  For intra frames (intra-screen predicted images) serving as random access points, H.264 is used. In H.264, called an IDR picture (instantaneous decoder refresh), this name is derived from refreshing an instantaneous frame memory (encoderane) immediately after encoding or decoding an IDR picture. On the other hand, according to the present invention, immediately after encoding or decoding an intra frame serving as a random access point (or immediately before encoding or decoding), the frame memory is not refreshed, but temporarily. In the present invention, this image is called a DDR picture (deferred decoder refresh or delayed decoder refresh) in order to refresh the frame memory after waiting (or delaying). As will be described in detail below, the refresh timing of the frame memory includes the display order information of the DDR picture, the display order information of the image (hereinafter referred to as “processing target image”) to be processed (encoding or decoding), and Is determined by comparing

[Characteristic Processing Operation of Video Predictive Encoding Method and Video Predictive Decoding Method]
Next, operations of the video predictive coding method and the video predictive decoding method according to the present invention will be described with reference to FIGS. 3 and 4. FIG. 3 is a flowchart showing a video predictive encoding / decoding method according to the present embodiment. In the following, FIG. 3 will be described as a video encoding method. However, FIG. 3 is also applicable to a moving image decoding method.

  First, the meaning of the variables used in FIG. 3 will be described. TR is the display order information, TR_DDR is the display order information of the DDR image, TR_CUR is the display order information of the processing target image at that time, and RP is a state variable indicating whether or not the refresh of the frame memory 104 is waiting. When RP = 1, this indicates a state in which the DDR picture is a processing target and the frame memory 104 has not yet been refreshed (that is, the frame memory refresh is waiting), and RP = 0. In this case, the frame memory 104 has already been refreshed or does not require refresh processing.

  In FIG. 3, when encoding of a moving image starts, first, TR_DDR and RP are initialized to 0 (step 301). In step 302, it is confirmed whether RP = 1 and TR_CUR is larger than TR_DDR. If this condition is satisfied, it means that the frame memory refresh is waiting and the image to be processed is an image after the DDR picture in the display order. A process of setting the reference image stored in the memory 104 as unnecessary is executed (step 303). However, the reference image set as unnecessary here is only the reference image whose display order information TR is smaller than the display order information (TR_DDR) of the latest DDR picture. Also, the latest DDR picture (or intra prediction encoded image) is not set as unnecessary. Upon completion of the refresh process as described above, the state variable RP is set to RP = 0.

  On the other hand, if the above condition is not satisfied in step 302, the process proceeds to step 304, and it is confirmed whether or not the processing target image at that time is a DDR picture. Note that in the video predictive coding apparatus 100, information on the picture coding type (DDR, inter-screen predictive coding or bi-directional predictive coding) is given from a control device (not shown) via the input terminal 113 in FIG. Shall be. If the processing target image at that time is a DDR picture in step 304, the display order information TR_CUR of the processing target image at that time is set to TR_DDR and the state variable RP is set to RP = 1 in step 305. The process proceeds to step 306. On the other hand, if the condition is not satisfied in step 304, the process proceeds to step 306.

  In step 306, a reproduced image corresponding to the processing target image is obtained. Here, with the encoding method described in FIG. 1, the processing target image is encoded to obtain compressed data, and the compressed data is further decoded to obtain a reproduction image (reproduction image corresponding to the processing target image). Note that the compressed data obtained by encoding is transmitted to the outside of the video predictive encoding device 100. Alternatively, the compressed data may be stored in a memory (not shown) in the moving picture predictive coding apparatus 100. In the next step 307, it is determined whether or not a reproduced image corresponding to the processing target image is used as a reference image in subsequent processing. This determination is made based on the coding type of the image. In the present embodiment, the DDR picture, the unidirectional predictive encoded image, and the specific bidirectional predictive encoded image are all reference images. However, the method is not limited to this method.

  If it is determined in step 307 that the reproduced image is not used as a reference image, the process proceeds to step 309 without storing the reproduced image in the frame memory 104. On the other hand, if it is determined in step 307 that the reproduced image is used as a reference image, the reproduced image is stored in the frame memory 104 in step 308 and then the process proceeds to step 309.

  In step 309, it is determined whether or not there is a next image (unprocessed image). If there is a next image, the process returns to step 302, and the processing in steps 302 to 308 is repeated for the next image. In this way, the processing in steps 302 to 308 is repeated until the last image, and when all the images are processed, the processing in FIG. 3 is terminated.

  After the processing of the random access image (here, the latest DDR picture) is completed by the processing of FIG. 3 described above, when processing an image whose display order information TR is larger than TR_DDR for the first time (actually, in step 306) In step 303 immediately before processing), the frame memory is refreshed. Note that the refresh timing of the frame memory may be any time when an image whose display order information TR is larger than TR_DDR is processed first after processing of a random access image (here, the latest DDR picture) is completed. It may be immediately after the process of 306.

  The above-described processing of FIG. 3 corresponds to the processing of the entire moving picture predictive encoding apparatus 100 of FIG. 1, and in particular, the processing of steps 302 to 305 is performed by the frame memory manager 114.

  Although FIG. 3 has been described as a moving image encoding method, it can also be applied to processing of a moving image decoding method. In the case of performing the decoding process, in step 301, the data (bit stream) of the compressed and encoded image is further input. The display order information and the encoding type of the target image are extracted from the data, and the control in steps 302 to 305 is performed by the same method as described above. In step 306, the compressed data of the target image is decoded and the image is restored. The processing after step 307 is as described above. Such processing corresponds to the processing of the entire moving picture predictive decoding apparatus 200 in FIG. 2, and in particular, the processing in steps 302 to 305 is performed by the frame memory manager 209.

  FIG. 4 is a schematic diagram for explaining processing of the video predictive encoding / decoding method according to the present embodiment. Images 401 to 409 shown in FIG. 4 are a part of an image group constituting a moving image, and an image 401 shows a state preceded by n images. Therefore, the display order information TR of the image 401 is represented as (n + 1) as shown in a region 418 in FIG. Further, in this embodiment, since encoding / decoding processing including bi-directional prediction is performed, images 403, 404, and 405 that precede the image 402 in display order after the TR = (n + 5) image 402 is processed first. It shows how to process. For the same reason, the image 403 whose display order is (n + 3) is processed before the image 404 whose display order is (n + 2). Such an order is the same as in FIG. In the following, “processing an image” means “encoding or decoding an image”.

  The English characters written in the frames of the images 401 to 409 in FIG. 4 mean the following. That is, P means an image encoded by unidirectional prediction, DDR means an image encoded as a DDR picture, and B and b mean images encoded by bidirectional prediction. In addition, images other than the image indicated by the lowercase letter b (that is, images indicated by the uppercase letters B, P, and DDR) are all used as reference images. The value of RP for each image shown in the area 420 of FIG. 4 and the value of TR_DDR shown in the area 419 are values immediately after the processing for each image is completed. It is not the value when entering step 302. For example, RP = 0 at the start of processing for the image 402, but immediately after completion of processing for the image 402, RP = 1.

  In the processing of the image 401, since the image 401 is not a DDR picture, RP = 0. TR_DDR corresponding to the image 401 may take an arbitrary value, but a value stored by the preceding process is set. The image 401 indicated by the capital letter P1 is used as a reference image and is stored in the frame memory.

  Next, the processing of the image 402 will be described with reference to FIG. At this time, the reproduced image P1 is stored in the frame memory, as indicated by an area 410 at the bottom of FIG. Since RP = 0 at the start of the processing of the image 402, a negative determination is made in step 302, and the process proceeds to step 304. Since the image 402 is a DDR picture, an affirmative determination is made in step 304, and RP = 1 and TR_DDR = n + 5 are set in step 305. The image 402 is stored in the frame memory because it is used as a reference image.

  Next, when processing of the image 403 starts, as shown in an area 411 in FIG. 4, the images P1 and DDR5 are stored in the frame memory. At this time, although RP = 1, the display order TR (n + 3) of the image 403 is smaller than TR_DDR (n + 5) and the image 403 is not a DDR picture. Decoded (step 306). The image 403 is stored in the frame memory because it is used as a reference image.

  Similarly, when the images 404 and 405 are processed, the frame memory refresh remains in the standby state (RP = 1). Further, since the images 404 and 405 are not used as reference images, the images 404 and 405 are not stored in the frame memory as shown in the areas 412 and 413 in FIG. 4, and the images P1, DDR5, and B3 are stored. .

  Although RP = 1 at the time when the processing of the image 406 starts, the display order information TR (n + 9) of the image 406 is larger than TR_DDR (n + 5), so an affirmative determination is made in step 302 and the reference image is selected in step 303. The frame memory is refreshed by setting it as unnecessary, and RP = 0 is set. At this time, the reference images set as unnecessary are only reference images having display order information TR smaller than that of the latest DDR picture 402, excluding the latest DDR picture 402. Therefore, as shown in an area 414 in FIG. 4, in the frame memory, the storage areas for the image P1 and the image B3 are released, and only the image DDR5 is stored. As shown in an area 415 in FIG. 4, the image 406 is stored in the frame memory after the processing of the image 406 is completed, and thereafter, the refresh control of the frame memory is performed in the same manner as described above.

  In this way, immediately after or immediately before the processing of the DDR picture 402, the reference image (image P1 in FIG. 4) in the frame memory is not set to be unnecessary, so the processing of the images 403, 404, and 405 processed after the DDR picture 402 is performed. The image P1 can be referred to at this time, which can contribute to the improvement of the coding efficiency. Also, when frame memory refresh is executed after processing of the DDR picture 402, the latest DDR picture 402 (image DDR5) is not set to be unnecessary, so that the latest DDR picture is processed in the subsequent processing of the images 407, 408, and 409. 402 (image DDR5) can be used as a reference image.

  As described above, the present embodiment uses the display order information attached to the image, and uses the display order information to indicate the timing of the memory refresh performed after the processing of the intra prediction image (DDR picture) that is a random access location. By setting, it is possible to efficiently compress and encode images before and after the random access image. Moreover, the inconvenience which concerns on the fault of a prior art can be eliminated as follows.

  That is, since the display order information is always attached to each image, it is not necessary to send new information (flag), and the disadvantage 2 of the prior art is solved. In addition, even when editing a moving image (for example, discarding some images or connecting another image), the display order information of each image constituting the moving image is set appropriately, which may cause a malfunction. There is no disadvantage of the prior art. Furthermore, the memory refresh timing according to the present invention is not limited to P pictures, and does not depend on the picture coding type (I picture, P picture, B picture), so coding is performed regardless of whether or not memory refresh is required. It is possible to process with the most efficient coding type, and the disadvantage 3 of the prior art is eliminated.

[Modification]
In the embodiment described above, the processing in the case where the image display order information is encoded as “absolute value” has been described. However, in order to increase the encoding efficiency, the image display order information is encoded as “difference value”. There is also an embodiment. Hereinafter, an embodiment in which the display order information is encoded as “difference value” will be described as a modification.

  FIG. 5 shows a flowchart of a modification of the moving picture predictive encoding / decoding method. In the modification, the image display order information is encoded as follows. That is, for an image to be processed while waiting for refresh of the frame memory, the difference value between the display order information and the display order information of the DDR picture is encoded. On the other hand, for images other than those described above, the display order information is encoded by an arbitrary method. For example, the difference from the display order information of the DDR picture may be encoded, or the difference from the display order information of the immediately preceding image in the encoding order may be encoded.

  In the following modification, FIG. 5 will be described as a moving picture decoding method, but of course, FIG. 5 is also applicable to a moving picture encoding method. In step 501 of FIG. 5, compression-encoded image data is input to the moving image predictive decoding device 200, and information regarding the difference value (delta_TR) of the display order information of the target image and the image encoding type from the data. To extract. At the same time, TR_DDR and RP are initialized to 0.

  In the next step 502, it is confirmed whether RP = 1. If this condition is satisfied, it means that the frame memory refresh is waiting, and the process proceeds to step 503. In step 503, the display order information TR_CUR of the processing target image at that time is set to the sum (added value) of TR_DDR and delta_TR.

  Next, in step 504, it is confirmed whether TR_CUR is larger than TR_DDR. If this condition is satisfied, it means that the frame memory is in a refresh standby state and the image to be processed is an image after the DDR picture in the display order. The process of setting the reference image stored in 207 as unnecessary is executed (step 505). However, the reference image set as unnecessary here is only the reference image whose display order information TR is smaller than the display order information (TR_DDR) of the latest DDR picture. Also, the latest DDR picture (or intra prediction encoded image) is not set as unnecessary. Upon completion of the refresh process as described above, the state variable RP is set to RP = 0. Then, it progresses to step 507 mentioned later. In addition, if a negative determination is made in step 504 described above, the process also proceeds to step 507.

  On the other hand, if a negative determination is made in step 502, the process proceeds to step 506, TR_CUR is set to the sum (added value) of the display order information TR_PREV and delta_TR of the previously processed image, and the process proceeds to step 507.

  In step 507, it is confirmed whether or not the processing target image at that time is a DDR picture. Note that the moving picture predictive decoding apparatus 200 can obtain information related to an image coding type (DDR, inter-screen predictive coding or bi-directional predictive coding) from compression-coded data input from the outside.

  If the processing target image at the time is a DDR picture at step 507, the display order information TR_CUR of the processing target image at the time is set to TR_DDR and the state variable RP is set to RP = 1 at step 508. The process proceeds to step 509. On the other hand, if the condition is not satisfied in step 507, the process proceeds to step 509.

  In step 509, a reproduced image corresponding to the processing target image is obtained. Here, a reproduced image corresponding to the processing target image is obtained by decoding the compressed data of the processing target image by the decoding method described in FIG. Note that the reproduced image obtained here is transmitted to the outside of the moving picture predictive decoding apparatus 200, for example. In the next step 510, it is determined whether or not a reproduced image corresponding to the processing target image is used as a reference image in subsequent processing. This determination is made based on the coding type of the image. Here, the DDR picture, the unidirectional predictive encoded image, and the specific bidirectional predictive encoded image are all reference images. However, the method is not limited to this method.

  If it is determined in step 510 that the reproduced image is not used as a reference image, the process proceeds to step 512 without storing the reproduced image in the frame memory 207. On the other hand, if it is determined in step 510 that the reproduced image is used as a reference image, the reproduced image is stored in the frame memory 207 in step 511 and then the process proceeds to step 512.

  In step 512, TR_CUR is set to TR_PREV for the processing in the subsequent step 506, and the process proceeds to step 513. In step 513, it is determined whether or not there is a next image (unprocessed image). If there is a next image, the process returns to step 502, and the processes in steps 502 to 512 are repeated for the next image. In this way, the processing of steps 502 to 512 is repeated until the last image, and when the processing of all the images is completed, the processing of FIG. 5 ends.

  After the processing of the random access image (here, the latest DDR picture) is completed by the processing of FIG. 5 described above, first, when processing an image whose display order information TR is larger than TR_DDR (actually, in step 509) In step 505 immediately before processing), the frame memory is refreshed. Note that the refresh timing of the frame memory may be any time when an image whose display order information TR is larger than TR_DDR is processed first after processing of a random access image (here, the latest DDR picture) is completed. It may be immediately after the processing of 509.

  The above-described processing of FIG. 5 corresponds to the entire processing of the moving picture predictive decoding apparatus 200 of FIG. 2, and in particular, steps 502 to 508 are performed by the frame memory manager 209.

  In addition, although FIG. 5 demonstrated as a moving image decoding method, it is applicable also to the process of a moving image encoding method. When performing the encoding process, in step 503, delta_TR is obtained from the difference between TR_CUR and TR_DDR, and in step 506, delta_TR is obtained from the difference between TR_CUR and TR_PREV and then entropy-encoded. In step 509, the target image is encoded and then decoded. Such processing corresponds to the processing of the entire moving picture predictive coding apparatus 100 in FIG. 1, and the processing of steps 502 to 508 is performed by the frame memory manager 114.

  FIG. 6 is a schematic diagram for explaining the process of the video predictive encoding / decoding method according to the modification. Images 601 to 609 shown in FIG. 6 are part of an image group constituting a moving image, and show the same processing as the images 401 to 409 described in FIG. However, in FIG. 6, delta_TR shown in the region 621 is added to FIG. As can be seen from this region 621, the method of obtaining delta_TR differs depending on the value of RP at the start of encoding processing of the target image (RP value of the previous image). That is, in the encoding process of the images 603 to 606, delta_TR is obtained as a difference value between TR of each image and TR_DDR. In the encoding process after the image 607, delta_TR is obtained as a difference value between the TR of each image and the TR of the immediately preceding image. On the other hand, when the display order information TR is restored from the difference value delta_TR in the decoding process of each image, the display order information TR is obtained by adding the difference value delta_TR and TR_DDR obtained by decoding the compressed data of the difference value. To restore. The subsequent processing is the same as in FIG.

  In FIG. 6, even if the images 603 to 605 are lost due to editing, the display order information TR of the image 606 is obtained from TR_DDR. Can be controlled without malfunction. If the delta_TR of all the images is obtained as a difference value between the display order information of the image and the display order information of the immediately preceding image in the decoding order, the display order information is correctly reproduced when the image 603 is missing. As a result, the frame memory is refreshed at the timing of the image 605 (originally, the timing of the image 606 is the correct timing).

  When FIG. 6 is applied to the moving image encoding process, the display order information of the images (images 603 to 606) waiting for the frame memory refresh after the processing of the random access image (here, the latest DDR picture) is completed. At the time of encoding, the frame memory refresh timing is encoded by encoding the difference value delta_TR between the display order information TR of the image and the display order information TR_DDR of the DDR picture instead of the display order information TR of the image itself. Can be restored correctly. For this reason, even if an image that is waiting for frame memory refresh is dropped, malfunctions can be avoided and the error tolerance is strong.

  As yet another example, the difference value delta_TR is to be encoded as an image first after the random access image (here, the latest DDR picture) and whose display order information TR is first larger than TR_DDR (image 606 in FIG. 6). One or more images may be included. That is, for one or more images including first an image having display order information TR larger than TR_DDR (image 606 in FIG. 6) after a random access image (here, the latest DDR picture), the display order information is encoded. At the time of conversion, the difference value delta_TR between the display order information TR of the image and the display order information TR_DDR of the DDR picture may be encoded instead of the display order information TR of the image.

[About a video predictive encoding program and a video predictive decoding program]
The invention relating to the moving picture predictive coding apparatus can be understood as an invention relating to a moving picture predictive coding program for causing a computer to function as a moving picture predictive coding apparatus. Similarly, the invention relating to the moving picture predictive decoding apparatus can be regarded as an invention relating to a moving picture predictive decoding program for causing a computer to function as a moving picture predictive decoding apparatus.

  The moving picture predictive encoding program and the moving picture predictive decoding program are provided by being stored in a recording medium, for example. Examples of the recording medium include a recording medium such as a flexible disk, a CD-ROM, a DVD, a recording medium such as a ROM, or a semiconductor memory.

  FIG. 9 shows a module of a moving picture predictive coding program for causing a computer to function as a moving picture predictive coding apparatus. As shown in FIG. 9, the moving picture predictive encoding program P100 includes an input module P101, an encoding module P102, a restoration module P103, an image storage module P104, and a memory management module P105.

  FIG. 10 shows a module of a moving picture predictive decoding program for causing a computer to function as a moving picture predictive decoding apparatus. As shown in FIG. 10, the moving picture predictive decoding program P200 includes an input module P201, a restoration module P202, an image storage module P203, and a memory management module P204.

  The moving picture predictive encoding program P100 and the moving picture predictive decoding program P200 configured as described above can be stored in the recording medium 10 shown in FIG. 8, and are executed by the computer 30 described later.

  FIG. 7 is a diagram showing a hardware configuration of a computer for executing a program recorded on the recording medium, and FIG. 8 is an overview diagram of the computer for executing the program stored on the recording medium. Examples of the computer include a DVD player, a set top box, a mobile phone, etc. that have a CPU and perform processing and control by software.

  As shown in FIG. 7, the computer 30 includes a reading device 12 such as a flexible disk drive device, a CD-ROM drive device, and a DVD drive device, a working memory (RAM) 14 in which an operating system is resident, and a recording medium 10. A memory 16 for storing programs stored therein, a display device 18 such as a display, a mouse 20 and a keyboard 22 as input devices, a communication device 24 for transmitting and receiving data, and a CPU 26 for controlling execution of the programs. And. When the recording medium 10 is inserted into the reading device 12, the computer 30 can access the moving image predictive encoding program stored in the recording medium 10 from the reading device 12, and the moving image predictive encoding program can It becomes possible to operate as a moving picture predictive encoding apparatus according to the invention. Similarly, when the recording medium 10 is inserted into the reading device 12, the computer 30 can access the moving picture predictive decoding program stored in the recording medium 10 from the reading apparatus 12, and the moving picture predictive decoding program can It becomes possible to operate as a moving picture predictive decoding apparatus according to the present invention.

  As shown in FIG. 8, the moving picture predictive encoding program or the moving picture predictive decoding program may be provided via a network as a computer data signal 40 superimposed on a carrier wave. In this case, the computer 30 can store the moving picture predictive encoding program or the moving picture predictive decoding program received by the communication device 24 in the memory 16 and execute it.

  DESCRIPTION OF SYMBOLS 10 ... Recording medium, 30 ... Computer, 100 ... Moving image predictive coding apparatus, 101 ... Input terminal, 102 ... Block divider, 103 ... Prediction signal generator, 104 ... Frame memory, 105 ... Subtractor, 106 ... Converter , 107 ... quantizer, 108 ... inverse quantizer, 109 ... inverse transformer, 110 ... adder, 111 ... entropy encoder, 112 ... output terminal, 113 ... input terminal, 114 ... frame memory manager, 200 DESCRIPTION OF SYMBOLS ... Moving picture predictive decoding apparatus 201 ... Input terminal 202 ... Data analyzer 203 ... Inverse quantizer 204 ... Inverse transformer 205 ... Adder 206 ... Output terminal 207 ... Frame memory 208 ... Predictive signal Generator, 209 ... frame memory manager, P100 ... moving picture predictive encoding program, P101 ... input module, P102 ... encoding module, 103 ... restoration module, P104 ... image storing module, P105 ... memory management module, P200 ... moving picture prediction decoding program, P201 ... input module, P202 ... restoration module, P203 ... image storing module, P204 ... memory management module.

Claims (6)

Input means for inputting a plurality of images constituting a moving image;
The input image is encoded by either intra-screen prediction or inter-screen prediction, thereby generating compressed image data including a random access image that is a random access image, and displaying the image Encoding means for encoding data relating to the order information;
Restoring means for restoring a reproduced image by decoding the generated compressed image data;
Image storage means for storing the reconstructed reproduced image as a reference image used for encoding a subsequent image;
The restoration means includes
As a reference image used when encoding an image whose display order information is larger than the display order information of a random access image, a reference image preceding the random access image is not included in the encoding process order or the display order information. Decoding the compressed image data including various reference image information;
A video predictive coding apparatus characterized by the above. Compressed image data including a random access image which is a random access image obtained by encoding a plurality of images constituting a moving image by either intra prediction or inter prediction method, and An input means for inputting display order encoded data obtained by encoding data relating to display order information of an image;
A restoration unit that restores a reproduction image by decoding the compressed image data, and that restores display order information by decoding the display order encoded data;
Image storage means for storing the reconstructed reproduced image as a reference image used for decoding subsequent images;
The restoration means includes
The reference image used when decoding the compressed image data of the image whose display order information is larger than the display order information of the random access image includes a reference image preceding the random access image in the decoding process order or the display order information. Decoding the compressed image data including no such reference image information;
A video predictive decoding apparatus characterized by the above. A video predictive encoding method executed by a video predictive encoding device comprising an image storage means for storing a reference image used for encoding a subsequent image,
An input step for inputting a plurality of images constituting the moving image;
The input image is encoded by either intra-screen prediction or inter-screen prediction, thereby generating compressed image data including a random access image that is a random access image, and displaying the image An encoding step for encoding data relating to the order information;
A restoration step of restoring a reproduced image by decoding the generated compressed image data;
An image storage step of storing the restored reproduced image in the image storage means as a reference image used for encoding a subsequent image;
In the restoration step, the video predictive encoding device is
As a reference image used when encoding an image whose display order information is larger than the display order information of a random access image, a reference image preceding the random access image is not included in the encoding process order or the display order information. Decoding the compressed image data including various reference image information;
A video predictive encoding method characterized by the above. A moving image predictive decoding method executed by a moving image predictive decoding device including an image storage means for storing a reference image used for decoding a subsequent image,
Compressed image data including a random access image which is a random access image obtained by encoding a plurality of images constituting a moving image by either intra prediction or inter prediction method, and An input step of inputting display order encoded data obtained by encoding data related to display order information of an image;
A restoration step of restoring the reproduction image by decoding the compressed image data and restoring the display order information by decoding the display order encoded data;
An image storage step of storing the restored reproduced image in the image storage means as a reference image used for decoding a subsequent image;
In the restoration step, the video predictive decoding device
The reference image used when decoding the compressed image data of the image whose display order information is larger than the display order information of the random access image includes a reference image preceding the random access image in the decoding process order or the display order information. Decoding the compressed image data including no such reference image information;
A video predictive decoding method characterized by the above. Computer
Input means for inputting a plurality of images constituting a moving image;
The input image is encoded by either intra-screen prediction or inter-screen prediction, thereby generating compressed image data including a random access image that is a random access image, and displaying the image Encoding means for encoding data relating to the order information;
Restoring means for restoring a reproduced image by decoding the generated compressed image data;
Image storage means for storing the restored reproduced image as a reference image used for encoding a subsequent image;
Act as
The restoration means includes
As a reference image used when encoding an image whose display order information is larger than the display order information of a random access image, a reference image preceding the random access image is not included in the encoding process order or the display order information. Decoding the compressed image data including various reference image information;
A moving picture predictive encoding program characterized by the above. Computer
Compressed image data including a random access image which is a random access image obtained by encoding a plurality of images constituting a moving image by either intra prediction or inter prediction method, and An input means for inputting display order encoded data obtained by encoding data relating to display order information of an image;
A restoration unit that restores a reproduction image by decoding the compressed image data, and that restores display order information by decoding the display order encoded data;
Image storage means for storing the restored reproduced image as a reference image used for decoding subsequent images;
Act as
The restoration means includes
The reference image used when decoding the compressed image data of the image whose display order information is larger than the display order information of the random access image includes a reference image preceding the random access image in the decoding process order or the display order information. Decoding the compressed image data including no such reference image information;
A moving picture predictive decoding program characterized by the above.

Images