JP5898924B2 - Video predictive encoding method, video predictive encoding device, video predictive encoding program, video predictive decoding method, video predictive decoding device, and video predictive decode program - Google Patents

Description

  One aspect of the present invention relates to a moving picture predictive coding method, apparatus, and program, and a moving picture predictive decoding method, apparatus, and program, and more particularly, in a buffer used for predictive coding between screens. The present invention relates to management of a certain reference image.

  A compression encoding technique is used to efficiently transmit and store moving image data. Specifically, MPEG1-4 and H.264. 261-H. H.264 is widely used. In these encoding schemes, encoding and decoding processes are performed after an image to be encoded is divided into a plurality of blocks. In order to increase coding efficiency, intra prediction encoding or inter prediction encoding is used.

  In intra prediction encoding, a prediction signal is generated using an adjacent reproduced image signal (image signal obtained by restoring image data compressed in the past) in the same screen as the target block. Then, the difference signal is generated by subtracting the prediction signal from the signal of the target block, and the difference signal is encoded. In inter-screen predictive coding, a displacement of a signal is searched with reference to an already reproduced image signal in a screen different from that of the target block, and the displacement (motion) is compensated to generate a prediction signal. Then, the difference signal is generated by subtracting the prediction signal from the signal of the target block, and the difference signal is encoded. An already-reproduced image that is referred to for motion search and compensation is referred to as a reference image.

  H. In the H.264 inter-frame prediction encoding, a plurality of reference images that have been encoded and reproduced in the past are referred to, and an image signal with the least error is selected as an optimal prediction signal for the target block by motion search. Subsequently, the difference between the pixel signal of the target block and its optimal prediction signal is calculated. Then, discrete cosine transform and quantization are performed on the difference signal, and the quantized signal is entropy-coded. Simultaneously with such processing, information on which reference image the optimal prediction signal for the target block is acquired from (referred to as “reference index”), and information on which region of the reference image the optimal prediction signal is acquired from (Referred to as “motion vector”) is also encoded. H. In H.264, four to five reproduced images are stored as reference images in a frame memory or a reproduced picture buffer (DPB).

  As a method for managing a plurality of reference images, there are an adaptive memory management method and a moving window memory management method. The adaptive memory management method is a method for explicitly instructing management related to storage and release of reference images. On the other hand, the moving window memory management method deletes the oldest reference image (that is, the reference image existing in the buffer for the longest time) from a plurality of reference images, and refers to the decoded reproduced image. It is an implicit method of storing as an image. These two types of management methods relating to reference images are described in Non-Patent Document 1 below.

Supervised by Satoshi Okubo, Shinya Tsuno, Yoshihiro Kikuchi, Teruhiko Suzuki, Impress Standard Textbook Series. H.264 / AVC textbook, Impressnet Business Company, 2004, p. 171-172

  However, in the adaptive memory management method, it is necessary to explicitly send memory management information to the decoding device, and the amount of code increases accordingly. On the other hand, in the moving window memory management method, it is not necessary to explicitly send management information, but it is not possible to flexibly specify a reference image with excellent quality. The compression rate can be increased by predictive encoding using a high-quality reference image (generally a finely quantized reference image having excellent image quality). However, in the moving window memory management method, Since the high-quality reference image is first deleted from the buffer, it is difficult to predictively encode the subsequent image with high efficiency.

  Therefore, it is possible to store a high-quality reference image in a buffer for a longer time without explicitly instructing storage and release of the reference image, and to use the reference image for encoding more target images. There is a demand for predictive coding at a compression rate.

A moving image predictive encoding device according to an aspect of the present invention includes an input unit that receives a plurality of target images constituting a moving image, and at least one image that has been encoded and decoded and reproduced as a reference image. Each of the target images is used for predictive encoding, and output order information indicating the output order of the target images subjected to the predictive encoding, and the dependency of the other images on the target image when other images are predictively encoded. Encoding means for generating compressed image data by adding dependency information to the target image, decoding means for decoding the compressed image data to generate a reproduced image, and encoding the reproduced image and subsequent images a buffer for storing a reference image to be used for, and a buffer managing means for controlling a buffer, the buffer management means, among the plurality of reference images stored in the buffer, depending A reference image whose output order based on the output order information is smaller than the output order of the target image that is the target of predictive encoding is identified from among the reference images having the lowest dependency according to the information. Of the reference images, the reference image having the smallest output order based on the output order information is determined as a reference image to be deleted from the plurality of reference images.

A video predictive encoding method according to an aspect of the present invention is a video predictive encoding method executed by a video predictive encoding device, and an input step of receiving a plurality of target images constituting a video, Output order information indicating the output order of the target images that have been predictively encoded using at least one image that has been previously encoded and decoded and reproduced as a reference image, and other images An encoding step for generating compressed image data by adding dependency information indicating the dependency of the other image to the target image to the target image, and decoding the compressed image data A decoding step for generating a reproduction image, a storage step for storing the reproduction image in a buffer as a reference image used for encoding a subsequent image, and a buffer for controlling the buffer And a management step, in the buffer management step, among the plurality of reference images stored in the buffer, from among the lowest reference image dependence according to dependency information, based on said output order information output order Identifies a reference image that is smaller than the output order of the target image that is the target of predictive coding, and among the identified reference images, the reference image that has the smallest output order based on the output order information is selected from the reference images. It is determined as a reference image to be deleted from inside.

A moving image predictive encoding program according to an aspect of the present invention includes an input unit that receives a plurality of target images constituting a moving image, and at least one image decoded and reproduced after being encoded in the past as a reference image. Each of the target images is used for predictive encoding, and output order information indicating the output order of the target images subjected to the predictive encoding, and the dependency of the other images on the target image when other images are predictively encoded. Encoding means for generating compressed image data by adding dependency information to the target image, decoding means for decoding the compressed image data to generate a reproduced image, and encoding the reproduced image and subsequent images A buffer for storing a reference image used for processing and a buffer management means for controlling the buffer are executed by the computer, and the buffer management means is stored in the buffer. Of the reference image, from among the dependence lowest reference image in accordance with the dependence information, the output order based on the output order information is smaller than the output sequence of the target image is a prediction encoding of the target A reference image is specified, and a reference image having a minimum output order based on output order information among the specified reference images is determined as a reference image to be deleted from the plurality of reference images.

A video predictive decoding apparatus according to an aspect of the present invention is an input unit that receives compressed image data for each of a plurality of target images constituting a moving image, and the compressed image data is decoded and reproduced in the past. Data that is predictively encoded using at least one image as a reference image, output order information that indicates the output order of the target image, and the other image relative to the target image when predictively encoding another target image. The input means including dependency information indicating the dependency, decoding means for decoding the compressed image data to generate a reproduction image, and the reproduction image as a reference image used for decoding the subsequent image dependent with a buffer, and a buffer managing means for controlling a buffer, the buffer management means, among the plurality of reference images stored in the buffer, in accordance with the dependence information Among the reference images having the lowest output order based on the output order information, the reference order is smaller than the output order of the target picture that is the target of decoding, and the output order information of the identified reference pictures The reference image with the smallest output order is determined as a reference image to be deleted from the plurality of reference images.

A moving image predictive decoding method according to an aspect of the present invention is a moving image predictive decoding method executed by a moving image predictive decoding device, and receives compressed image data for each of a plurality of target images constituting a moving image. An input step, in which the compressed image data is predictively encoded using at least one image decoded and reproduced in the past as a reference image, output order information indicating the output order of the target image, other The input step including dependency information indicating the dependency of the other image with respect to the target image when predictively encoding the target image, and a decoding step of decoding the compressed image data to generate a reproduced image; A storage step for storing the reproduced image in a buffer as a reference image used for decoding subsequent images, and a buffer management step for controlling the buffer. In the buffer management step, among the plurality of reference images stored in the buffer, from among the dependence lowest reference image in accordance with the dependence information, the output order based on the output order information, the decoding of the target identify the smaller the reference image than the output sequence of the target image is, the output order is the smallest of the reference image based on the output order information of the specified reference images, as a reference image to be deleted from among the plurality of reference images decide.

A moving image predictive decoding program according to an aspect of the present invention is an input unit that receives compressed image data for each of a plurality of target images constituting a moving image, and the compressed image data is decoded and reproduced in the past. Data that is predictively encoded using at least one image as a reference image, output order information that indicates the output order of the target image, and the other image relative to the target image when predictively encoding another target image. The input means including dependency information indicating the dependency, decoding means for decoding the compressed image data to generate a reproduction image, and the reproduction image as a reference image used for decoding the subsequent image a buffer for, by executing the buffer management means for controlling a buffer in the computer, the buffer management means, among the plurality of reference images stored in the buffer A reference image whose output order based on the output order information is smaller than the output order of the target image to be decoded is identified from the reference images having the lowest dependency according to the dependency information. Of the reference images, the reference image having the smallest output order based on the output order information is determined as a reference image to be deleted from the plurality of reference images.

According to such a form, the reference image to be deleted is determined based on the output order information and dependency information of the reference image stored in the buffer. Therefore, it is possible to store a reference image with excellent quality in a buffer for a longer period without explicitly sending management information, and use the reference image for encoding or decoding more target images. As a result, predictive coding can be performed at a high compression rate.

  According to one aspect of the present invention, a high-quality reference image is stored in a buffer for a longer time without explicitly instructing storage and release of the reference image, and the reference image is encoded with more target images. By using this, predictive coding can be performed at a high compression rate.

It is a block diagram which shows the moving image predictive coding apparatus which concerns on embodiment. It is a block diagram which shows the moving image predictive decoding apparatus which concerns on embodiment. It is a flowchart which shows the outline | summary of an encoding process and a decoding process. 4 is a flowchart showing details of management of the reference image shown in FIG. 3. It is a figure which shows the 1st example of the management method of the reference image in embodiment. It is a figure which shows the 2nd example of the management method of the reference image in embodiment. It is a block diagram which shows the moving image predictive encoding program which concerns on embodiment. It is a block diagram which shows the moving image prediction decoding program which concerns on embodiment. It is a figure which shows the hardware constitutions of the computer with which a program is performed. FIG. 10 is a perspective view of the computer shown in FIG. 9.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant description is omitted.

  First, a video predictive encoding apparatus (hereinafter simply referred to as “encoding apparatus”) 100 according to the embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing an encoding apparatus 100. The encoding apparatus 100 includes a first input terminal (input means) 101, a block divider 102, a prediction signal generator 103, a frame memory (also referred to as a buffer or DPB) 104, a subtractor 105, a converter 106, a quantizer 107, An inverse quantizer 108, an inverse transformer 109, an adder 110, an entropy encoder 111, an output terminal 112, a second input terminal 113, and a buffer manager (buffer management means) 114 are provided. The subtractor 105, the converter 106, and the quantizer 107 correspond to encoding means. The inverse quantizer 108, the inverse transformer 109, and the adder 110 correspond to decoding means.

  The first input terminal 101 is means for receiving a moving image signal composed of a plurality of images and outputting it to the block divider 102.

  The block divider 102 is means for dividing an image to be encoded (target image) represented by a signal input from the first input terminal 101 into a plurality of regions. An area generated by this division is called a “target block”. In the present embodiment, the block divider 102 divides the image into blocks composed of 8 × 8 pixels, but the image may be divided into other block sizes or shapes. The block divider 102 outputs the signal of the target block to the prediction signal generator 103 and the subtractor 105.

  The prediction signal generator 103 is means for generating a prediction signal for the target block that is the target of the encoding process. In the present embodiment, the prediction signal generator 103 generates a prediction signal using two kinds of prediction methods, that is, inter-screen prediction and intra-screen prediction.

  In inter-screen prediction, the prediction signal generator 103 obtains motion information that gives a prediction signal with the smallest error for the target block from the reference image, using a reproduced image that has been encoded in the past as a reference image. Depending on the case, the prediction signal generator 103 may subdivide the target block and determine an inter-screen prediction method for the subdivided small regions. In this case, the prediction signal generator 103 determines the most efficient division method for the entire target block from various division methods, and determines the motion information of each small region.

  The prediction signal generator 103 uses the target block input from the block divider 102 and the reference image read from the frame memory 104. The prediction signal generator 103 uses one or more images encoded and restored in the past as reference images. Details of the method of determining motion information and the method of dividing a small region are described in H.264, which is a conventional technology. This is the same as the H.264 method. The prediction signal generator 103 acquires a reference image from the frame memory 104 based on the small region dividing method, the reference image corresponding to each small region, and the motion information, and generates a prediction signal.

  On the other hand, in the intra prediction, the prediction signal generator 103 generates an intra prediction signal using the already reproduced pixel values spatially adjacent to the target block. Specifically, the prediction signal generator 103 obtains already reproduced pixel signals in the same screen from the frame memory 104, and extrapolates these signals to generate an in-screen prediction signal. A method for generating an intra-screen prediction signal is the conventional technique of H.264. This is the same as the H.264 method.

  The prediction signal generator 103 outputs the prediction signal with the smallest error among the prediction signals generated by the inter-screen prediction and the intra-screen prediction to the subtractor 105.

  When outputting the prediction signal generated by the inter-screen prediction, the prediction signal generator 103 outputs the determined motion information and information on the subregion division method to the entropy encoder 111. These pieces of information are encoded by the entropy encoder 111 and transmitted from the output terminal 112. Further, the prediction signal generator 103 also outputs information (reference index) indicating which reference image from which the prediction signal is acquired among the plurality of reference images to the entropy encoder 111.

  When outputting the prediction signal generated by the intra prediction, the prediction signal generator 103 outputs information on the extrapolation method to the entropy encoder 111. This information is encoded by the entropy encoder 111 and transmitted from the output terminal 112.

  The subtractor 105 is means for generating a residual signal by subtracting the prediction signal input from the prediction signal generator 103 from the signal of the target block input from the block divider 102. The subtractor 105 outputs the residual signal to the converter 106.

  The converter 106 is means for performing a discrete cosine transform on the residual signal input from the subtractor 105. The quantizer 107 is means for quantizing the transform coefficient obtained by the discrete cosine transform.

  The entropy encoder 111 is a unit that encodes the transform coefficient input from the quantizer 107 and outputs the encoded transform coefficient to the output terminal 112 together with information on the prediction method. The output terminal 112 is means for outputting such information to the outside.

  In order to perform intra prediction or inter prediction for the subsequent target block, the compressed signal of the target block is restored by inverse processing. For this purpose, an inverse quantizer 108, an inverse transformer 109, and an adder 110 are used.

  The inverse quantizer 108 is means for inversely quantizing the quantized transform coefficient. The inverse transformer 109 is means for restoring the residual signal by inverse discrete cosine transform. The adder 110 is means for adding the prediction signal input from the prediction signal generator 103 to the restored residual signal and reproducing the signal of the target block. The adder 110 stores the reproduced signal (reproduced image) in the frame memory 104 as a reference image.

  In the present embodiment, the converter 106 and the inverse converter 109 are used, but other conversion processes in place of these converters may be used. In some cases, the converter 106 and the inverse converter 109 may be omitted.

  The frame memory 104 is finite, and it is impossible to store all reproduced images as reference images. Therefore, the frame memory 104 stores only a reproduced image used for encoding and decoding of subsequent images. In the present embodiment, the frame memory 104 stores a maximum of 4 or 5 reproduced images as reference images.

  The buffer manager 114 is a means for controlling the frame memory 104. Specifically, the buffer manager 114 uses the POC (Picture Output Count) number and dependency information (D_ID (dependency ID) number) of each image received by the second input terminal 113 to refer to according to a predetermined rule. Manage the list of images.

  The POC number is information indicating the output order of images and is also referred to as output order information. The dependency information is information indicating how many images having the information are referred to from other images. In other words, the dependency degree information is information indicating the dependency degree of another image with respect to an image having the information. In this embodiment, a small D_ID number is assigned to an important reference image that is referenced more frequently from other images. That is, a reference image with a smaller D_ID number is referred to more from other images. The POC number and dependency information are set for each image. The D_ID number may be called a temporal ID, a priority ID, a prediction ID, a reference ID, a hierarchy ID, a level ID, a persistence ID, or a long-term ID in addition to the dependency ID.

  As described above, the size of the value of the D_ID number indicates how many images having the D_ID number are referred to by other images. When the value of the D_ID number is different between the two images, the image with the smaller D_ID number is required to reproduce the image with the larger D_ID number. However, the image with the larger D_ID number is not used for reproducing the image with the smaller D_ID number. When the two images have the same D_ID number, these two images can refer to each other.

  For example, it is assumed that there is a reference image Pa whose D_ID number value is Da and a reference image Pb whose D_ID number value is Db. In this case, if Da <Db, the reference image Pa is required for reproducing the reference image Pb, but the reference image Pb is not used for reproducing the reference image Pa. When Da = Db, the reference images Pa and Pb may refer to each other. If Da> Db, the reference image Pa is not used for reproducing the reference image Pb, but the reference image Pb is required for reproducing the reference image Pa.

  In this embodiment, four values of 0, 1, 2, and 3 are used as the value of the D_ID number. An image whose D_ID number is 3 is not used as a reference image. The D_ID number of each image is determined by the encoding apparatus 100. In this embodiment, a small value is set for the D_ID number of an image that is finely quantized. Alternatively, in temporal hierarchy (scalable) coding, 0 may be set for the D_ID number of the base layer image, and q may be set for the D_ID number of the q-th layer image.

  The POC number and D_ID number of each image are sent from the buffer manager 114 to the entropy encoder 111 together with an encoding type indicating an image encoding method (intra-frame prediction encoding or inter-frame prediction encoding). These three types of information are encoded by the entropy encoder 111 and sent out from the output terminal 112 together with the compressed image data.

  The buffer manager 114 manages a list of reference images used for encoding and decoding of the target image based on the POC number and D_ID number of each reference image stored in the frame memory 104. Therefore, it is not necessary to explicitly encode information about the reference image and transmit it to the decoding device. Details of the buffer manager 114 will be described later.

  Next, a video predictive decoding apparatus (hereinafter simply referred to as “decoding apparatus”) 200 according to the embodiment will be described with reference to FIG. FIG. 2 is a block diagram showing the decoding device 200. The decoding device 200 includes an input terminal (input means) 201, a data analyzer 202, an inverse quantizer 203, an inverse transformer 204, an adder 205, an output terminal 206, a frame memory (buffer) 207, a prediction signal generator 208, And a buffer manager (buffer management means) 209. The inverse quantizer 203, the inverse transformer 204, and the adder 205 correspond to decoding means. Note that a decoding unit other than these components may be used, or the inverse converter 204 may be omitted.

  The input terminal 201 is a means for acquiring data (compressed image data) that has been compressed (encoded) by the encoding device 100 and transmitted. The compressed image data includes a residual signal obtained by predicting and encoding a target block generated by dividing an image into a plurality of blocks, and information related to generation of a prediction signal. A POC number and dependency information (D_ID number) are added to each image. In the case of inter-screen prediction, the compressed image data includes information related to block division (block size) and motion information as information related to generation of a prediction signal. In the case of intra prediction, the compressed image data includes information relating to the extrapolation method as information relating to generation of the prediction signal.

  The data analyzer 202 includes a POC number and dependency information D_ID number of each image, a residual signal of a target block obtained by dividing the image, information related to generation of a prediction signal, a quantization parameter, Is a means for extracting from the compressed image data.

  The inverse quantizer 203 is means for inversely quantizing the residual signal of the target block based on the quantization parameter. The inverse transformer 204 is means for performing inverse discrete cosine transform on the inversely quantized residual signal.

  The prediction signal generator 208 is means for acquiring a reference image corresponding to information related to generation of a prediction signal from the frame memory 207 and generating a prediction signal from the reference image. The prediction signal generator 208 outputs the generated prediction signal to the adder 205. The adder 205 is means for reproducing the target block signal by adding the prediction signal to the residual signal input from the inverse transformer 204. The adder 205 outputs the target block signal to the output terminal 206 and stores it in the frame memory 207.

  The frame memory 207 is means for storing a reproduced image used for decoding and reproducing subsequent images as a reference image.

  The buffer manager 209 is a means for controlling the frame memory 207. Specifically, the buffer manager 209 uses the POC number and D_ID number of each image input from the data analyzer 202 to manage a list of reference images according to a predetermined rule. Details of the buffer manager 209 will be described later.

  Next, a method for controlling the frame memories 104 and 207 by the buffer manager 114 of the encoding device 100 and the buffer manager 209 of the decoding device 200 will be described with reference to FIGS. FIG. 3 is a flowchart showing an outline of the moving picture predictive encoding method and the moving picture predictive decoding method, and FIG. 4 is a flowchart showing details of management of the reference picture shown in FIG. FIG. 5 is a diagram illustrating a first example of a reference image management method in the embodiment, and FIG. 6 is a diagram illustrating a second example of the management method.

  The buffer manager manages the reproduced image as a reference image in the frame memory. First, the buffer manager initializes the frame memory to an “empty” state without a reference image in order to set the frame memory to an initial state before starting the encoding (or decoding) process.

  When encoding (or decoding) processing starts, management (update) of the reference image in the frame memory is performed (step 320), and encoding or decoding processing for the target image (k) is executed using the reference image. (Step 340). Since the reproduced image is used as a reference image for subsequent images, it is stored in the frame memory. This series of processes is repeated until the last image (see step 350). In FIG. 3, the process of step 320 is performed before step 340, but the process of step 320 may be performed after step 340.

  Details of reference image management will be described. Here, a target image to be processed (encoded or decoded) next is an image (k), and an image reproduced before this image (k) is a reproduced image (k−1). First, the buffer manager determines whether to use the reproduced image (k-1) as a reference image (step 420). In this embodiment, when the D_ID number of the reproduced image (k−1) is 3 (step S420; NO), the buffer manager does not use the reproduced image as a reference image.

If the D_ID number of the reproduced image (k-1) is other than 3 (step S420; YES), the buffer manager acquires information necessary for subsequent processing (step 430). Specifically, the buffer manager has already registered the POC number of the image (k) as POC _k , the maximum value of the frame memory (maximum value of the number of images that can be stored) as buffer_size, and has already been registered as a reference image in the frame memory. The number of images is substituted for N (step 430). Next, the buffer manager determines whether or not the number N of images is less than the maximum value of the frame memory (step 440). If N <buffer_size, that is, if there is still free space in the frame memory (step 440; YES), the buffer manager increments N (step 470) and uses the reproduced image (k-1) as a reference image. Registration in the frame memory (step 480). At this time, the buffer manager labels the reproduced image (k−1) as “used for reference”.

On the other hand, when the frame memory is full (step 440; NO), the buffer manager specifies a reference image to be deleted from the frame memory (step 450). First, the buffer manager specifies a reference image having a POC number smaller than POC _k from among reference images having the largest D_ID number among reference images registered in the frame memory. Subsequently, the buffer manager finally identifies the reference image having the smallest POC number among the identified reference images as an “unused reference image”. If there is no reference image satisfying such a condition, the buffer manager specifies a reference image having a POC number smaller than POC _k from among the reference images having the second largest D_ID number. Of the reference images, the reference image having the smallest POC number is set as an unnecessary reference image. Until an unnecessary reference image is found, the buffer manager repeats the same process while decreasing the value of the D_ID number that is the basis of the process one by one. Subsequently, the buffer manager deletes the unnecessary reference image from the frame memory (step 460), and then registers the reproduced image (k-1) as a reference image in the frame memory (step 480).

  A specific example of such reference image management will be described with reference to FIGS. The example of FIG. 5 is processing when there is no rearrangement of the target image. Reference numeral 501 indicates the order (or time) for reproducing (decoding) the target image. Squares (505, 520, 540, 560, 580, 581,...) In the area 502 indicate target images to be reproduced. The rectangles (500, 510, 530, 550, 570, 590,...) In the area 503 indicate the state in the frame memory (that is, the reference image list) before and after the target image reproduction process. In this example, it is assumed that the frame memory can store a maximum of four images. As shown in the legend in FIG. 5, the numerical value written in the center of the square indicating the image is the POC number, and the superscript numerical value is the D_ID number. In the present embodiment, a value (m + 1) is assigned to the D_ID number of an image that is encoded and decoded with reference to an image with D_ID number = m. However, even if the value of the D_ID number is determined by other rules. Good.

  When the target image 505 with POC number = 0 is reproduced, the frame memory is empty. That is, nothing is registered in the reference image list. When the target image 505 is reproduced, the reproduced image is stored in the frame memory as a reference image, and the POC number and D_ID number of the reproduced image are registered in the frame memory (state 510). Thereafter, the target images 520, 540, and 560 are reproduced and stored in the frame memory (states 530, 550, and 570). When the target image 560 is reproduced and stored in the frame memory, the frame memory becomes full (state 570).

Since the frame memory is full when the image 580 is reproduced, it is necessary to delete the reference image in the frame memory and secure an empty area in the frame memory. In this embodiment, in step 450 in FIG. 4, the reference image having the largest D_ID number in the frame memory and having a POC number smaller than the POC number (= POC _k ) of the target image to be reproduced next. Of these, the reference image with the smallest POC number is identified as an unnecessary reference image. When the image 580 is reproduced, the reference image list 570 having the largest D_ID number “2” is the reference image 571, 572, and the POC number of these two images is the target image 581 to be reproduced next. Is smaller than the POC number “5”. Therefore, the reference image 571 having the smallest POC number (POC number = 1) among these two images is specified as an unnecessary reference image. Therefore, after the reference image 571 is deleted from the frame memory, the target image 580 is reproduced and stored in the frame memory as a reference image (state 590).

Thereafter, similar processing is executed, and the state in the frame memory changes as indicated by reference numerals 591, 592, 593, 594,. In this way, by deleting the reference image located farthest from the target image among the reference images (reference images having a large D_ID number) that are not often referred to due to poor quality, the quality is good and close to the target image. The reproduced image can be stored in the frame memory as a reference image. As a result, predictive coding can be performed efficiently. In the example of FIG. 5, since the images are processed in the output order (display order), the POC number (= POC _k ) of the target image to be reproduced next is always larger than the POC number of any reference image in the frame memory. Therefore, the condition that “the POC number of the reference image is smaller than POC _k ” in step 450 of FIG. 4 may be omitted.

  The example of FIG. 6 is processing when the target images are rearranged. Reference numeral 621 indicates the order (or time) of decoding the target image. Squares in the region 622 (601, 602, 604, 606, 608, 620, 640, 660, 661, 670, 680,...) Indicate target images to be reproduced. A rectangle (600, 610, 612, 614, 616, 618, 630, 650, 651, 690, 695,...) In the area 623 indicates the state of the reference image (reference image) in the frame memory before and after the decoding process of the target image. List). In the example of FIG. 6, it is assumed that the frame memory can store a maximum of five images. The way of expressing the POC number and D_ID number of each image is the same as in FIG.

  In the example of FIG. 6, the decoding order of the target images is different from the output order (display order). For example, after the target image 601 with POC number = 0 is reproduced, the target image 602 with POC number = 8 is reproduced, and then the target image 604 with POC number = 4 is reproduced. Since “bidirectional prediction” can be performed by encoding and decoding images in an order different from the output order (display order) in this way, a moving image can be encoded with high compression efficiency.

  The process starts from a state where the frame memory is empty (state 600). First, target images 601, 602, 604, 606, and 608 are sequentially reproduced and stored in the frame memory. When the target image 608 is stored in the frame memory, the frame memory becomes full (state 618).

  Subsequently, four target images 620 are sequentially reproduced using these reference images. Since the target image 620 is a “non-reference image” (an image that is not referred to when decoding other images), the reproduced target image 620 is sent from the output terminal without being stored in the frame memory. Accordingly, the state 630 of the frame memory remains unchanged from the state 618.

Next, an image 640 with POC number = 16 is reproduced. Since the image 640 can be a reference image, it is necessary to delete the reference image in the frame memory and secure an empty area in the frame memory. In this embodiment, in step 450 in FIG. 4, among the reference images having the largest D_ID number in the frame memory and having a POC number smaller than the POC number (POC _k ) of the target image to be reproduced next. , The reference image with the smallest POC number is identified as an unnecessary reference image. At the time of reproducing the image 640, the reference image list 630 having the largest D_ID number “2” is the reference image 631, 632, and the POC number of these two images is the target image 660 to be reproduced next. Is smaller than the POC number “12”. Therefore, the reference image 631 having the smallest POC number (POC number = 2) among these two images is specified as an unnecessary reference image. As a result, the reference image 631 is deleted from the frame memory, and the target image 640 is reproduced and stored in the frame memory as a reference image (state 650).

  Next, a case where the target image 670 with POC number = 14 is reproduced will be described. Since the target image 670 can be a reference image, it is necessary to delete the reference image in the frame memory and secure an empty area in the frame memory.

  In the process of step 450 at this time, first, a reference image having the largest D_ID number “2” and a POC number smaller than the POC number “9” of the target image 680 to be processed next is searched. No reference image satisfying this condition exists in the list 690. Therefore, a reference image having the second largest D_ID number “1” and a POC number smaller than the POC number “9” of the target image 680 to be processed next is searched. The reference image does not exist in the list 690.

  Therefore, next, a reference image having a D_ID number “0” and a POC number smaller than the POC number “9” of the target image 680 to be processed next is searched, and the reference images 691 and 692 are specified. Therefore, the reference image 691 having the smallest POC number (POC number = 0) among these two images is specified as an unnecessary reference image. Thereby, after the reference image 691 is deleted from the frame memory, the target image 670 is reproduced and stored in the frame memory as a reference image (state 695).

  Thus, since there is a condition that “the POC number is smaller than the POC number of the target image 680 to be processed next”, the reference image 696 with the POC number = 10 is left in the reference image list 695. Since the target image with the POC number = 9 is closer to the reference image with the POC number = 10 than the reference image with the POC number = 0, the reference image is managed as described above, so It is possible to efficiently predict and encode a target image having a high correlation on the time axis.

  As shown in FIGS. 5 and 6, the condition in step 450 of FIG. 4 can be any one of the following a to c according to the structure of predictive coding.

(A) A reference image having the smallest POC number among reference images having a predetermined D_ID number (for example, the largest D_ID number) is set as an unnecessary reference image. This method is applicable when there is no rearrangement of the target images as shown in FIG.
(B) Among the reference images having a predetermined D_ID number (for example, the maximum D_ID number) and having a POC number smaller than that of the target image to be processed next, the reference image having the smallest POC number is unnecessary reference An image. This technique is applicable when there is a rearrangement of images.
(C) Of the reference images having a POC number smaller than that of the target image to be processed next, the reference image having the smallest POC number is set as an unnecessary reference image. This method is applicable when the D_ID numbers of all images are the same.

  The description of FIGS. 4 to 6 is the same for the decoding and reproduction processing in the encoding processing.

  The example of FIG. 6 described above corresponds to the “bidirectional prediction” coding structure. In order to perform bi-directional prediction, two prediction signals are acquired, and these signals are averaged (or weighted average) to generate a final prediction signal. In order to obtain two prediction signals, the reference image stored in the frame memory is logically divided into two reference image lists, that is, List0 and List1, and the first prediction signal is included in List0. From the plurality of reference images, the second prediction signal is obtained from each of the plurality of reference images included in List1. For each target block, an instruction as to which list the prediction signal is to be acquired from is determined by the encoding device based on the minimum error method and then transmitted to the decoding device.

  In the present embodiment, logical reference image lists List0 and List1 are prepared by the following procedure using the POC number and D_ID number of the reference image in the frame memory.

(1) A plurality of reference images in the frame memory are divided into a set A (Set A) and a set B (Set B). Set A includes a reference image having a POC number smaller than the POC number of the target image, and set B includes a reference image having a POC number larger than the POC number of the target image.
(2) The order of each reference image in set A and set B is sorted. Specifically, the sorting is performed so that reference images having a small D_ID number (that is, important reference images) are arranged at the top of the list. For reference images with the same D_ID number, sorting is performed so that images close to the POC number of the target image are arranged at the end of the list.
(3) Set A sorted in (2) is List0. If the size of set A (the number of reference images registered in set A) is smaller than the size of List0 (the maximum number of reference images that can be registered in List0), the sorted set B is added to List0. However, the reference image overlapping with the set A is removed. In the present embodiment, the size of List0 is the same as the size of the frame memory.
(4) Set B sorted in (2) is List1. If the size of set B (the number of reference images registered in set B) is smaller than the size of List1 (the maximum number of reference images that can be registered in List1), the sorted set A is added to List1. However, the reference image overlapping with the set B is removed. In the present embodiment, the size of List1 is the same as the size of the frame memory.

  In this way, based on the POC number and the D_ID number, a reproduction image having a high quality and closer to the target image is arranged at the end of the reference image list so that the reproduction image can be referred to more easily. It is possible to generate a prediction signal based on a high-quality reference image having a high correlation. As a result, the effect of improving the coding efficiency can be obtained.

The pseudo code for the above process can be written as:
cIdx = 0
offsetB = min (size (List1), size (Set B));
for (i = 0; i <size (Set A) && cIdx <size (List0); cIdx ++, i ++)
List0 [cIdx] = Set A [i]
for (i = 0; i <size (Set B) && cIdx <size (List0); cIdx ++, i ++)
List0 [cIdx] = Set B [(i + offsetB)% size (Set B)]

cIdx = 0
offsetA = min (size (List0), size (Set A));
for (i = 0; i <size (Set B) && cIdx <size (List1); cIdx ++, i ++)
List1 [cIdx] = Set B [i]
for (i = 0; i <size (Set A) && cIdx <size (List1); cIdx ++, i ++)
List1 [cIdx] = Set A [(i + offsetA)% size (Set A)]

In the present embodiment, a small D_ID number is assigned to an important reference image that is referred to more frequently, but a large D_ID number may be assigned to such an important reference image. In this case, in step 450 of FIG. 4, the presence or absence of unnecessary reference images may be determined in order from the reference image having the smallest D_ID number (if necessary, D_ID numbers are set one by one until an unnecessary reference image is found. Repeat search process while increasing).

In the present embodiment, POC _k indicates “the POC number of the next image to be processed”, but instead, a condition using the POC number of the reference image already in the buffer may be set. That is, POC _k may be “the second largest POC number among the POC numbers of the reference images having the smallest D_ID number”.

  In this embodiment, the POC number is used, but display time, decoding order or decoding time, frame number, and the like may be used as information indicating the order of images.

  Next, the program according to the present embodiment will be described with reference to FIGS. FIG. 7 is a block diagram showing a video predictive encoding program according to the embodiment. FIG. 8 is a block diagram showing a moving picture predictive decoding program according to the embodiment. FIG. 9 is a diagram illustrating a hardware configuration of the computer 30, and FIG. 10 is a perspective view of the computer 30.

  The moving picture predictive coding program P100 includes a block division module P102, a prediction signal generation module P103, a storage module P104, a subtraction module P105, a transformation module P106, a quantization module P107, an inverse quantization module P108, an inverse transformation module P109, and an addition module. P110, an entropy encoding module P111, and a buffer management module P114. The function of each module is as follows: block divider 102, prediction signal generator 103, frame memory 104, subtractor 105, transformer 106, quantizer 107, inverse quantizer 108, inverse transformer 109, adder 110, entropy The functions of the encoder 111 and the buffer manager 114 are the same.

  The moving picture predictive decoding program P200 includes a data analysis module P202, an inverse quantization module P203, an inverse transform module P204, an addition module P205, a storage module P207, a prediction signal generation module P208, and a buffer management module P209. The functions of each module are the same as the functions of the data analyzer 202, inverse quantizer 203, inverse transformer 204, adder 205, frame memory 207, prediction signal generator 208, and buffer manager 209.

  The moving picture predictive encoding program P100 and the moving picture predictive decoding program P200 are provided by the recording medium 10 shown in FIGS. The computer 30 functions as the encoding device 100 when the moving image predictive encoding program P100 is read and executed, and functions as the decoding device 200 when the moving image predictive decoding program P200 is read and executed. The device that executes the programs P100 and P200 is not limited to the computer 30, but may be a DVD player, a set top box, a mobile phone, or the like that includes a CPU and performs processing and control by software.

  As shown in FIG. 9, the computer 30 includes a reading device 12 such as a flexible disk drive device, a CD-ROM drive device, and a DVD drive device, a working memory (RAM) 14 in which an operating system is resident, and a recording medium 10. A memory 16 for storing programs stored therein, a display (display device) 18, a mouse 20 and a keyboard 22 as input devices, a communication device 24 for transmitting and receiving data, and a CPU 26 for controlling execution of the programs. And. When the recording medium 10 is inserted into the reading device 12, the computer 30 can access the moving picture predictive encoding program or the moving picture predictive decoding program stored in the recording medium 10, and by reading and executing this program. It operates as the encoding device 100 or the decoding device 200.

  As shown in FIG. 10, the moving picture predictive encoding program and the moving picture decoding program may be provided as a data signal 40 superimposed on a carrier wave via a network. In this case, the computer 30 stores the moving image predictive encoding program or moving image decoding program received by the communication device 24 in the memory 16, and operates as the encoding device 100 or the decoding device 200 by executing the program. .

  As described above, according to the present embodiment, based on at least one of the output order information (POC number) and dependency information (D_ID number) of the reference image stored in the buffer (frame memories 104 and 207). Thus, the reference image to be deleted is determined. Therefore, it is possible to store a reference image with excellent quality in a buffer for a longer period without explicitly sending management information, and use the reference image for encoding or decoding more target images. As a result, the image compression rate can be increased. Specifically, a high D_ID number is assigned to a reference image with poor quality, and a reference image that is farthest from the target image is sequentially deleted from these reference images, thereby improving the quality and the target image. A reproduced image close to can be stored in the buffer as a reference image. As a result, it is possible to secure a reference image having a high correlation with the target image and perform predictive coding efficiently.

  The present invention has been described in detail based on the embodiments. However, the present invention is not limited to the above embodiment. The present invention can be variously modified without departing from the gist thereof.

  In the above embodiment, the buffer manager directly controls the reference image in the frame memory. However, the present invention can be applied to the case where the image in the frame memory is mapped to the reference image via a logic means called “reference image list”. Can be applied.

  DESCRIPTION OF SYMBOLS 100 ... Moving image predictive coding apparatus 101 ... 1st input terminal, 102 ... Block divider, 103 ... Prediction signal generator, 104 ... Frame memory, 105 ... Subtractor, 106 ... Converter, 107 ... Quantizer, DESCRIPTION OF SYMBOLS 108 ... Inverse quantizer, 109 ... Inverse transformer, 110 ... Adder, 111 ... Entropy encoder, 112 ... Output terminal, 113 ... Second input terminal, 114 ... Buffer manager, 200 ... Moving picture predictive decoding apparatus , 201 ... input terminal, 202 ... data analyzer, 203 ... inverse quantizer, 204 ... inverse transformer, 205 ... adder, 206 ... output terminal, 207 ... frame memory, 208 ... prediction signal generator, 209 ... buffer Manager, P100 ... moving picture predictive encoding program, P102 ... block division module, P103 ... prediction signal generation module, P104 ... storage module, 105 ... Subtraction module, P106 ... Conversion module, P107 ... Quantization module, P108 ... Inverse quantization module, P109 ... Inverse transformation module, P110 ... Addition module, P111 ... Entropy encoding module, P114 ... Buffer management module, P200 ... Movie Image prediction decoding program, P202 ... data analysis module, P203 ... inverse quantization module, P204 ... inverse transform module, P205 ... addition module, P207 ... storage module, P208 ... prediction signal generation module, P209 ... buffer management module.

Claims (6)

Input means for receiving a plurality of target images constituting a moving image;
Output order information indicating the output order of the target images that have been predictively encoded using at least one image that has been previously encoded and decoded and reproduced as a reference image, and other images Encoding means for generating compressed image data by adding, to the target image, dependency information indicating the dependency of the other image with respect to the target image at the time of predictive encoding,
Decoding means for decoding the compressed image data to generate a reproduced image;
A buffer for storing the reproduced image as a reference image used for encoding a subsequent image;
Buffer management means for controlling the buffer,
The buffer management means, among the plurality of reference images stored in the buffer, out of the reference images having the lowest dependency according to the dependency information, the output order based on the output order information is a prediction code A reference image that is smaller than the output order of the target image that is an object to be converted is identified, and among the identified reference images, the reference image that has the smallest output order based on the output order information is deleted from the plurality of reference images determining as a reference image, motion image predictive coding apparatus. A video predictive encoding method executed by a video predictive encoding device,
An input step for receiving a plurality of target images constituting the moving image;
Output order information indicating the output order of the target images that have been predictively encoded using at least one image that has been previously encoded and decoded and reproduced as a reference image, and other images Encoding step of generating compressed image data by adding dependency information indicating the dependency of the other image to the target image when predictively encoding the image to the target image;
A decoding step of decoding the compressed image data to generate a reproduced image;
Storing the reproduced image in a buffer as a reference image used to encode a subsequent image;
A buffer management step for controlling the buffer,
In the buffer management step, out of a plurality of reference images stored in the buffer, an output order based on the output order information is a prediction code based on the reference image having the lowest dependency according to the dependency information. A reference image that is smaller than the output order of the target image that is an object to be converted is identified, and among the identified reference images, the reference image that has the smallest output order based on the output order information is deleted from the plurality of reference images A moving picture predictive encoding method that is determined as a reference picture. Input means for receiving compressed image data for each of a plurality of target images constituting a moving image, wherein the compressed image data is predictively encoded using at least one image decoded and reproduced in the past as a reference image The output data, output order information indicating the output order of the target image, and dependency information indicating the dependency of the other image with respect to the target image when predictively encoding the other target image, Input means;
Decoding means for decoding the compressed image data to generate a reproduced image;
A buffer for storing the reproduced image as a reference image used for decoding subsequent images;
Buffer management means for controlling the buffer,
The buffer management means outputs the decoding order based on the output order information from among the reference pictures having the lowest dependence according to the dependence information among the plurality of reference pictures stored in the buffer . A reference image that identifies a reference image that is smaller than the output order of the target image that is the target , and that deletes the reference image that has the minimum output order based on the output order information from the plurality of reference images. A video predictive decoding apparatus that determines as follows. A video predictive decoding method executed by a video predictive decoding device,
An input step for receiving compressed image data for each of a plurality of target images constituting a moving image, wherein the compressed image data is predictively encoded using at least one image decoded and reproduced in the past as a reference image The output data, output order information indicating the output order of the target image, and dependency information indicating the dependency of the other image with respect to the target image when predictively encoding the other target image, An input step;
A decoding step of decoding the compressed image data to generate a reproduced image;
Storing the playback image in a buffer as a reference image used to decode subsequent images;
A buffer management step for controlling the buffer,
In the buffer management step, out of a plurality of reference images stored in the buffer, an output order based on the output order information is selected from among reference images having the lowest dependency according to the dependency information. A reference image that identifies a reference image that is smaller than the output order of the target image that is the target , and that deletes the reference image that has the minimum output order based on the output order information from the plurality of reference images. A video predictive decoding method that is determined as : Input means for receiving a plurality of target images constituting a moving image;
Output order information indicating the output order of the target images that have been predictively encoded using at least one image that has been previously encoded and decoded and reproduced as a reference image, and other images Encoding means for generating compressed image data by adding, to the target image, dependency information indicating the dependency of the other image with respect to the target image at the time of predictive encoding,
Decoding means for decoding the compressed image data to generate a reproduced image;
A buffer for storing the reproduced image as a reference image used for encoding a subsequent image;
Causing a computer to execute buffer management means for controlling the buffer;
The buffer management means, among the plurality of reference images stored in the buffer, out of the reference images having the lowest dependency according to the dependency information, the output order based on the output order information is a prediction code A reference image that is smaller than the output order of the target image that is an object to be converted is identified, and among the identified reference images, the reference image that has the smallest output order based on the output order information is deleted from the plurality of reference images A moving picture predictive coding program that is determined as a reference picture. Input means for receiving compressed image data for each of a plurality of target images constituting a moving image, wherein the compressed image data is predictively encoded using at least one image decoded and reproduced in the past as a reference image The output data, output order information indicating the output order of the target image, and dependency information indicating the dependency of the other image with respect to the target image when predictively encoding the other target image, Input means;
Decoding means for decoding the compressed image data to generate a reproduced image;
A buffer for storing the reproduced image as a reference image used for decoding subsequent images;
Causing a computer to execute buffer management means for controlling the buffer;
The buffer management means outputs the decoding order based on the output order information from among the reference pictures having the lowest dependence according to the dependence information among the plurality of reference pictures stored in the buffer . A reference image that identifies a reference image that is smaller than the output order of the target image that is the target , and that deletes the reference image that has the minimum output order based on the output order information from the plurality of reference images. A moving picture predictive decoding program which is determined as

Images