JP2008294978A - Image decoding apparatus, image decoding method and image decoding program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To shorten an access time when reading a reference image. <P>SOLUTION: In a system of the present invention, an image encoding apparatus which transmits a stream, and an image decoding apparatus which decodes the stream to produce decoded data, are communicably connected. The image decoding apparatus then holds a frame structure memory for storing a reference image in a frame structure corresponding to frame prediction, and a field structure memory for storing a reference image in a field structure corresponding to field prediction. In such a configuration, in the case where a stream is accepted, the image decoding apparatus analyzes the relevant stream to discriminate a prediction form used for encoding, reads a reference image being stored in the discriminated prediction form from the memory, produces new decoded data from the stream and the reference image, produces reference images corresponding to prediction forms in the plurality of memories from the produced new decoded data and stores the reference images in the plurality of memories, respectively. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention accepts encoded data from an image encoding device that generates encoded data that is encoded data, and uses a reference image that is already decoded data and the encoded data to generate new data. The present invention relates to an image decoding apparatus, an image decoding method, and an image decoding program that generate decoded data.

  Conventionally, moving picture compression techniques such as MPEG-2 (Moving Picture Experts Group phase 2) and H.264 / AVC have used inter-frame prediction in order to obtain a high compression rate in compression encoding of moving images. This frame prediction uses the property of a moving image that the input frame and the frame that is temporally similar to each other are used to generate a prediction frame that predicts the current frame from the previous and subsequent frames, and This is a technique for encoding a difference from a predicted frame. Further, as described above, predicting a predicted frame in units of fields instead of in units of frames is called field prediction.

  In such frame prediction and field prediction, a technique called motion compensation is used to further increase the compression efficiency. As an example of frame prediction, in frame prediction, the current frame is predicted from the previous and next frames to create a predicted frame, but the previous and next frames are not exactly the same as the current frame. The object is moving. That is, there is continuity between frames. A technique for compensating for the continuity between frames is motion compensation.

  More specifically, an image encoding device (encoder) that encodes and transmits data calculates a difference for each prediction unit constituting a frame to calculate a motion vector. Then, the image encoding device generates a stream (data) by encoding the motion vector data calculated in this way and difference image data that is a difference between the input frame and the prediction frame, and the generated stream It transmits to an image decoding apparatus (decoder).

  Then, the image decoding apparatus analyzes the received stream, acquires an image corresponding to motion vector data (referred to as a reference image) from the memory for each prediction unit, and adds the difference image data to the reference image. Decrypt the data. In addition, the image decoding apparatus stores the decoded data thus decoded in the memory as a reference image necessary for newly generating decoded data from a stream received later.

  As described above, in order for the image decoding apparatus to generate decoded data, the reference image stored in the memory is important, and for generating decoded data, the access time for reading the reference image from the memory is important. Is related to. That is, if the access time is high, the data can be decoded at high speed, and if the access time is low, it takes time to decode the data.

  In order to explain this relationship with a specific example, first, a memory for storing a reference image will be described. In general, the reference image stored in the memory has a frame structure in which the first field and the second field are alternately stored so as to correspond to sequential scanning as shown in FIG. 12, and as shown in FIG. In order to correspond to the interlaced scanning, the first field and the second field are stored in one of the field structures for storing them in different areas. As a memory for storing such a reference image, a memory capable of high-speed data transfer such as DDR2-SDRAM (Double Data Rate 2-Synchronous Dynamic Random Access Memory) that can easily secure an access band is usually used. .

  This DDR2-SDRAM is accessed in units of 16 bytes when the minimum access is 2 clocks (4 bursts) and the data bus is 32 bits. One pixel of the stored moving image is composed of a luminance component (Y) and a color difference component (C). In other words, if the luminance component and the color difference component are each 8 bits and calculation is made to store the luminance component, the DDR2-SDRAM accumulates and stores data of 16 pixels (16 bytes) as a unit. In this case, the relationship between memory access units and pixels is, for example, 16 pixels of 4 pixels × 4 lines as shown in FIG. The access unit illustrated here is merely an example, and does not necessarily have to be 4 pixels × 4 lines, and may be 8 pixels × 2 lines.

  Next, focusing on only the above-described luminance component (one motion vector data), an access time for the reference image to be read by the image decoding apparatus including such a DDR2-SDRAM will be described. In the following, a case where an image of 9 pixels × 9 lines including “◆” and “●” is required as a reference image will be described as an example.

  Thus, when the image decoding apparatus receives a stream in which the motion compensation prediction format is frame prediction when the reference image is stored in a memory such as DDR2-SDRAM in a frame structure, for example, As shown in FIG. 15, it is necessary to read (read) nine 4 × 4 access units (total 144 pixels) including image information “♦” and “●” that are actually necessary in order to decode data from the stream. There is.

  On the other hand, when a reference image is stored in such a memory such as DDR2-SDRAM in a field structure, the image decoding apparatus receives a stream in which the motion compensation prediction format is frame prediction. As shown in FIG. 16, the image decoding apparatus reads six pieces of 4 × 4 access units (96 pixels in total) from the first field in order to read out actually required image information “◆” and “●”. It is necessary to read six 4 × 4 access units (96 pixels in total) from the second field.

  Further, when a reference image is stored in a frame structure and a stream whose motion compensation prediction format is field prediction is received, for example, as illustrated in FIG. 17, the image decoding apparatus is actually required. It is necessary to read 15 4 × 4 access units including the image information “♦” (240 pixels in total).

  On the other hand, if a reference image is stored in a field structure and a stream whose motion compensation prediction format is field prediction is received, for example, as shown in FIG. In order to read out the image information “♦”, 0 4 × 4 access units from the first field (total 0 pixels) and 9 4 × 4 access units from the second field (total 144 pixels) Need to read.

  Based on the result of the 1-motion vector data described above with respect to the luminance component, FIG. 19 shows the number of pixels (number of read-out pixels) necessary for reference in the 2-motion vector data (when there are two reference images). As shown in FIG. 19, when the data structure stored in the memory is different from the prediction format used for the received stream, the number of pixels to be read increases and the access time is delayed. On the other hand, when the data structure stored in the memory and the prediction format used for the received stream are the same, the number of pixels to be read is reduced and the access time is shortened.

  As described above, the access time for reading the reference image from the memory is important for data decoding. Therefore, various techniques for shortening access time by efficiently reading a reference image from a memory in an image decoding apparatus are disclosed.

  For example, in Patent Document 1 (Japanese Patent Laid-Open No. 8-275181), even when a reference image is divided and stored in a plurality of memories, it is possible to prevent an increase in time for accessing the memories, regardless of the reading position. A technique capable of referring to a reference image with a certain access time is disclosed. Specifically, when the reference image is divided and stored in a plurality of memories, the image data in the vicinity of the division point is written in two areas in which image data before and after the division point of the memory is stored. In this way, when reading the divided reference image, the reference image can be read from one memory without changing the read address.

JP-A-8-275181

  However, the above-described conventional technique (Patent Document 1) is a technique for preventing deterioration of reading efficiency when the reference image is divided and stored, and the reading efficiency cannot be improved, and the access time There was a problem that it was not possible to shorten. In the above description, the case where field prediction and frame prediction are separately used as image compression techniques has been described. However, image compression techniques in which field prediction and frame prediction are mixed have the same problem.

  Therefore, the present invention has been made to solve the above-described problems of the prior art, and an image decoding apparatus, an image decoding method, and an image decoding apparatus capable of reducing access time when reading a reference image An object is to provide an image decoding program.

  In order to solve the above-described problems and achieve the object, the invention according to claim 1 is configured to receive encoded data from an image encoding device that generates encoded data that is encoded image data. An image decoding device that generates new decoded data using a reference image that is already decoded data and the encoded data, and that matches each of the prediction formats that can be used for the encoded data Reference image storage means for storing the reference image in a structure, and when the encoded data is received, the encoded data is analyzed to determine a prediction format used for encoding, and stored in the determined prediction format A decoded data generating unit that reads a reference image to be read from the reference image storage unit and generates new decoded data from the encoded data and the reference image; and the decoded data A reference image storage unit that generates a reference image corresponding to each of the prediction formats from the new decoded data generated by the generation unit, and stores the generated reference image in the reference image storage unit, respectively. It is characterized by that.

  Further, in the invention according to claim 2, in the above invention, when the decoded data generation means receives the encoded data, the prediction format used for encoding by analyzing the encoded data is: Whether it is frame prediction or field prediction, and if it is frame prediction, a reference image having a frame structure is read from the reference image storage means, and new decoded data is obtained from the encoded data and the reference image. When the field prediction is performed, a reference image having a field structure is read from the reference image storage unit, and new decoded data is generated from the encoded data and the reference image. The reference image storage unit And generating a frame-structured reference image and a field-structured reference image from the new decoded data generated by the decoded data generating means, A reference image of the reference image and the field structure of the frame structure, characterized in that respectively stored in the reference image storage means.

  According to a third aspect of the present invention, in the above invention, the reference image storage unit includes a single storage unit, and the reference image storage unit is a reference image corresponding to the prediction format. Are generated, and the generated reference image is stored in the reference image storage means in divided regions.

  According to a fourth aspect of the present invention, in the above invention, the reference image storage means includes a plurality of storage units each storing a reference image for each prediction format, and stores the reference image. The means generates a reference image for each of the prediction formats, and stores the generated reference images in reference image storage means for storing the reference images in the respective prediction formats.

  According to a fifth aspect of the present invention, in the above invention, the reference image storage unit is configured to generate, for each prediction format of the plurality of reference image storage units, from the new decoded data generated by the decoded data generation unit. When the corresponding reference image is generated and stored, the reference image is stored by dividing the region into a luminance component and a color difference component.

  According to a sixth aspect of the present invention, in the above invention, when the decoded data generating means receives the encoded data, the decoded data is analyzed to determine a prediction format used for encoding. Then, when reading the reference image corresponding to the determined prediction format from the plurality of reference image storage means, two or more lines are used as access units in the vertical direction with respect to the storage format of the plurality of reference image storage means. It is characterized by.

  According to a seventh aspect of the present invention, a reference image that is decoded data that has already been decoded into image data by receiving the encoded data from an image encoding device that generates encoded data that is encoded image data. And a reference image storage unit that stores the reference image for each prediction format used for the encoded data, and a decoding method suitable for generating new decoded data using the encoded data, When the encoded data is received, the encoded data is analyzed to determine the prediction format used for encoding, the reference image stored in the determined prediction format is read from the reference image storage means, From the decoded data generation step of generating new decoded data from the encoded data and the reference image, and from the new decoded data generated by the decoded data generation step, Generates a reference image corresponding to each measurement type, characterized in that the reference image thus generated containing a reference image storing step of storing each of said reference image storage means.

  According to an eighth aspect of the present invention, there is provided a reference image that is decoded data that has been decoded into image data by receiving the encoded data from an image encoding device that generates encoded data that is encoded image data. Image storage means for storing the reference image for each prediction format used for the encoded data, the decoding program causing a computer to generate new decoded data using the encoded data and the encoded data When the encoded data is received, the encoded data is analyzed to determine a prediction format used for encoding, and a reference image stored in the determined prediction format is read from the reference image storage unit A decoded data generation procedure for generating new decoded data from the encoded data and the reference image, and a new data generated by the decoded data generation procedure. Generating a reference image corresponding to each of the prediction formats from the decoded data, and causing a computer to execute a reference image storage procedure for storing the generated reference image in the reference image storage unit, respectively. .

  According to the first, seventh, and eighth aspects of the invention, the reference image is stored in a structure that matches each of the prediction formats that can be used for the encoded data, and when the encoded data is received, the encoded data is analyzed. Then, the prediction format used for the encoding is determined, the reference image stored in the determined prediction format is read, new decoded data is generated from the encoded data and the reference image, and the generated new decoding Since the reference image corresponding to each prediction format is generated from the data and the generated reference image is stored, the access time can be shortened when reading the reference image.

  For example, according to various prediction formats used for frame prediction, field prediction, etc., by storing a reference image in a memory or the like in advance, encoded data encoded using any prediction format is accepted. Even in such a case, the reference image can be read out with the shortest access time regardless of the state of the reference image to be read out.

  In addition, as a result of writing the reference image into the memory in a plurality of formats, there is a concern that the access time is not shortened in total when the write access time to the memory and the read access time to read the reference image from the memory are taken into consideration. . However, in general, the amount of reading (number of readings) is overwhelmingly larger than the amount of writing to the memory (number of writings), so that the access time can be shortened considering both writing and reading. Is possible.

  To give a specific example, when MPEG-2 is used as an encoding technique (compression technique), the number of pixels to be written in a memory is 16 × 16 256 pixels when calculated per macroblock. Thus, the number of pixels read from the memory is 17 × 17 × 2 578 pixels. In addition, when H.264 / AVC is used as the encoding technique (compression technique), when the prediction unit is calculated as 8 × 8, the number of pixels to be written in the memory is 64 × 8 × 8, The number of pixels read from the memory is 338 pixels of 13 × 13 × 2. As described above, even if the number of times of writing to the memory is doubled, the number of pixels from which data is read out from the memory is larger, so that the access time can be shortened considering both writing and reading.

  The calculation of the number of pixels to be read will be described. In the case of MPEG-2, the motion vector is 0.5 unit. In this case, the interpolation calculation of the value after the decimal point requires a value for each of the upper, lower, left and right pixels, so the number of pixels to be decoded and written to the memory Is 16 × 16, the maximum number of pixels to be read out is 17 × 17. Further, since the number of motion vectors is two at the maximum, the number of pixels that need to be read as a whole is 17 × 17 × 2 = 578 pixels at maximum. Similarly, in the case of H.264 / AVC, the motion vector is 0.25 units, and in this case, interpolation calculation of values below the decimal point requires values of three pixels at the top, bottom, left, and right. Assuming that the number of pixels written to the memory is 8 × 8, the maximum number of pixels to be read out is 13 × 13. Further, since the number of motion vectors is two at the maximum, the total number of pixels that need to be read is 13 × 13 × 2 = 338 pixels.

  According to the invention of claim 2, when encoded data is received, the encoded data is analyzed to determine whether the prediction format used for encoding is frame prediction or field prediction. In case of frame prediction, the frame structure reference image is read out, and new decoded data is generated from the encoded data and the reference image. In the case of field prediction, the field structure reference image is read out. Then, new decoded data is generated from the encoded data and the reference image, a frame-structured reference image and a field-structured reference image are generated from the generated new decoded data, and the generated frame-structured reference image And the field structure reference image, respectively, the access time can be further reduced when the reference image is read out.

  For example, frame prediction and field prediction, which are generally used as encoding technologies (compression technologies), are stored in memory in advance, so that compression such as MPEG-2 and H.264 / AVC is often used. When encoded data using a technique is received, the access time can be further shortened when the reference image is read.

  According to the invention of claim 3, each of the reference images corresponding to the prediction format is generated, and the generated reference images are stored separately in areas. It is possible to prevent the storage area from being wasted.

  For example, if the reference image is generated for each prediction format but the capacity is small, storing in a plurality of memories increases the free capacity and wastes the capacity. In such a case, it is possible to prevent the area of the storage unit (memory) from being wasted by dividing the area into one memory and storing the reference image.

  According to the fourth aspect of the present invention, reference images are generated for each prediction format, and the generated reference images are stored in the respective prediction formats, so that the reference image writing time is shortened. It is possible.

  For example, when there are two types of prediction formats, frame prediction and field prediction, if two memories for storing reference images of the respective prediction formats are prepared, a reference image for each generated prediction format is obtained. As a result of being able to start writing to two memories at the same time, it is possible to further shorten the reference image writing time compared to writing two types of reference images to one memory.

  According to the invention of claim 5, when generating and storing a reference image corresponding to each prediction format from the generated new decoded data, the reference image is divided into a luminance component and a color difference component. Therefore, the access time can be further shortened when only necessary components are referred to.

  For example, in addition to analyzing data such as analyzing the decoding status, when it is necessary to refer to the luminance component and chrominance component, the access time for referencing only the luminance component or referencing only the chrominance component is shortened. It is possible. Further, it is possible to cope with the case where the prediction format is different between the luminance component and the color difference component.

  According to the sixth aspect of the present invention, when encoded data is received, the encoded data is analyzed to determine a prediction format used for encoding, and a reference image corresponding to the determined prediction format is determined. At the time of reading, since two or more lines are used as access units in the vertical direction for a plurality of storage formats, it is possible to reduce the access amount when reading the reference image.

  Specifically, as a general example of reading a reference image, an access unit is 16 pixels and a size that needs to be read is 9 × 9 pixels with reference to FIGS. 9 to 11. FIG. 9 is a diagram illustrating an example of the number of pixels necessary for reading when four lines are used as access units in the vertical direction, and FIG. 10 is a diagram necessary for reading when two lines are used as access units in the vertical direction. FIG. 11 is a diagram illustrating an example of the number of pixels, and FIG. 11 is a diagram illustrating an example of the number of pixels necessary for reading when one line is used as an access unit in the vertical direction.

  As shown in FIG. 9, when a reference image is stored in a configuration of 4 pixels × 4 lines, a maximum of (4 × 4) × 9 = 144 pixels needs to be read for 9 × 9 reading. Further, as shown in FIG. 10, when a reference image is stored in a configuration of 8 pixels × 2 lines, readout of a maximum of (8 × 2) × 10 = 160 pixels is necessary for 9 × 9 readout. As shown in FIG. 11, when a reference image is stored in a configuration of 16 pixels × 1 line, readout of a maximum of (16 × 1) × 18 = 288 pixels is necessary for 9 × 9 readout.

  Thus, if the vertical unit is two lines or more (see FIG. 9 and FIG. 10), the total read amount can be reduced as compared with the case where the vertical unit is one line (see FIG. 11). As a result, it is possible to reduce the access amount when reading the reference image. Note that the same effect can be obtained for compression formats that are commonly used such as MPEG-2 and H.264 / AVC.

  Exemplary embodiments of an image decoding device, an image decoding method, and an image decoding program according to the present invention will be explained below in detail with reference to the accompanying drawings. In the following, the main terms used in the present embodiment, the outline and features of the image decoding apparatus according to the present embodiment, the configuration of the image decoding apparatus and the flow of processing will be described in order, and finally various types of the present embodiment will be described. A modified example will be described.

[Explanation of terms]
First, main terms used in this embodiment will be described. The “image decoding device (corresponding to the“ image decoding device ”recited in the claims”) used in the present embodiment encodes image data that is data such as a moving image or a still image. A stream that is the encoded data is received from an image encoding device such as an encoder that generates data via a network such as the Internet, the stream is analyzed, and a reference image that is decoded data that has already been decoded into image data A device that decodes encoded data, which is image data, using the encoded data. Specifically, a decoder or the like corresponds to this. The image decoding apparatus does not necessarily receive encoded data via a network. For example, a flexible disk (FD), a CD, an MO disk, a DVD disk, a magneto-optical disk, an IC card, an HDD, a USB memory, etc. May be received (input) from the physical medium.

  Here, the flow of processing from when the encoded data is transmitted from the image encoding device to when it is decoded by the image decoding device will be specifically described. The image encoding device generates a stream using a prediction format such as frame prediction or field prediction and motion compensation that is vector data representing the motion of a moving image, and transmits the stream to the image decoding device. Then, the image decoding apparatus analyzes the received stream and generates analyzed data including a prediction format and vector data. Subsequently, the image decoding apparatus acquires a reference image necessary for generating decoded data from the analyzed data from a storage unit such as a memory, and decodes the data using the acquired reference image and the analyzed data. To do.

  In recent years, since the image size has been increased to increase the resolution of moving image data to be encoded and transmitted, it is necessary to increase the speed of data decoding from the stream in the image decoding device. Yes. In other words, among the processes described above, the image decoding apparatus acquires a reference image from a storage unit such as a memory, and uses the acquired reference image and the analyzed data to decode the data and generate decoded data. Speeding up is necessary. For example, in the case of a television image, the digital broadcast has a resolution approximately six times that of the analog broadcast image size, and decoding requires six times the performance.

  In addition, the “stream” used in the present embodiment means that the image encoding device generates a prediction image in various prediction formats from the current image, and how much the object moves in which direction from the preceding and following images. Is encoded as motion vector data, and encoded data obtained by encoding information including the motion vector data calculated in this way and the remaining difference image data obtained by subtracting the vector data from the difference between the input image and the predicted image That is.

[Outline and Features of Image Decoding Device]
Next, the outline and features of the image decoding apparatus according to the first embodiment will be described with reference to FIGS. 1 and 2. FIG. 1 is a system configuration diagram illustrating an overall configuration of a system including an image decoding apparatus according to the first embodiment, and FIG. 2 is a diagram illustrating an example of a reference image area to be read.

  As shown in FIG. 1, the system includes an image encoding device that encodes image data and transmits a stream, and an image decoding device that decodes an accepted stream and generates decoded data (original image data). Are connected to each other via a network such as the Internet. The image decoding apparatus holds a decoded data memory for storing the decoded data.

  In such a configuration, as described above, the image decoding apparatus in this system receives encoded data from an image encoding apparatus that generates encoded data that is encoded image data, and has already been decoded. The outline is to generate new decoded data by using the reference image that is the decoded data and the encoded data, and in particular, it is possible to shorten the access time when reading the reference image. The point has the main feature.

  This main feature will be described in detail. The image decoding apparatus holds a plurality of memories that store reference images in a structure that matches each of prediction formats that can be used for encoded data (stream). To give a specific example, the image decoding apparatus supports a frame structure memory for storing a reference image in a frame structure corresponding to the frame prediction used for the encoded data, and a field prediction used for the encoded data. A field structure memory for storing a reference image in the field structure is held.

  In such a state, the image encoding device transmits a stream created by frame prediction to the image decoding device (see (1) in FIG. 1), and the image decoding device analyzes the received stream. The prediction format used for encoding is determined, the reference image stored in the determined prediction format is read from the memory, and new decoding is performed as original moving image data before being encoded from the stream and the reference image. Data is generated (see (2) to (4) in FIG. 1).

  Specifically, an image decoding apparatus that receives a stream is used to create the stream from data including a prediction format obtained by analyzing the stream, motion compensation, difference image data, and the like. It is determined whether the prediction format is frame prediction or field prediction. When the image decoding apparatus determines that the prediction format used for the received stream is frame prediction, as illustrated in (1) of FIG. 2 (actually required image information “◆” and “●”). The reference image necessary for decoding the stream and generating decoded data is acquired from the frame structure memory.

  Subsequently, the image decoding apparatus that has acquired the reference image generates decoded data by adding data including motion compensation and difference image data obtained from the stream to the acquired reference image. Here, when no prediction format is used for the received stream, or when a stream that does not require a reference image is received, the image decoding apparatus decodes the stream without acquiring the reference image. Then, new decoded data is generated, and the generated new decoded data is stored in the decoded data memory.

  Then, the image decoding device generates a reference image corresponding to each prediction format of each of the plurality of memories from the generated new decoded data, and stores the reference image in each of the plurality of memories (FIG. 1). (See (5) and (6)).

  Specifically, in the above example, the image decoding device generates a frame-structured reference image and a field-structured reference image from the generated new decoded data, and the generated frame-structured reference image is framed. The frame structure memory that stores the reference image in the structure is stored, and the generated field structure reference image is stored in the field structure memory.

  After that, the image decoding apparatus that has received the stream from the image encoding apparatus, as described above, from the prediction format obtained by analyzing (decoding) the stream, data including motion compensation and difference image data, It is determined whether the prediction format used for creating the stream is frame prediction or field prediction (see (7) and (8) in FIG. 1).

  When the image decoding apparatus determines that the prediction format used for the received stream is field prediction, as shown in (2) of FIG. 2 (actually required image information “◆” “●”). A reference image necessary for decoding the stream and generating decoded data is acquired from the field structure memory (see (9) in FIG. 1), and motion compensation, difference image data, and the like obtained from the stream are obtained. Decoded data is generated by adding the included data to the acquired reference image (see (10) in FIG. 1).

  Subsequently, the image decoding apparatus generates and stores a frame-structured reference image from the generated new decoded data in a frame-structure memory that stores the reference image in the frame structure, and stores the reference image in the field structure. Is generated and stored in the field structure memory for storing (see (11) and (12) in FIG. 1).

  As described above, the image decoding apparatus in this system stores a reference image in a memory or the like in advance according to various prediction formats used for frame prediction, field prediction, and the like. Even when the encoded stream is received, the reference image can be read out with the shortest access time regardless of the state of the reference image to be read. As a result, the access time can be shortened when reading the reference image. Is possible.

[Configuration of Image Decoding Device]
Next, the configuration of the image decoding apparatus shown in FIG. 1 will be described with reference to FIG. FIG. 3 is a block diagram illustrating the configuration of the image decoding apparatus according to the first embodiment. As illustrated in FIG. 3, the image decoding device 10 includes a communication control I / F unit 11, a storage unit 12, and a control unit 20.

  The communication control I / F unit 11 controls communication related to various types of information exchanged with an image encoding device connected to the image decoding device 10. Specifically, the communication control I / F unit 11 receives a stream obtained by encoding image data using motion compensation based on a prediction format from the image encoding device, and outputs the stream to a decoded data generation unit 21 of the control unit 20 described later. To do.

  The storage unit 12 stores data and programs necessary for various types of processing by the control unit 20, and particularly those closely related to the present invention include a decoded data memory 13, a frame structure memory 14, a field structure And a memory 15 for use.

  The decoded data memory 13 stores the decoded data decoded by the decoded data generation unit 21 of the control unit 20 described later. Specifically, the decoded data memory 13 stores decoded data that is original moving image data obtained by decoding the stream transmitted from the image encoding device by the decoded data generation unit 21 of the control unit 20. To do.

  The frame structure memory 14 stores a reference image in a frame structure. Specifically, as shown in FIG. 4, the frame structure memory 14 alternately stores the first field and the second field in each frame so as to correspond to the sequential scanning. FIG. 4 is a diagram illustrating an example of information stored in the frame structure memory.

  The field structure memory 15 stores a reference image in a field structure. As a specific example, as shown in FIG. 5, the field structure memory 15 stores the first field and the second field in each frame in different areas so as to correspond to the interlaced scanning. FIG. 5 is a diagram illustrating an example of information stored in the field structure memory. The frame structure memory 14 and the field structure memory 15 correspond to “a plurality of reference image storage units” recited in the claims.

  The control unit 20 has a control program such as an OS (Operating System), a program that defines various processing procedures, and an internal memory for storing necessary data. In particular, the control unit 20 is closely related to the present invention. Decoded data generating unit 21, read address creating unit (for frame structure) 23, read address creating unit (for field structure) 24, selection circuit 25, reference image storage unit 30, data buffer 31, and write address A creation unit (for frame structure) 32, a write address creation unit (for field structure) 33, and a selection circuit 34 are provided, and various processes are executed by these units.

  The decoded data generation unit 21 analyzes the stream received by the communication control I / F unit 11 to determine the prediction format used for encoding, and reads the reference image stored in the determined prediction format from the memory. Then, new decoded data is generated from the stream and the reference image, the generated decoded data is stored in the decoded data memory 13, and the decoded data is output to a reference image storage unit 30 described later.

  More specifically, the decoded data generation unit 21 determines whether or not a reference image is required to decode the stream received by the communication control I / F unit 11. When the decoded data generation unit 21 determines that no prediction format is used and the reference image is not required, the decoded data generation unit 21 decodes the stream without acquiring the reference image and generates new decoded data. The generated new decoded data is stored in the decoded data memory 13 and the decoded data is output to a reference image storage unit 30 described later.

  On the other hand, when one of the prediction formats is used and it is determined that a reference image is necessary, the decoded data generation unit 21 analyzes whether the prediction format used for encoding by analyzing the stream is a frame prediction field. Determine if it is a prediction.

  When it is determined that the frame prediction is made, the decoded data generation unit 21 reads out a reference image corresponding to the stream from the frame structure memory 14 that stores the reference image in the frame structure among the plurality of memories. The read address creating unit (for frame structure) 23 is instructed to calculate the address of the reference image to be read. The address generated by the read address generator (for frame structure) 23 is selected by the selection circuit 25, and the decoded data generator 21 uses the address to generate a reference image corresponding to the stream as the frame structure memory 14. Read from. The decoded data generation unit 21 that has read the reference image in this way generates new decoded data from the received stream and the read reference image, stores the generated decoded data in the decoded data memory 13, and The decoded data is output to a reference image storage unit 30 described later.

  On the other hand, when it is determined that the prediction format used for encoding by analyzing the stream is field prediction, the decoded data generation unit 21 stores the reference image in the field structure of the plurality of memories. In order to read a reference image corresponding to the stream from the memory 15, a read address creating unit (for field structure) 24 described later is instructed to calculate an address of the reference image to be read. Then, the address created by the read address creation unit (for field structure) 24 is selected by the selection circuit 25, and the decoded data generation unit 21 uses the address to generate a reference image corresponding to the stream as a field structure memory 15. Read from. The decoded data generation unit 21 that has read the reference image in this way generates new decoded data from the received stream and the read reference image, stores the generated decoded data in the decoded data memory 13, and The decoded data is output to a reference image storage unit 30 described later.

  The read address creating unit (for frame structure) 23 calculates an address for reading a reference image stored in the frame structure memory 14. Specifically, in the example described above, when the read address creating unit (for frame structure) 23 is instructed to calculate the address of the reference image to be read from the decoded data generating unit 21, the necessary reference image and access are accessed. The read start address is calculated in consideration of the unit, and the calculated address is output to the selection circuit 25 described later.

  The read address creation unit (for field structure) 24 calculates an address for reading a reference image stored in the field structure memory 15. More specifically, in the above example, when the read address creation unit (for field structure) 24 is instructed to calculate the address of the reference image to be read from the decoded data generation unit 21, the required reference image and access are accessed. The read start address is calculated in consideration of the unit, and the calculated address is output to the selection circuit 25 described later.

  The selection circuit 25 selects an address to be output to the storage unit 12 from the address created by the read address creation unit (for frame structure) 23 or the address created by the read address creation unit (for field structure) 24. Specifically, in the above example, when an address is input from the read address generation unit (for frame structure) 23, the selection circuit 25 outputs the address to the frame structure memory 14 and outputs a decoded data generation unit. 21 reads out a frame-structured reference image.

  When an address is input from the read address creation unit (for field structure) 24, the selection circuit 25 outputs the address to the field structure memory 15, and the decoded data generation unit 21 outputs the reference image of the field structure. Is read. It should be noted that the decoded data generation unit 21, the read address generation unit (for frame structure) 23, the read address generation unit (for field structure) 24, and the selection circuit 25 are described in “Decoded Data” in the claims. Corresponds to “generating means”.

  The reference image storage unit 30 generates a reference image corresponding to each prediction format of each of the plurality of memories from the new decoded data generated by the decoded data generation unit 21, and stores the reference image in each of the plurality of memories. To do. Specifically, in the above example, the reference image storage unit 30 to which the generated new decoded data is input from the decoded data generation unit 21 includes a frame-structured reference image, a field-structured reference image, and the like. And the generated reference images are stored in a data buffer 31 to be described later, and at the same time, the write address creation unit (for frame structure) 32 and the write address creation unit (for field structure) 33 start writing. To calculate.

  Then, the selection circuit 34 has an address created by the write address creation unit (for frame structure) 32 in the frame structure memory 14 and an address created by the write address creation unit (for field structure) 33 in the field structure memory. 15 to be output.

  The reference image storage unit 30 reads the generated frame structure reference image from the data buffer 31 using the address created by the write address creation unit (for frame structure) 32 and stores it in the frame structure memory 14. Using the address created by the write address creation unit (for field structure) 33, the generated reference image of the field structure is read from the data buffer 31 and stored in the field structure memory 15.

  The data buffer 31 temporarily stores the reference image generated by the reference image storage unit 30. Specifically, in the above example, when the generated frame-structured reference image and the field-structured reference image are input from the reference image storage unit 30, the data buffer 31 then receives a frame from the reference image storage unit 30. The input reference image is temporarily stored until it is output to the structure memory 14 and the field structure memory 15.

  The write address creation unit (for frame structure) 32 calculates an address for storing decoded data in the frame structure memory 14. Specifically, in the above example, the write address creation unit (for frame structure) 32 is instructed to calculate an address for storing the reference image of the frame structure generated by the reference image storage unit 30. The write start address is calculated, and the calculated address is output to the selection circuit 34 described later.

  The write address creation unit (for field structure) 33 calculates an address for storing the decoded data in the field structure memory 15. More specifically, the write address creation unit (for field structure) 33 is instructed to calculate an address for storing the reference image of the field structure generated by the reference image storage unit 30. The write start address is calculated, and the calculated address is output to the selection circuit 34 described later.

  The selection circuit 34 selects an address to be output to the storage unit 12 from the address created by the write address creation unit (for frame structure) 32 and the address created by the write address creation unit (for field structure) 33. Specifically, in the above example, when the reference image is written into the frame structure memory 14, the address created by the write address creation unit (for frame structure) 32 is selected and output to the frame structure memory 14. To do.

  Further, when the reference image is written in the field structure memory 15, the address created by the write address creation unit (for field structure) 33 is selected and output to the field structure memory 15. The reference image storage unit 30, the data buffer 31, the write address creation unit (for frame structure) 32, the write address creation unit (for field structure) 33, and the selection circuit 34 are included in the scope of claims. This corresponds to the “reference image storage means” described.

[Decoding process by image decoding apparatus]
Next, processing performed by the image decoding apparatus will be described with reference to FIGS. 6 and 7. FIG. 6 is a flowchart illustrating a flow of decoding processing in the image decoding apparatus according to the first embodiment, and FIG. 7 is a flowchart illustrating a flow of reference image storage processing in the image decoding apparatus according to the first embodiment. .

(Decryption process)
First, the flow of a decoding process in the image decoding apparatus according to the first embodiment will be described with reference to FIG. As shown in FIG. 6, when a stream is received from the image encoding device (Yes in step S601), the decoded data generation unit 21 of the image decoding device 10 needs a reference image to analyze and decode the stream. It is determined whether or not (step S602).

  If it is determined that a reference image is necessary to decode the stream (Yes in step S602), the decoded data generation unit 21 of the image decoding apparatus 10 uses frame prediction as the prediction format used for the stream. Or field prediction (step S603).

  Subsequently, when the prediction format used for the stream is frame prediction (Yes in step S604), the decoded data generation unit 21 calculates an address for reading the reference image in the read address creation unit (for frame structure) 23. Using the calculated address, the reference image corresponding to the stream is acquired from the frame structure memory 14 (step S605), and the decoded data generation unit 21 receives the received stream, the read reference image, From this, new decoded data is generated (step S607).

  On the other hand, when the prediction format used for the stream is not frame prediction (No in step S604), the decoded data generation unit 21 instructs the read address generation unit (for field structure) 24 to calculate an address for reading the reference image. Then, using the calculated address, the reference image corresponding to the stream is acquired from the field structure memory 15 (step S606), and the decoded data generation unit 21 creates a new one from the received stream and the read reference image. Decoded data is generated (step S607).

  Returning to step S602, when the reference image is not necessary to decode the stream (No at step S602), the decoded data generation unit 21 of the image decoding apparatus 10 decodes the stream without using the reference image (step S602). S607).

(Reference image storage processing)
Next, the flow of the reference image storing process in the image decoding apparatus according to the first embodiment will be described with reference to FIG. As illustrated in FIG. 7, when the generated new decoded data is received from the decoded data generation unit 21 (Yes in step S701), the reference image storage unit 30 refers to the frame structure reference image and the field structure from the decoded data. Images are generated, and the generated reference images are stored in the data buffer 31 (steps S702 and S703).

  Then, the reference image storage unit 30 instructs the write address creation unit (for frame structure) 32 and the write address creation unit (for field structure) 33 to calculate the write start address, and writes the write address creation unit (for frame structure). The generated frame structure reference image is written in the frame structure memory 14 using the address calculated in step 32 (step S704). Similarly, the address calculated by the write address creation unit (for field structure) 33 is used. Then, the generated reference image of the field structure is written in the field structure memory 15 (step S705).

[Effects of Example 1]
As described above, according to the first embodiment, the reference image is stored in a structure that matches each of the prediction formats that can be used for the stream, and when the stream is received, the stream is analyzed and used for encoding. A prediction format is determined, a reference image stored in the determined prediction format is read from the memory, new decoded data is generated from the stream and the reference image, and each of the plurality of memories is generated from the generated new decoded data. Since a reference image corresponding to each prediction format is generated and stored in each of a plurality of memories, the access time can be shortened when reading the reference image.

  For example, according to various prediction formats used for frame prediction, field prediction, etc., by storing a reference image in a memory or the like in advance, encoded data encoded using any prediction format is accepted. Even in such a case, the reference image can be read out with the shortest access time regardless of the state of the reference image to be read out. In addition, writing overhead can be eliminated by writing each reference image in a separate memory.

  In addition, as a result of writing the reference image to a plurality of memories, there is a concern that the access time is not shortened in total when considering the write access time to the memory and the read access time for reading the reference image from the memory. Generally, the amount of reading (number of readings) is overwhelmingly larger than the amount of writing (number of writings) to the memory, so the access time can be shortened even when considering both writing and reading. It is.

  To give a specific example, when MPEG-2 is used as an encoding technique (compression technique), the number of pixels to be written in a memory is 16 × 16 256 pixels when calculated per macroblock. Thus, the number of pixels read from the memory is 17 × 17 × 2 578 pixels. In addition, when H.264 / AVC is used as the encoding technique (compression technique), when the prediction unit is calculated as 8 × 8, the number of pixels to be written in the memory is 64 × 8 × 8, The number of pixels read from the memory is 338 pixels of 13 × 13 × 2. As described above, even if the number of times of writing to the memory is doubled, the number of pixels from which data is read out from the memory is larger, so that the access time can be shortened considering both writing and reading.

  Also, according to the first embodiment, when a stream is received, the stream is analyzed to determine whether the prediction format used for encoding is frame prediction or field prediction. The frame structure reference image is read out from the frame structure memory 14, new decoded data is generated from the stream and the reference image, and in the case of field prediction, the field structure memory 15 A reference image is read, new decoded data is generated from the stream and the reference image, a frame-structured reference image and a field-structured reference image are generated from the generated new decoded data, and the generated frame structure The reference image and the reference image of the field structure are stored in the frame structure memory 14 and the field structure memory 15. Since stores Les, when read the reference image, it is possible to further shorten the access time.

  For example, frame prediction and field prediction, which are generally used as encoding technologies (compression technologies), are stored in memory in advance, so that compression such as MPEG-2 and H.264 / AVC is often used. When encoded data using a technique is received, the access time can be further shortened when the reference image is read. In addition, writing overhead can be eliminated by writing each reference image in a separate memory.

  Further, according to the first embodiment, as shown in FIG. 8, when only a frame structure or a field structure memory is provided, if a stream having a format different from that of the memory is received, a large number of pixels are wasted. Will be read. For example, when the memory structure is a frame structure and the stream generated by the frame prediction is received, the image decoding device may read 288 pixels, but it is generated by the field prediction instead of the frame prediction. If the stream is received, the image decoding apparatus reads 480 pixels. Therefore, as described in the above-described embodiments, when the memory having the frame structure and the field structure is provided, the image decoding apparatus can always select and read out the minimum number of pixels. FIG. 8 is a diagram illustrating an example of the number of pixels necessary for reading.

  Further, according to the first embodiment, reference images are generated for each prediction format, and the generated reference images are stored in the respective prediction formats, so that the reference image writing time can be shortened. Is possible.

  For example, when there are two types of prediction formats, frame prediction and field prediction, if two memories for storing reference images of the respective prediction formats are prepared, a reference image for each generated prediction format is obtained. As a result of being able to start writing to two memories at the same time, it is possible to further shorten the reference image writing time compared to writing two types of reference images to one memory.

  Further, according to the first embodiment, since the reference image is written in the macro block, the number of pixels to be written matches the number of pixels actually written, but the reference image is not necessarily read out in units of macro blocks. Since the number of pixels to be read does not necessarily match the number of pixels that are actually read, by storing a reference image in a memory or the like in advance, encoded data encoded using any prediction format was accepted. Even in this case, as a result of minimizing the number of pixels to be read, the access time can be shortened.

  For example, when the encoding technique is H.264 / AVC, it is necessary to read out a 9 pixel × 9 line reference image in order to decode a 4 pixel × 4 line image (16 pixels in total). 9 to 16 illustrate the case of reading a reference image of 9 pixels × 9 lines. However, since reading is performed in units of 4 pixels × 4 lines, it is read with 81 pixels which is the actual number of pixels. However, a maximum of 240 pixels and a minimum of 144 pixels are read out. Therefore, even when the reference image is written twice or more into the memory, the number of pixels to be written is 162 pixels or more, which is larger than the minimum number of pixels to be read. In other words, by storing the reference image in advance in a memory or the like, the number of pixels to be read can be minimized even when encoded data encoded using any prediction format is received, It is possible to shorten the access time.

  Although the embodiments of the present invention have been described so far, the present invention may be implemented in various different forms other than the embodiments described above. Therefore, as shown below, different embodiments will be described by dividing into (1) storage area, (2) access unit, (3) number of memories, (4) system configuration, and (5) programs. .

(1) Storage Area For example, in the first embodiment, when the generated reference image is stored in the frame structure memory and the field structure memory, the luminance component and the color difference component of the reference image are stored in the same storage area. However, the present invention is not limited to this, and the luminance component and the color difference component may be stored separately. By doing so, it is possible to further shorten the access time when referring to only the necessary components.

  For example, in addition to analyzing data such as analyzing the decoding status, when it is necessary to refer to the luminance component and chrominance component, the access time for referencing only the luminance component or referencing only the chrominance component is shortened. It is possible. Further, it is possible to cope with the case where the prediction format is different between the luminance component and the color difference component.

(2) Access Unit In the first embodiment, the case of reading without specifying the access unit for reading the reference image has been described. However, the present invention is not limited to this, and the frame structure memory and the field structure For each storage format of the main memory, two or more lines in the vertical direction may be used as an access unit. In this way, it is possible to reduce the access amount when reading the reference image.

(3) Number of memories In the first embodiment, the case where there are two memories for storing the reference images has been described. However, the present invention is not limited to this, and the number of memories is not limited. . For example, as described in the first embodiment, even when two types of reference images of a frame structure and a field structure are stored, they may be stored in one memory.

  By doing so, it is possible to prevent the memory area from being wasted. For example, if the reference image is generated for each prediction format but the capacity is small, storing in a plurality of memories increases the free capacity and wastes the capacity. In such a case, it is possible to prevent the memory area from being wasted by dividing the area into one memory and storing the reference image.

(4) System configuration, etc. Also, among the processes described in the present embodiment, all (for example, stream analysis process, address calculation process, etc.) or part of the processes described as being automatically performed are manually performed. You can also do it. In addition to the processing procedures, control procedures, specific names, and information including various data and parameters (for example, FIG. 2, FIG. 4, FIG. 5, FIG. 9 to FIG. 18 etc.) Can be arbitrarily changed unless otherwise specified.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. It is possible to configure by integrating (for example, integrating the decoded data generation unit and the reference image storage unit). Further, all or any part of each processing function performed in each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware by wired logic.

(5) Program The communication control method described in the present embodiment can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. This program can be distributed via a network such as the Internet. The program can also be executed by being recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, an MO, and a DVD and being read from the recording medium by the computer.

  As described above, the image decoding apparatus, the image decoding method, and the image decoding program according to the present invention receive encoded data from an image encoding apparatus that generates encoded data that is encoded image data. It is useful for generating new decoded data using a reference image that is already decoded data and the encoded data, and is particularly suitable for shortening the access time when reading the reference image.

1 is a system configuration diagram illustrating an overall configuration of a system including an image decoding device according to Embodiment 1. FIG. It is a figure which shows the example of the area | region of the reference image read. 1 is a block diagram illustrating a configuration of an image decoding device according to Embodiment 1. FIG. It is a figure which shows the example of the information stored in the memory for frame structures. It is a figure which shows the example of the information stored in the memory for field structures. 6 is a flowchart illustrating a flow of a decoding process in the image decoding apparatus according to the first embodiment. 6 is a flowchart illustrating a flow of reference image storage processing in the image decoding apparatus according to the first embodiment. It is a figure which shows the example of the pixel number required for read-out. It is a figure which shows the example of the pixel number required for reading when 4 lines are made into an access unit in the perpendicular direction. It is a figure which shows the example of the pixel number required for reading when 2 lines are made into an access unit in the perpendicular direction. It is a figure which shows the example of the pixel number required for reading when 1 line is made into the access unit in the perpendicular direction. It is a figure for demonstrating a prior art. It is a figure for demonstrating a prior art. It is a figure for demonstrating a prior art. It is a figure for demonstrating a prior art. It is a figure for demonstrating a prior art. It is a figure for demonstrating a prior art. It is a figure for demonstrating a prior art. It is a figure for demonstrating a prior art.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image decoding apparatus 11 Communication control I / F part 12 Memory | storage part 13 Memory for decoding data 14 Memory for frame structure 15 Memory for field structure 20 Control part 21 Decoding data generation part 22 Address generation part (for decoding data)
23 Read address generator (for frame structure)
24 Read address generator (for field structure)
25 selection circuit 30 reference image storage unit 31 data buffer 32 write address creation unit (for frame structure)
33 Write address generator (for field structure)
34 Selection circuit

Claims (8)

The encoded data is received from an image encoding device that generates encoded data that is encoded image data, and a reference image that is decoded data that has already been decoded into the image data and the encoded data are used. An image decoding device for generating new decoded data,
Reference image storage means for storing the reference image in a structure that matches each of the prediction formats that can be used for the encoded data;
When the encoded data is received, the encoded data is analyzed to determine the prediction format used for encoding, the reference image stored in the determined prediction format is read from the reference image storage means, Decoded data generating means for generating new decoded data from the encoded data and the reference image;
Reference image storage means for generating a reference image corresponding to each of the prediction formats from the new decoded data generated by the decoded data generation means, and storing the generated reference image in the reference image storage means,
An image decoding apparatus comprising: When the decoded data generation unit receives the encoded data, the decoded data generation unit analyzes the encoded data and determines whether the prediction format used for encoding is frame prediction or field prediction. When it is prediction, a reference image having a frame structure is read from the reference image storage means, and new decoded data is generated from the encoded data and the reference image. When it is field prediction, the reference image A field structure reference image is read from the storage means, and new decoded data is generated from the encoded data and the reference image.
The reference image storage means generates a frame structure reference image and a field structure reference image from the new decoded data generated by the decoded data generation means, and generates the generated frame structure reference image and field structure reference. The image decoding apparatus according to claim 1, wherein an image is stored in each of the reference image storage units. The reference image storage means includes a single storage unit,
The reference image storage means generates a reference image corresponding to the prediction format, and stores the generated reference image in the reference image storage means by dividing an area. The image decoding apparatus according to claim 1 or 2. The reference image storage means includes a plurality of storage units that store reference images for each prediction format,
The reference image storage unit generates a reference image for each prediction format, and stores the generated reference image in a reference image storage unit that stores the reference image in each prediction format. Item 3. The image decoding device according to Item 1 or 2.   The reference image storage means generates the reference image corresponding to each prediction format of each of the plurality of reference image storage means from the new decoded data generated by the decoded data generation means, and stores the reference image. The image decoding apparatus according to claim 1, wherein the image is stored by dividing an area into a luminance component and a color difference component.   When the decoded data generation unit receives the encoded data, the decoded data generation unit analyzes the encoded data to determine a prediction format used for encoding, and generates a reference image corresponding to the determined prediction format as the plurality of reference images. 6. When reading from a reference image storage unit, an access unit includes two or more lines in the vertical direction with respect to the storage format of the plurality of reference image storage units. The image decoding device described. The encoded data is received from an image encoding device that generates encoded data that is encoded image data, and a reference image that is decoded data that has already been decoded into the image data and the encoded data are used. An image decoding method suitable for generating new decoded data,
Reference image storage means for storing the reference image for each prediction format used for the encoded data;
When the encoded data is received, the encoded data is analyzed to determine the prediction format used for encoding, the reference image stored in the determined prediction format is read from the reference image storage means, A decoded data generation step of generating new decoded data from the encoded data and the reference image;
A reference image storage step of generating a reference image corresponding to each of the prediction formats from the new decoded data generated by the decoded data generation step, and storing the generated reference image in the reference image storage unit;
The image decoding method characterized by including. The encoded data is received from an image encoding device that generates encoded data that is encoded image data, and a reference image that is decoded data that has already been decoded into the image data and the encoded data are used. An image decoding program for causing a computer to generate new decoded data,
Reference image storage means for storing the reference image for each prediction format used for the encoded data;
When the encoded data is received, the encoded data is analyzed to determine the prediction format used for encoding, the reference image stored in the determined prediction format is read from the reference image storage means, A decoded data generation procedure for generating new decoded data from the encoded data and the reference image;
A reference image storing procedure for generating a reference image corresponding to each of the prediction formats from the new decoded data generated by the decoded data generating procedure, and storing the generated reference image in the reference image storage unit;
An image decoding program that causes a computer to execute the above.

Images