JP2009194405A - Dynamic image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To suppress a data transfer amount of a dynamic image decoder. <P>SOLUTION: A dynamic image processing apparatus includes at least two first image processing units and a second image processing unit which process dynamic image data, and includes at least two first and second memories to store first processed image data output from the first image processing units, wherein the first image processing units read the second memory and the second image processing unit reads the first memory in a process handing the first processed image data as a source, and the second memory includes smaller memory access units than the first memory. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image decoding apparatus that decodes or expands a compressed image, and more particularly, to an image decoding technique suitable for decoding a compressed image that complies with the MPEG (Moving Picture Experts Group) standard.

  2. Description of the Related Art Conventionally, when moving images are recorded or recorded on a recording medium, there is a technique for performing compression in accordance with an MPEG standard such as the MPEG2 video standard (see Non-Patent Document 1) or the MPEG4 AVC standard (see Non-Patent Document 2). An image decoding apparatus that decodes an image compressed according to these standards is also known.

  Such an image decoding apparatus reads out a variable-length encoded stream related to a compressed image taken into a memory such as a large-capacity DRAM (Dynamic Random Access Memory), reads the stream from the memory, and performs variable-length decoding. Motion vectors, block data, and the like are extracted for each block, and a reference image specified in accordance with the motion vector in the memory is referred to (FIG. 7). This process is referred to as “motion compensation process”), and the decoded image obtained as a result of the motion compensation process is recorded in the memory. Here, the decoded image group that has been decoded and stored in the memory is used as a reference image group in subsequent decoding of a compressed image and as an image group for display on a display or the like.

  It is generally known that the moving image decoding process requires a high data transfer amount for memory access.

  As an example, FIG. 11 shows a requested data transfer amount of an MPEG2 moving picture decoder as shown in FIG. The process of requesting memory access in the MPEG2MP @ HL moving picture decoder is as follows: (1) Reading a bitstream; (2) Reading a reference picture for motion compensation; (3) Storing a decoded picture; (4) Decoding There are four types of display image reading.

  Assuming that the maximum bit rate of the bit stream is 25 Mbps, the reading of the reference image is a maximum of 4 fields in one frame period, and the access unit to the memory is 16 bytes, the data transfer amount required for each memory access is (1) bit stream Is about 3 MB / sec, (2) is about 567 MB / sec for reading the reference image, (3) is about 90 MB / sec for storing the decoded image, and (4) is about 90 MB per second for reading the image for image display. / Sec, and a data transfer capacity of a maximum of 750 MB / sec is required.

  In order to realize a high data transfer amount in the system including the moving picture decoder and the image display, a method for suppressing the peak of the data transfer amount as a whole process and a memory system having a high data transfer amount are used. There are methods. Among them, for example, Patent Document 1 has been proposed as a technique for suppressing the peak of the data transfer amount as a whole process.

In Patent Document 1, an apparatus including an audio decoder, a moving image decoder, and an image display performs processing with a large data transfer amount of the audio decoder during a blanking period of the image display. In the data transfer period, the peak of the data transfer amount is suppressed by performing processing with a small data transfer amount of the audio decoder (FIG. 14).
ISO / IEC 13818-2 International Standard MPEG-2 Video ISO / IEC 14496-10 International Standard Information technology-Coding of Audio-Visual Objects-Part 10: Advanced Video Coding Japanese Patent No. 3532796

  In general, in a video decoder, the amount of data transfer increases in accordance with the size of an image to be decoded. Therefore, further measures are required to target the size of a decoding target image up to HD (High Density). However, Patent Document 1 does not disclose a method using a memory system having a high data transfer amount.

  The memory transfer capability is proportional to the operating frequency of the memory device and the memory access unit. For this reason, to realize a memory system having a high data transfer capability, there are i) a method of increasing the memory access unit and ii) a method of increasing the operating frequency of the memory device.

  First, i) a method of increasing the memory access unit will be described.

  For example, if the memory accessed by the video decoder is simply i) the memory access unit is increased from 16 bytes to 32 bytes, the data transfer capability is doubled. In addition, the data transfer amount for (1) reading a bit stream, (3) storing a decoded image, and (4) reading a decoded image for image display is the same as that when the memory access unit is 16 bytes. ), (3), and (4) can easily cope with an increase in data transfer amount.

  However, (2) the reading of the reference image is about 1135 MB / sec, which is about 567 MB / sec higher than that in the case of the 16-byte memory access unit. This is because (2) the reading of the reference image data is an access to a different address for each reference block, and the invalid data reading area 970 increases with an increase in the memory access unit. (FIG. 10). For this reason, in the method i), power consumption and heat generation are unnecessarily increased with an increase in the amount of invalid data transfer.

  The other, ii) increasing the operating frequency of the memory device also causes an increase in power consumption and heat generation.

  Simply, when the memory access unit is reduced from 16 bytes to 4 bytes, invalid transfer at the time of transfer of reference image data is suppressed, and (2) the data transfer amount for reading the reference image is 280 MB / sec, and the memory access of 16 bytes About 290 MB / sec can be suppressed compared with the unit. However, since the memory access unit is reduced, the data transfer capability is reduced to 1/4, and in order to realize access to all the video decoders with a memory having a small access unit, ii) the operating frequency of the memory device is greatly increased. It is necessary to increase the power consumption and heat generation amount.

  In order to solve the above problems, an object of the present invention is to provide a memory system that efficiently realizes data transfer of a video decoder.

  A moving image processing apparatus comprising at least two first image processors for processing moving image data and a second image processor, wherein the first processed image data output from the first image processor Among the processes using the first processed image data as a source, the first image processor reads out from the second memory, and stores at least two first memory and second memory for storing The second image processor performs reading from the first memory, and the second memory provides a moving image processing apparatus characterized in that the memory access unit is smaller than that of the first memory.

  According to the present invention, the moving image processing apparatus includes a moving image decoder, an image display, a reference image memory, and a shared memory, and among the decoded images by the moving image decoder, the reference image is a reference image. The moving image decoder reads from the reference image memory, and the image display device reads from the shared memory.

  According to the present invention, the discontinuous access to the memory (2) reading of the reference image can suppress invalid data transfer and the total data transfer amount because the memory access unit is small. Power consumption and heat generation can be suppressed.

  In addition, (1) reading a bit stream, (3) storing a decoded image, and (4) reading a decoded image for displaying an image are accesses to a memory having a large memory access unit. . As a result, only the access in (2) is performed on a memory having a small memory access unit, and ii) the improvement in the operating frequency of the memory device can be minimized, so that power consumption and heat generation can be suppressed.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
An MPEG2 decoding apparatus will be described. FIG. 2 shows a conventional video decoding device. FIG. 1 is a configuration diagram of a moving picture decoding apparatus according to Embodiment 1 of the present invention.

  In addition, the thing of the same structure is numbered the same, and abbreviate | omits description. The difference is the addition of the reference image memory 300.

  The image decoding unit 100 in FIG. 1 reads the bit stream in the shared memory 200, decodes the image, and stores the decoded image in the reference image memory 300 and the shared memory 200. The display processing unit 400 reads out the decoded images in the shared memory 200 in the order necessary for display, and displays them on a monitor or the like. The VLD processing unit 150 in FIG. 1 performs variable-length decoding on the bit stream read from the DMA 110, and converts it into block data subjected to quantization / frequency conversion and motion vector data.

  The quantized / frequency converted block data is transferred to the inverse quantization / inverse frequency conversion processing unit 160, and the motion vector data is transferred to the DMA 120. The inverse quantization / inverse frequency conversion processing unit 160 performs inverse quantization and inverse frequency conversion, and transfers the processed block data to the MC processing unit 180 or the DMA 130. The MC processing unit 180 targets the block data predicted between the screens, reads out the reference image data indicated by the motion vector data from the reference image memory 300 using the DMA 120, performs the motion compensation process, and decodes the decoded block. Output data. The image data decoded by the DMA 130 is stored in the shared memory 200, and an image that may be referred to is also stored in the reference image memory 300.

  Next, the processing flow will be described with reference to FIG.

  FIG. 4 is a processing flow in the conventional video decoding device.

  In the VLD processing unit 150, processes 510, 515, 520, 525, and 528 are performed. In processes 510 and 515, the picture type of the decoding target block is determined. In processes 520 and 525, it is determined whether decoding is possible by intra prediction. If the prediction is not in-screen, the number of motion vectors is also determined at 528. Depending on the number of motion vectors, the number of transfer processing of reference image data in the DMA 120 is also increased. If it is a block that performs intra prediction, it is transferred to the DMA 130 after being processed by the inverse quantization / inverse frequency conversion processing unit 160.

  On the other hand, in the case of inter-screen prediction, after the processing of the inverse quantization / inverse frequency conversion processing unit 160, the MC processing unit 180 performs motion compensation processing according to the number of motion vectors in processing 565 and 568, and the DMA 130 Forwarded to

  In the case of an image that may be referred to as an I picture or a P picture according to the type of the picture type in the processes 511 and 518, the DMA 130 transfers the image to the reference image memory 300 and the shared memory 200. In the case of a B picture, the DMA 130 transfers only to the shared memory 200. The shared memory 200 has a memory access unit of 16 bytes, and the reference image memory 300 has a memory access unit of 4 bytes.

  The transfer of the reference image in the conventional method is performed from the shared memory 200, and an extra transfer area is increased (FIG. 10), and the data transfer amount required for the MPEG2 MP @ HL decoding process is as shown in FIG. To become bigger. However, in this method, the reference image is read out from the reference image memory 300 having a small access width, and the extra transfer area is reduced (FIG. 9). As a result, the amount of data transfer required for the decoding process can be kept small as shown in FIG.

  As described above, all the images that may be referred to are also arranged in the reference image memory 300, but some reference images may be arranged only in the shared memory 200. In that case, it is only necessary to select NO in processes 511 and 518.

  Further, although the memory access unit of the shared memory 200 is 16 bytes and the reference image memory 300 is 4 bytes, the memory access unit may be changed. Furthermore, in the arrangement of the decoded image in the reference image memory 300, processing may be performed so as to overwrite the image area in which the possibility of being referred to becomes zero.

  In MPEG2, the only images that can be referred to are the two most recently decoded I and P pictures. Therefore, an image area for three images is secured in the reference image memory 300 and processed. At 580, the addresses may be managed so as to be sequentially overwritten as a FIFO.

(Embodiment 2)
An MPEG4AVC decoding apparatus will be described.

  The configuration is shown in FIG. 5, and an INTRA processing unit 170 and a DBF processing unit 190 are added compared to the configuration of the first embodiment. In addition, the thing of the same structure attaches the same number, and abbreviate | omits description.

  The VLD processing unit 150 in FIG. 5 performs variable-length decoding and arithmetic decoding on the bit stream read from the DMA 110, and converts the block data into quantized / frequency converted block data and motion vector data. The quantized / frequency converted block data is transferred to the inverse quantization / inverse frequency conversion processing unit 160, and the motion vector data is transferred to the DMA 120. The inverse quantization / inverse frequency conversion processing unit 160 performs inverse quantization and inverse frequency conversion, and transfers the processed block data to the MC processing unit 180 or the INTRA processing unit 170.

  The MC processing unit 180 targets the block data predicted between the screens, reads out the reference image data indicated by the motion vector data from the reference image memory 300 using the DMA 120, performs the motion compensation process, and decodes the decoded block. The data is transferred to the DBF processing unit 190.

  The INTRA processing unit 170 performs an INTRA prediction process on the block data predicted in the screen, and transfers the block data to the DBF processing unit 190.

  The DBF processing unit 190 performs deblocking filter processing, stores the image data decoded by the DMA 130 in the shared memory 200, and stores images that may be referred to in the reference image memory 300.

  Next, the processing flow will be described with reference to FIG.

  Also in this embodiment, processes 560, 565, 568, 570, 575, and 578 in the INTRA processing unit 170 and the DBF processing unit 190 are added to the first embodiment.

  In the VLD processing unit 150, processes 520, 525, 528, 710, and 715 are performed. In processes 710 and 715, the slice type of the decoding target block is determined.

  In processes 520 and 525, it is determined whether decoding is possible by intra prediction. If the prediction is not in-screen prediction, the number of motion vectors is also determined in process 528. Depending on the number of motion vectors, the number of transfer processing of reference image data in the DMA 120 is also increased. In the case of a block that performs intra prediction, after the processing of the inverse quantization / inverse frequency conversion processing unit 160, INTRA prediction is performed in the INTRA processing unit 170 in process 560 and transferred to the DBF processing unit 190. .

  On the other hand, in the case of inter-screen prediction, after the processing of the inverse quantization / inverse frequency conversion processing unit 160, in the processing 565 and 568, motion compensation processing according to the number of motion vectors in the MC processing unit 180 is performed, Transferred to the DBF processing unit 190. In processes 570, 575, and 578, deblock filter processing is performed in the DBF processing unit 190 and transferred to the DMA 130.

  In the DMA 130, if there is a block that may be referred to in the processes 720, 721, and 722, the block is transferred to the reference image memory 300 and the shared memory 200.

  In MPEG4AVC, nal_ref_idc is not 0 for an image that may be referred to. In the case of an image that is not referenced, the image is transferred only to the shared memory 200. The shared memory 200 has an access width of 16 bytes, and the reference image memory 300 has an access width of 4 bytes.

  The transfer of the reference image by the conventional method is performed from the shared memory 200, the extra transfer area is increased (FIG. 10), the reference block unit is small (FIG. 8), and MPEG4AVC High Profile Level 4 is used. The amount of data transfer required for the decryption process increases as shown in FIG. However, in this method, the reference image is read out from the reference image memory 300 having a small access width, and the extra transfer area is reduced (FIG. 9). As a result, the amount of data transfer required for the decoding process can be kept small.

  As described above, all the images that may be referred to are also arranged in the reference image memory 300, but some reference images may be arranged only in the shared memory 200. In such a case, NO may be selected in processes 720, 721, and 722. Further, although the access width of the shared memory 200 is 16 bytes and the access width of the reference image memory 300 is 4 bytes, the access width may be changed.

  Furthermore, in the arrangement of the decoded image in the reference image memory 300, processing may be performed so as to overwrite the image area in which the possibility of being referred to becomes zero. When the reference picture management method is the moving window memory management method, the address may be managed so that the area of the short-time reference memory is overwritten in the process 580 sequentially.

  If the reference picture management method is an adaptive memory management method, the designated short-time reference image can be overwritten according to the ID if it is 1. In the case of 2, the designated long-time reference image can be overwritten. In the case of 5, all images can be overwritten. Further, even when an IDR picture is decoded, all images can be overwritten.

  The image decoding apparatus according to the present invention can be used in a moving image processing apparatus such as a DVD player, a digital television, and a mobile phone that reproduces a moving image compressed according to the MPEG standard.

Configuration diagram of moving picture decoding unit in Embodiment 1 of the present invention Configuration diagram of conventional video decoding unit Processing flow diagram of moving picture decoding unit in Embodiment 1 of the present invention Process flow diagram of conventional video decoding unit The block diagram of the moving image decoding part in Embodiment 2 of this invention Processing flow diagram of moving picture decoding unit in embodiment 2 of the present invention Relationship diagram between MPEG2 decoding target image and reference image Relationship diagram between MPEG4AVC decoding target picture and reference picture Reference block in the reference image memory 300 and invalid transfer area diagram Reference block and invalid transfer area diagram in shared memory 200 Data transfer amount graph of conventional video decoder and image display in MPEG2 Data transfer amount graph of moving picture decoder and picture display in conventional MPEG2 and in the present invention Data transfer amount graph of moving picture decoder and picture display in conventional MPEG4AVC and in the present invention Data transfer amount graph in the system of Patent Document 1

Explanation of symbols

100 Image decoding unit 110, 120, 130 DMA
150 VLD processing unit 160 Inverse quantization / inverse frequency conversion processing unit 170 INTRA processing unit 180 MC processing unit 200 Shared memory 300 Reference image memory 400 Display processing unit 500 Decoding processing start 505 Decoding processing end 510, 511, 515, 518 Picture type Determination 520,525 Intra-screen prediction / inter-screen prediction determination 528 Reference number determination 550,555 Reference image data acquisition processing from reference image memory 300 560 Intra processing 565, 568 MC processing 580 Decoded image storage processing to reference image memory 300 650, 655 Reference image data acquisition processing from shared memory 200 685 Decoded image storage processing to shared memory 200 710, 715 Slice type determination 720, 721, 722 Reference determination 800 Decoding target image 850 Decoding target block 888 It can vector 900 reference image 950 reference block 970 invalid transfer area

Claims (6)

A moving image processing apparatus comprising at least two first image processors for processing moving image data and a second image processor,
Holding at least two first storage means for storing the first processed image data output from the first image processor, and a second storage means;
Among the processes using the first processed image data as a source,
The first image processor reads from the second storage means,
The second image processor reads from the first storage means,
The moving image processing apparatus, wherein the second storage means has a smaller access unit than the first storage means. The first image processor is a video decoder, and the first processed image data is a decoded image;
The process using the first processed image data as a source is a motion compensation process,
The moving image processing apparatus according to claim 1, wherein the second image processor is a process of reading image data for display. A video decoder;
First storage means;
Holding a second storage means;
The first storage means stores all the decoded images by the video decoder,
The second storage means stores a decoded image that may be referred to by the video decoder,
The motion compensation process by the video decoder reads a reference image from a second storage unit, and the second storage unit has a smaller access unit than the first storage unit. Processing equipment. The moving image processing apparatus according to any one of claims 1 to 3, wherein the first image processor is an MPEG2 moving image decoder. The moving image processing apparatus according to any one of claims 1 to 3, wherein the first image processor is an MPEG4AVC moving image decoder. 6. The moving picture decoder is controlled so as to overwrite a decoded picture that has a possibility of being referred to of the decoded pictures stored in the second storage means. The moving image processing apparatus according to any one of the above.

Images