JPH08163571A - Picture decoder - Google Patents

Abstract

PURPOSE: To provide a picture decoder capable of eliminating a waiting time part for not performing processings. CONSTITUTION: The read accesss of display pictures and the write access of bit streams are turned to a first order at all times. Or, for the succeeding order, based on the storage amount of a scanning conversion buffer provided inside a scanning converter 30 and the storage amount of a buffer E 75 for storing reproducing pictures before writing them in a memory 50 provided inside a memory interface part 70, a priority order judgement control part 70A decides the orders of the read access of the bit streams, the read access of reference pictures and the write access of the reproducing pictures.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image decoding device for decoding an encoded image signal.

[0002]

2. Description of the Related Art When transmitting or storing digitally represented image data, encoding is performed to reduce the amount of data. As an encoding method, there is a method of reducing redundancy by utilizing temporal or spatial correlation of image information (image data).

As a method of utilizing the temporal correlation, there is a method of encoding a difference between two consecutive screens (frames) or detecting a motion of an image to perform motion compensation.
As a method of utilizing spatial correlation, an image is divided into blocks of a predetermined size (for example, 8 pixels in each of the vertical direction and the horizontal direction), the data in each block is orthogonally converted, and the conversion coefficient is scan-converted. However, there is one that performs variable-length coding (for example, rearranges in order from low-frequency components to high-frequency components). An image coding method (hereinafter abbreviated as MPEG2), which is being standardized by the Moving Picture Experts Group (MPEG), uses the above two methods together. The provisional recommendation for MPEG2 is “Generic Coding of Movi
ISO / IEC entitled "ng Pictures and Associated Audio"
It is described in 13818-2.

The present invention is applicable to the process of restoring all MPEG2 images, so the decoding process will be described.

FIG. 4 shows an example of the configuration of an image decoding apparatus for decoding data encoded by such a method. FIG.
In the decoding process, the buffer control unit 1, the variable length decoder 2, the scan converter 3, the inverse quantizer 4, the inverse DCT unit 5, the motion compensation image reproducing unit 6 (6A is an image reproducing unit), and the memory interface unit 7 perform decoding processing. Is executed. Reference numeral 50 denotes a memory, which includes a buffer memory 51 and a frame memory (three I, P, B frame memories described later) 52, 53, 54.
Consists of In addition, reference numeral 100 denotes an input bitstream representing an encoded image, and 200 denotes a reproduced image.

Next, the operation will be described. The input bit stream 100 is stored in the buffer memory 51 as the data 40 through the memory interface unit 7 under the control of the buffer memory control unit 1. The data 41 read from the buffer memory 51 is the variable length decoder 2
Variable-length decoding is performed.

Although not all data is variable length coded, it is assumed that fixed length code is also decoded by this variable length decoder 2. Next, the order of the data is rearranged by the scan converter 3 and then dequantized by the dequantizer 4. Next, the inverse DCT unit 5 performs inverse discrete cosine transform. The motion-compensated image reproduction unit 6 performs reproduction in consideration of the movement of the image. In MPEG2, a temporally intermediate frame (here, B frame) is predicted from both a temporally previous frame (here, I frame) and a temporally later frame (here, P frame). Therefore, to reproduce the B frame, the I frame and P that have been decoded in advance are used.
The predicted frame data 42, 43 of the frame is passed from the frame memories 52, 53 through the memory interface unit 7,
It is necessary to read (in MPEG2, P which is later in time)
The frame is decoded prior to the B frame). The B frame is reproduced by the motion compensation image reproducing unit 6 due to the prediction frame data 42 and 43 and the prediction error output from the inverse DCT unit 5, and is written as reproduction pixel data 44 in the frame memory 54 through the memory interface unit 7. The I, P, and B frames in the frame memories 52, 53, and 54 are read out from the memories in a predetermined order (the B frame data 45 is read out in FIG. 4), and the reproduced image 200 is output.

The above-mentioned data write access to the memory 50 is managed by the memory interface section 7 and is read and written using the same data bus 46, so that the memory 50 is accessed by a certain processing request. Is
Even if there is a request for other processing, access cannot be made and the internal processing stops. Therefore, the memory 50
In order to store the data for the time for which access to the memory is waited, to further reduce the width of the data bus interfacing with the memory 50 and to increase the data transfer rate with the memory 50, a built-in buffer in the memory interface unit 7 is provided. Is stored in the built-in buffer and then written in the memory 50. Data read from the memory 50 is once stored in the built-in buffer and then used in each process.

The present invention can be applied to a device for processing all MPEG2 images, but as an example, consider the case of reproducing an NTSC image. As shown in FIG. 5, one frame of an NTSC image consists of horizontal 720 pixels and vertical 480 lines. This is divided horizontally and vertically into 16 pixels each. 1
A unit of division is called a macroblock (hereinafter abbreviated as MB). The NTSC image is divided into 45 MB horizontally, 30 MB vertically, and 1350 MB in total. In MPEG2, a group of macroblocks closed in one horizontal row is called a slice, and an NTSC image is divided into at least 30 slices.

The image data that has been variable-length coded naturally has a large amount of data in the order of I frame, P frame, and B frame, and the amount of data per 1 MB in each frame varies depending on the MB. For example, suppose that only a certain amount of data can be read per read access from the buffer memory 51, and MBs having a large amount of data concentrated in a certain position on the screen in the variable length coded state. In order to decode that part at a constant rate by the variable length decoder 2, the buffer memory 5
The frequency of occurrence of 1 read access increases. On the other hand, when MBs having a small amount of data are concentrated at a certain position on the screen, the frequency of read access to the buffer memory 51 decreases in order to decode that portion at a certain constant rate by the variable length decoder 2.

The frame predicted between frames is
The prediction method differs for each MB. For example, M in B frame
B is intra-frame prediction, forward inter-frame prediction from a temporally previous I frame or P frame, temporally late I
Backward inter-frame prediction from frame or P frame,
The prediction method with the smallest prediction error is selected from the bidirectional prediction from the I frame or P frame that precedes in time and the I frame or P frame that follows in time. Therefore, the frequency of accessing the memory 50 to decode one frame is not uniform even within one frame. Assuming that only a certain amount of data can be read out per reference image read access, if the MBs encoded by bidirectional inter-frame prediction are concentrated in a certain portion of the screen, the motion compensation image reproducing unit 6 In order to reproduce the MB image at a certain constant rate, the frequency of memory access for reading the reference image from the frame memories 52 and 53 increases. On the contrary, when MBs coded by intra-frame prediction, that is, intra MBs, are concentrated on a certain part of the screen, naturally the reference image is not required to reproduce the MBs by the motion compensation image reproducing unit 6. Therefore, reference image read access is not performed.

If the display image cannot be read out from any of the frame memories 52, 53, 54 without interruption of the display image, that is, without the internal buffer in the memory interface section 7 underflowing. I won't. Also, the input bitstream 100
Must be written to the buffer memory 51 at the input rate, that is, can be processed without overflow of the internal buffer in the memory interface section 7. However, access to the other memory in the memory 50 is performed by the MB as described above.
Depending on the amount of data in the bitstream that makes up the MB, and whether the MB prediction is encoded by intraframe prediction or interframe prediction,
The occurrence frequency of the read access of the buffer memory necessary for reproducing the image of the MB and the read access of the reference image dynamically changes.

Therefore, the transfer rate of any MB is designed to be large so that the entire memory can be accessed within a certain fixed time determined in units of MB. For example, as shown in FIG. 6, one frame time is divided by the number of MB,
In order to process all memory accesses within the divided time, this 1 MB time is further added to the bitstream writing period, the bitstream reading period, the reference image reading period, the reproduction image writing period, and the display. Dividing into the image reading period, each memory access accesses the memory 50 within a predetermined time. In this case, the effective efficiency is lowered because the time for unnecessary memory access is paused.

[0014]

As described above, the conventional image decoding apparatus needs to be designed with a transfer rate higher than the actual access rate because it includes a waiting time portion in which no processing is performed. However, there is a big drawback that the internal buffer of the memory interface unit 7 becomes large because the interval between each memory access becomes large.

It is an object of the present invention to provide an image decoding device that solves the above-mentioned drawbacks of conventional devices.

[0016]

According to an image decoding apparatus of the present invention, in an apparatus for predicting an image frame from a decoding prediction frame and decoding encoded image data, a plurality of accesses to a memory are performed. The access control is performed by prioritizing the access, and the priority determination control unit controls the priority according to the storage amount of the built-in buffer.

Further, the priority order determination control unit makes a plurality of accesses to the memory by the storage amount of the internal buffer used for scan conversion and the storage amount of the internal buffer which is stored before writing the reproduced image in the memory. It is controlled by giving priority to each access.

Further, more specifically, the read access of the display image and the write access of the bit stream are always the highest priority, the storage amount of the internal buffer used for scan conversion is small, and the reproduction image is stored before being written in the memory. When the accumulated amount of the built-in buffer is small, the priority order is determined in the order of the bitstream read access, the reference image read access, and the reproduced image write access, followed by the highest order, and is used for scan conversion. In a state in which the storage amount of the buffer is large and the storage amount of the built-in buffer for storing the reproduced image before writing to the memory is small, the reference image read access, the reproduced image write access,
When the priority order is determined in the order of bitstream read access and the accumulated amount of the internal buffer used for scan conversion is small and the accumulated amount of the internal buffer that stores the reproduced image in the memory is large, Next to the order, the playback image write access, reference image read access, and bit stream read access are prioritized in order, and the accumulated amount of the internal buffer used for scan conversion is large, and the reproduced image is accumulated before writing to the memory. In the state where the storage amount of the built-in buffer is large, the priority order is determined in the order of the playback image write access, the reference image read access, and the bit stream read access after the priority order.

[0019]

According to the present invention, since a plurality of accesses to the memory are performed in accordance with a predetermined priority order, the transfer rate can be determined without including the waiting time for no processing, and the low transfer rate can be realized. It is possible to reduce the amount of built-in buffer.

The priority order is determined by the storage amount of the built-in buffer used for scan conversion and the storage amount of the built-in buffer that is stored before writing the reproduced image in the memory.

Further, the priority order is specifically determined by
When the amount of storage in the built-in buffer used for scan conversion and the amount of storage in the built-in buffer that is stored before writing the playback image to memory are both smaller than the capacity, when one is large and the other is small, and when one is small and the other is large , When both are large.

[0022]

【Example】

[Embodiment 1] FIG. 1 shows an embodiment of the present invention.
Reference numeral 0 is a scan converter, which is basically the same as the scan converter 3 in FIG. 4, but also performs control peculiar to the present invention as described later. Reference numeral 70 denotes a memory interface unit, which is basically the same as the memory interface unit 7 in FIG. 4, but also performs control peculiar to the present invention as described later. A control signal 130 is a control signal peculiar to the present invention and is sent from the scan conversion unit 30 to the memory interface unit 70 as described later. The same reference numerals as those in FIG. 4 indicate the same parts.

FIG. 2 shows a detailed configuration example of the scan conversion unit 30. The scan conversion buffer 30A as a built-in buffer used for scan conversion and a buffer amount monitoring unit 30B are provided, and the variable length decoder 2 is provided. The data input from is stored in the scan conversion buffer 30A in the scan converter 30, and the buffer amount monitoring unit 30B in the scan converter 30 monitors the buffer amount and outputs the control signal 1
30 is output to the memory interface unit 70.

FIG. 3 shows a detailed configuration example of the memory interface unit 70. The memory interface unit 70 includes a priority determination control unit 70A, a buffer amount monitoring unit 70B, and buffers A71 to F76.

To explain the operation, the input bit stream 100 of FIG.
Before being written in the buffer memory 51 in the memory interface unit 70, the data is stored in the buffer A71 in the memory interface unit 70 and stored in the memory 5
The data read from the buffer memory 51 in 0 is also temporarily stored in the buffer B 72 and output to the buffer control unit 1. The data (data in the scan conversion buffer 30A) decoded by the variable length decoder 2 is inversely quantized by the inverse quantizer 4, and further inversely DCT processed by the inverse DCT unit 5, and the buffer C73 and the buffer D74. The image is reproduced by the data supplied to the image reproducing unit 6A in the motion compensation image reproducing unit 6 and is stored in the buffer E75. Therefore, the variable-length decoded data (scan conversion buffer 3
(Data in 0A) is either inversely quantized and inversely DCT processed, or the image is reproduced from the buffer C73 and the buffer D74.
Ideally, the data supplied to A are input at synchronized timings.

The image referred to by the motion-compensated image reproducing unit 6 is also read out from the memory 50, buffered in the buffer C73 and the buffer D74 in the memory interface unit 70, and then output, and also the motion-compensated image. The image reproduced by the reproducing unit 6 is temporarily stored in the buffer E75 and then stored in the frame memory 52 in the memory 50 shown in FIG.
~ 54. Further, the display image is also output after being buffered by the buffer F76 in the memory interface unit 70.

The access for writing data to the memory 50 and the access for reading data from the memory 50 are
It occurs when the amount of each buffer reaches a certain amount. When a certain access is generated, when the generated access is being executed, it waits until the access is completed, and when there is no access being executed, the access is started. If another access occurs while waiting for the end of the previous access, the access is started in the order of priority of the access after the end of the previous access.

The priority order of access is determined as follows. In addition, the built-in buffer (3
The storage amount of 0A) is A, and the storage amount (E75) of the built-in buffer which is stored before writing the reproduced image in the memory is B.

The memory access for reading the display image and the memory access for writing the input bit stream 100 must be able to be processed at a certain constant rate, so these two accesses are always priorities, that is, one of them is priority 1. , The other has priority 2, and the priority of the read access of the bit stream, the read access of the reference image, and the write access of the reproduced image is determined by the scan conversion buffer 30A and the built-in buffer (buffer E75 in FIG. 3) for accumulating the reproduced image. ) The buffer amount is determined as follows. (1) When A is small and B is large The image reproduced by the motion compensation image reproducing unit 6 is the buffer E7.
When the reference image is read from the memory 50, the reference image is used even if the reference image is read out from the memory 50, and the reproduced image data cannot be written in the buffer E75. The reproduced image cannot be written in the buffer E75. Similarly, even if the bit stream is read by the variable length decoder 2 and the decoded data is input to the motion compensation image reproducing unit 6, the reproduced image cannot be written in the buffer E75. Therefore, when the data cannot be written in the buffer E75, the priority order of the write access of the reproduced image is set to the priority order 3.

In a state in which the data decoded by the variable length decoder 2 can be written, that is, in a state in which A is small and a reproduced image cannot be written in the memory 50, the processing of the variable length decoder 2 can be performed even if the reference image is read. Since it is considered that the image cannot be reproduced because it is not in time, the read access of the bit stream is set to priority 4, and the read access of the reference image is set to priority 5. (2) When both A and B are large, the image reproduced by the motion compensation image reproducing unit 6 is the buffer E7.
5 cannot be written to and the data converted by the variable length decoder 2 cannot be written to the scan conversion buffer 30A, even if the bitstream is read,
Since the data decoded by the variable-length decoder 2 cannot be written in the scan conversion buffer 30A, the reference image read access has priority 4, and the bitstream read access has priority 5. For the same reason as the above (1), the priority order of the write access of the reproduced image is set to the priority order 3. (3) When A is large and B is small The image reproduced by the motion compensation image reproducing unit 6 is the buffer E
In the state in which the reference image can be written in 75 and the data decoded by the variable length decoder 2 cannot be written in the scan conversion buffer 30A, it is considered that the reference image is not read in time, and thus the reference image read access is set to priority 3. And Even if the bitstream is read, the data decoded by the variable length decoder 2 cannot be written in the scan conversion buffer 30A. Therefore, the write access to the reproduced image is set to priority 4, and the read access to the bitstream is set to priority 5. (4) When both A and B are small, the image reproduced by the motion compensation image reproducing unit 6 is the buffer E
In the state where the data can be written in 75 and the data decoded by the variable length decoder 2 can be written in the scan conversion buffer 30A, it is considered that the bitstream is not read in time, so the bitstream read access is set to priority 3. To do. Further, since it is considered that the reproduction image can be written in the buffer E75, it is considered that there is no need to urgently write the reproduction image. Therefore, the reference image read access has priority 4, and the restored image write access has priority 5. To do. [Embodiment 2] This embodiment is an embodiment in which access control by priority order is performed by interruption, and other processing and method of assigning priority order are the same as those in the first embodiment.

When a certain access occurs, the access that occurred before is being executed, and if the priority is higher than that of the executing access, the executing access is suspended and the new access is processed first. After the processing is completed, the interrupted access is restarted. When the priority of the new access is lower than that of the access being executed, the new access is started after waiting for the end of the access being executed.

[0032]

As described above, in the image decoding apparatus according to the present invention, in an apparatus for predicting an image frame from a decoding prediction frame and decoding encoded image data, a plurality of accesses to a memory are performed. , It has a priority judgment control unit that controls each access by prioritizing it, and controls this priority by the accumulated amount of the built-in buffer. Therefore, when various memory accesses occur, it is possible to efficiently prioritize multiple memory accesses. It is possible to determine the order and improve the effective efficiency of memory access.

Then, the priority determination control unit determines a plurality of accesses to the memory by the storage amount of the internal buffer used for scan conversion and the storage amount of the internal buffer to be stored before writing the reproduced image in the memory. Since each access is controlled by prioritizing it, the transfer rate can be determined without including the waiting time for no processing, so that decoding can be realized at a low rate.

Further, the determination of the priority order is, specifically,
Whether the amount of storage in the internal buffer used for scan conversion and the amount of storage in the internal buffer that is stored before writing the playback image to the memory are both small, one is large and the other is small, and one is small and the other is large. Since it is determined by the above, the interval of each memory access changes as necessary, and therefore the amount of built-in buffer of the memory access unit can be reduced.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention.

FIG. 2 is a block diagram showing details of essential parts of the embodiment shown in FIG.

FIG. 3 is a block diagram showing details of a main part of the embodiment of FIG.

FIG. 4 is a block diagram showing a configuration example of a conventional image decoding device.

FIG. 5 is an explanatory diagram of macroblocks in one frame of an NTSC image.

FIG. 6 is a diagram for explaining a conventional memory access procedure.

[Description of Codes] 1 buffer control unit 2 variable length decoder 3 scan converter 4 inverse quantizer 5 inverse DCT unit 6 motion compensation image reproducing unit 7 memory interface unit 30 scan converter 30A scan conversion buffer 30B buffer amount monitoring unit 40 data 41 data 42 predicted frame data 44 reproduced image data 45 data 46 data bus 50 memory 51 buffer memory 52 frame memory 53 frame memory 54 frame memory 70 memory interface unit 70A priority order determination control unit 70B buffer amount monitoring unit 71 buffer A 72 Buffer B 73 Buffer C 74 Buffer D 75 Buffer E 76 Buffer F 100 Input bitstream 200 Playback image

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Shigeru Komatsu 4-36-19 Yoyogi, Shibuya-ku, Tokyo Within Graphics Communication Laboratories, Inc. (72) Inventor, Kaoru Kawamura 4-yoyogi, Shibuya-ku, Tokyo 36-19 No. 19 in Graphics Communications Laboratories Co., Ltd. (72) Inventor Ryuji Nishito 4-36-19 Yoyogi, Shibuya-ku, Tokyo Within Graphics Communications Laboratories Co., Ltd. (72) Inventor Katsumi Goto Tokyo 4-36-19 Yoyogi, Shibuya-ku, Tokyo Within Graphics Communications Laboratories, Inc. (72) Inventor Takayuki Kobayashi 4-36-19 Yoyogi, Shibuya-ku, Tokyo Graphics Communications, Inc. Laboratories in the

Claims (3)

[Claims] 1. In an apparatus for predicting an image frame from a decoding predicted frame and decoding encoded image data, a plurality of accesses to a memory are controlled by prioritizing each access. An image decoding apparatus, comprising: a priority determination control unit that controls the priority according to the amount of storage in a built-in buffer. 2. The image decoding apparatus according to claim 1, wherein the priority determination control unit determines the storage amount of the internal buffer used for scan conversion and the storage amount of the internal buffer to be stored before writing the reproduced image in the memory. An image decoding device, characterized in that a plurality of accesses to a memory are controlled by prioritizing each access. 3. The image decoding apparatus according to claim 1 or 2, wherein the priority determination control unit always sets the read access of the display image and the write access of the bit stream as the highest priority, and the built-in buffer used for scan conversion. Accumulation is small,
In the state where the storage amount of the built-in buffer that stores the reproduced image before writing to the memory is small, the priority order is followed by the bit stream read access, the reference image read access, and the reproduced image write access. The amount of storage in the internal buffer used for scan conversion is large,
When the amount of accumulated data in the built-in buffer that stores the reproduced image in the memory is small, the reference image read access, the bitstream read access, and the reproduced image write access are given priority in this order. The rank is decided, the accumulated amount of the internal buffer used for scan conversion is small,
In a state where the amount of data stored in the built-in buffer that stores the replay image before writing it in the memory is large, the highest priority is given to the replay image write access, the bitstream read access, and the reference image read access. The amount of storage in the internal buffer used for scan conversion is large,
When the amount of data stored in the built-in buffer that stores the reproduced image before writing to the memory is large, the priority order is determined in the order of the reproduced image write access, the image read access, and the bitstream read access after the priority order. An image decoding device characterized by: