JP3078992B2 - Image decoding device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image decoding apparatus for decoding an encoded image signal.

[0002]

2. Description of the Related Art When digitally represented image data is transmitted or stored, encoding is performed to reduce the amount of data. As an encoding method, there is a method of reducing the degree of redundancy by utilizing temporal or spatial correlation of image information (image data).

As a method utilizing the temporal correlation, there are methods of encoding a difference between two consecutive screens (frames) and detecting motion of an image to perform motion compensation.
As a method utilizing spatial correlation, an image is divided into blocks of a predetermined size (for example, 8 pixels each in the vertical and horizontal directions), data in the blocks are subjected to orthogonal transform, and transform coefficients are scan-converted. Some (for example, rearrange from low frequency components to high frequency components) and perform variable length coding. An image coding system (hereinafter, abbreviated as MPEG2), which is being standardized by the Moving Picture Experts Group (MPEG), uses the above two methods in combination. The provisional recommendation for MPEG2 is “Generic Coding of Movi
ISO / IEC entitled “ng Pictures and Associated Audio”
13818-2.

[0004] The present invention decodes all MPEG2 images.
Because it is adaptable to the process of issue, explaining the decoding process.

FIG. 4 shows an example of the configuration of an image decoding apparatus for decoding data encoded by such a method. FIG.
, Decoding processing is performed by a buffer control unit 1, a variable length decoder 2, a scan converter 3, an inverse quantizer 4, an inverse DCT unit 5, a motion compensation image reproducing unit 6 (6A is an image reproducing unit), and a memory interface unit 7. Is executed. Reference numeral 50 denotes a memory, which includes a buffer memory 51 and frame memories (three I, P, and B frame memories described later) 52, 53, and 54.
Consists of Reference numeral 100 denotes an input bit stream representing an encoded image, and reference numeral 200 denotes a reproduced image.

Next, the operation will be described. The input bit stream 100 is stored as data 40 in the buffer memory 51 through the memory interface unit 7 under the control of the buffer memory control unit 1. The data 41 read from the buffer memory 51 is
Is subjected to variable-length decoding.

Although not all data is variable-length coded, it is assumed that fixed-length codes are also decoded by the variable-length decoder 2. Next, after the order of the data is rearranged by the scan converter 3, the data is inversely quantized by the inverse quantizer 4. Next, the inverse DCT unit 5 performs an inverse discrete cosine transform. The motion-compensated image reproducing unit 6 performs reproduction in consideration of the motion of the image. In MPEG2, a temporally intermediate frame (here, a B frame) is predicted from both a temporally earlier frame (here, an I frame) and a temporally later frame (here, a P frame). Therefore, to reproduce the B frame, the previously decoded I frame and P frame
The predicted frame data 42, 43 of the frame is passed from the frame memories 52, 53 through the memory interface unit 7,
It is necessary to read out (in MPEG2, P
The frame is decoded prior to the B frame). The B-frame is reproduced by the motion-compensated image reproducing unit 6 based on the predicted frame data 42 and 43 and the prediction error output from the inverse DCT unit 5, and is written as reproduced pixel data 44 to the frame memory 54 through the memory interface unit 7. The I, P, and B frames in the frame memories 52, 53, and 54 are read from each memory in a predetermined order (the B frame data 45 is read in FIG. 4), and a reproduced image 200 is output.

The above-mentioned data write access to the memory 50 is managed by the memory interface unit 7, and the read / write is performed using the same data bus 46. Is
Even if there is a request for other processing, access cannot be made, and internal processing stops. Therefore, the memory 50
In order to store data for the time required for access to the memory 50, to further reduce the data bus width for interfacing with the memory 50, and to increase the data transfer rate with the memory 50, an internal buffer is provided in the memory interface unit 7. Is stored in the built-in buffer, and then written to the memory 50. Data read from the memory 50 is temporarily stored in the built-in buffer and used for each process.

Although the present invention can be applied to an apparatus for processing any MPEG2 image, as an example, consider the case of reproducing an NTSC image. As shown in FIG. 5, one frame of the NTSC image is composed of 720 horizontal pixels and 480 vertical lines. This is divided into 16 pixels both horizontally and vertically. 1
The division unit is called a macroblock (hereinafter abbreviated as MB). The NTSC image is divided into 45 MB in width and 30 MB in height, that is, 1350 MB in total. In MPEG2, a set of macroblocks closed in one horizontal line is called a slice, and an NTSC image is divided into at least 30 slices.

Of course, variable-length coded image data has a large data amount in the order of an I frame, a P frame, and a B frame, and the data amount per MB varies in each frame depending on the MB. For example, assuming that only a certain amount of data can be read per one read access from the buffer memory 51, and MBs with a large amount of data are concentrated at a certain position on the screen in the state of being subjected to variable length coding. In order to decode the portion at a fixed rate by the variable length decoder 2, the buffer memory 5
The frequency of occurrence of one read access increases. Conversely, when MBs with a small amount of data are concentrated at a certain position on the screen, the frequency of read accesses to the buffer memory 51 decreases in order for the variable length decoder 2 to decode the portion at a certain constant rate.

[0011] The inter-frame predicted frame is
The prediction method differs for each MB. For example, M in the B frame
B is an intra-frame prediction, a forward inter-frame prediction from a temporally previous I-frame or a P-frame,
Backward inter-frame prediction from frame or P-frame,
A prediction method with the smallest prediction error is selected from bi-directional prediction from a temporally preceding I frame or P frame and a temporally subsequent I frame or P frame. Therefore, the frequency of accessing the memory 50 to decode one frame is not uniform even within one frame. Assuming that only a certain amount of data can be read per one reference image read access, and MBs encoded by bidirectional inter-frame prediction concentrate on a certain portion of the screen, the motion compensated image reproducing unit 6 In order to reproduce an MB image at a certain constant rate, the frequency of memory accesses for reading reference images from the frame memories 52 and 53 increases. Conversely, if MBs encoded by intra-frame prediction, that is, intra MBs, are concentrated on a certain portion of the screen, a reference image is not required to reproduce the MB with the motion-compensated image reproducing unit 6 as a matter of course. Therefore, read access of the reference image is not performed.

The reading of the display image from any of the frame memories 52, 53, and 54 cannot be processed without interruption of the display image, that is, without underflow of the built-in buffer in the memory interface unit 7. No. Also, the input bit stream 100
Must be written to the buffer memory 51 at the input rate, that is, it must be able to be processed without overflowing the built-in buffer in the memory interface unit 7. However, access to other memories in the memory 50 is performed as described above.
, And whether the MB prediction is coded by intra-frame prediction or inter-frame prediction, etc.
The frequency of occurrence of the read access of the buffer memory and the read access of the reference image required to reproduce the image of the MB changes fluidly.

Therefore, the transfer rate of any MB is designed to be large so that all memory accesses can be performed within a certain fixed time determined in MB units. For example, as shown in FIG. 6, the time of one frame is divided by the number of MBs,
This 1 MB time is further divided into a bit stream write period, a bit stream read period, a reference image read period, a reproduced image write period, and a display time so that all memory accesses can be processed within the divided time. Divided into image reading periods, each memory access accesses the memory 50 within a predetermined time. In this case, since the time for unnecessary memory access is stopped, the effective efficiency is reduced.

[0014]

As described above, in the conventional image decoding apparatus, it is necessary to design at a transfer rate higher than the actual access rate because of including a waiting time portion where no processing is performed. In addition, since the interval between each memory access is increased, the internal buffer of the memory interface unit 7 has a large disadvantage.

An object of the present invention is to provide an image decoding apparatus which has solved the above-mentioned drawbacks of the conventional apparatus.

[0016]

According to the present invention, there is provided an image decoding apparatus for predicting an image frame from a predicted frame to be decoded and decoding encoded image data. It has a priority determination control unit that controls access by assigning a priority and controls the priority based on the storage amount of the built-in buffer.

[0017] Then, the priority determination control unit includes a storage amount of the internal buffer used for scan conversion, the accumulation amount of the internal buffer that accumulates before writing a reproduced image in the memory, a plurality of access of the memory, This is controlled by assigning a priority to each access.

That is, in the control , the read access of the display image and the write access of the bit stream are always given the highest priority, the storage amount of the built-in buffer used for scan conversion is small, and the reproduced image is stored before being written to the memory. When the accumulated amount of the built-in buffer is small, the priority is determined in the order of bit stream read access, reference image read access, and reproduced image write access next to the first order, and the internal buffer used for scan conversion is determined. In the state where the storage amount of the internal image buffer is large and the storage amount of the built-in buffer for storing the reproduced image before writing to the memory is small, the read access of the reference image, the write access of the reproduced image,
The priority is determined in the order of bit stream read access, and when the storage amount of the built-in buffer used for scan conversion is small and the storage amount of the built-in buffer for storing the reproduced image before writing to the memory is large, the priority is determined. write access reproduced image Following ranking, a bit stream read access, the reference image reading Accession
In the state where the storage amount of the internal buffer used for scan conversion is large and the storage amount of the internal buffer for storing the reproduced image before writing to the memory is large, the priority is determined after the priority order. The priority is determined in the order of image write access, reference image read access, and bit stream read access.

[0019]

According to the present invention, since a plurality of accesses to the memory are performed in accordance with a predetermined priority, the transfer rate can be determined without including a waiting time during which no processing is performed, and a low transfer rate can be realized. It is possible to reduce the amount of the built-in buffer.

The priority is determined based on the storage amount of the built-in buffer used for scan conversion and the storage amount of the built-in buffer for storing the reproduced image before writing it in the memory.

That is , the priority order is determined, specifically,
When the storage amount of the built-in buffer used for scan conversion and the storage amount of the built-in buffer that stores the reproduced image before writing it to the memory are both smaller than the capacity, when one is large and the other is small, one is small and the other is large , Are both larger.

[0022]

【Example】

[Embodiment 1] FIG. 1 shows an embodiment of the present invention.
Reference numeral 0 denotes a scan converter, which is basically the same as the scan converter 3 in FIG. 4, but also performs control unique to the present invention as described later. Reference numeral 70 denotes a memory interface unit, which is basically the same as the memory interface unit 7 in FIG. 4, but also performs control unique to the present invention as described later. Reference numeral 130 denotes a control signal unique to the present invention, which will be described later, and is sent from the scan conversion unit 30 to the memory interface unit 70. The same reference numerals as in FIG. 4 denote the same parts.

FIG. 2 shows details of an example of the configuration inside the scan conversion unit 30. The scan conversion unit 30 includes a scan conversion buffer 30A as a built-in buffer used for scan conversion and a buffer amount monitoring unit 30B. Is stored in a scan conversion buffer 30A in the scan converter 30, and a buffer amount monitoring unit 30B in the scan converter 30 monitors this buffer amount and outputs the control signal 1
30 is output to the memory interface unit 70.

FIG. 3 shows details of an example of the configuration of the memory interface unit 70. The memory interface unit 70 includes a priority determination control unit 70A, a buffer amount monitoring unit 70B, and buffers A71 to F76.

The operation will be described. The input bit stream 100 shown in FIG.
Before writing in the buffer memory 51 in the memory interface unit 70, the data is stored in the buffer A71 in the memory interface unit 70, and
The data read from the buffer memory 51 in 0 is also temporarily stored in the buffer B 72 and output to the buffer control unit 1. The data (data in the scan conversion buffer 30A) decoded by the variable length decoder 2 is inversely quantized by the inverse quantizer 4 and further subjected to inverse DCT processing by the inverse DCT unit 5, and the buffer C73 and the buffer D74. The image is reproduced by the data supplied to the image reproducing unit 6A in the motion-compensated image reproducing unit 6 and stored in the buffer E75. Therefore, the variable-length decoded data (scan conversion buffer 3
0A) is inversely quantized and inversely DCT-processed, or data is transferred from the buffer C73 and the buffer D74 to the image reproducing unit 6
Ideally, data supplied to A is input at synchronized timing.

An image referred to by the motion-compensated image reproducing unit 6 is also read out from the memory 50, buffered by the buffers C73 and D74 in the memory interface unit 70, and then output. The image reproduced by the reproducing unit 6 is temporarily stored in the buffer E75, and then stored in the frame memory 52 in the memory 50 shown in FIG.
~ 54. Further, the display image is also output after being buffered by the buffer F76 in the memory interface unit 70.

The access for writing data to the memory 50 and the access for reading data from the memory 50 are as follows.
It occurs when each buffer amount reaches a certain fixed amount. When a certain access occurs, if the generated access is being executed, the process waits until the access ends, and if no access is being executed, the access is started. If another access occurs while waiting for the end of the previous access, the access is started in the order of priority of the access after the end of the previous access.

The priority of access is determined as follows. The built-in buffer (3
A is the storage amount of 0A), and B is the storage amount of the built-in buffer E75 that stores the reproduced image before writing it in the memory.

Since the memory access for reading the display image and writing the input bit stream 100 must always be able to be processed at a certain fixed rate, these two accesses are always given priority, that is, one of them is given priority 1. The priority of read access of the bit stream, read access of the reference image, and write access of the reproduced image is determined by the scan conversion buffer 30A and the built-in buffer for storing the reproduced image (the buffer E75 in FIG. 3). ) Is determined as follows according to the buffer amount. (1) When A is small and B is large: The image reproduced by the motion compensation image reproducing unit 6 is stored in the buffer E7.
5 and still remain untransferred to the memory 50, and in a state where the reproduced image data cannot be written to the buffer E75, even if the reference image is read from the memory 50, this reference image is used. Cannot be written to the buffer E75. Similarly, even if the bit stream is read by the variable length decoder 2 and the decoded data is input to the motion compensated image reproducing unit 6, it cannot be written into the buffer E75 where the reproduced image is stored. Therefore, when data cannot be written into the buffer E75, the priority of the write access to the reproduced image is set to the priority 3.

In a state where the data decoded by the variable length decoder 2 can be written, that is, when A is small and a reproduced image cannot be written in the memory 50, even if a reference image is read, the processing of the variable length decoder 2 is not performed. Since it is considered that the image cannot be reproduced because it is not in time, the read access of the bit stream is set to the priority 4 and the read access of the reference image is set to the priority 5. (2) When both A and B are large The image reproduced by the motion compensated image reproducing unit 6 is stored in the buffer E7.
5 and the data decoded by the variable length decoder 2 cannot be written to the scan conversion buffer 30A, even if the bit stream is read,
Since the data decoded by the variable-length decoder 2 cannot be written to the scan conversion buffer 30A, the read access of the reference image is given priority 4 and the read access of the bit stream is given priority 5. For the same reason as the above (1), the priority of the write access of the reproduced image is set to the priority 3. (3) When A is large and B is small The image reproduced by the motion compensation image reproducing unit 6 is stored in the buffer E
75 and the data decoded by the variable-length decoder 2 cannot be written to the scan conversion buffer 30A, it is considered that the reference image is not read in time. And Even if the bit stream is read, the data decoded by the variable length decoder 2 cannot be written to the scan conversion buffer 30A. Therefore, the write access to the reproduced image is given priority 4 and the read access to the bit stream is given priority 5. (4) When both A and B are small The image reproduced by the motion compensated image reproducing unit 6 is stored in the buffer E
75 and the data decoded by the variable length decoder 2 can be written to the scan conversion buffer 30A, it is considered that the reading of the bit stream is not in time. I do. Further, the fact that the reproduced image can be written in the buffer E75 is considered to mean that there is no need to hurry to write the reproduced image. Therefore, the read access of the reference image is given priority 4 and the write access of the reproduced image is given priority 5. I do. [Embodiment 2] This embodiment is an embodiment in which access control based on priority is performed by interruption, and other processes and the manner of assigning priorities are the same as those in the first embodiment. When an access occurs and a previously generated access is being executed, if the priority is higher than the currently executed access, the currently executed access is suspended, the new access is processed first, and the After the termination, resume the interrupted access. If the new access has a lower priority than the ongoing access,
Wait for the end of the current access and start a new access.

When a certain access occurs and a previously generated access is being executed, if the priority is higher than the currently executed access, the currently executed access is temporarily suspended, and the new access is processed first. After the processing, the interrupted access is resumed. If the new access has a lower priority than the access being executed, the new access is started after waiting for the end of the access being executed.

[0032]

As described above, according to the image decoding apparatus of the present invention, in an apparatus for predicting an image frame from a predicted frame for decoding and decoding coded image data, a built-in buffer used for scan conversion is used. Storage
Built-in buffer for storing raw images before writing to memory
Depending on the storage capacity of the
Prioritization controlled by assigning priorities to access
A control unit is provided.
The amount of data stored in the internal buffer used for scan conversion
Built-in buffer for storing raw images before writing to memory
When the amount of stored keys is small, one is large and the other is small.
When one is small and the other is large, are both large?
The interval between each memory access is necessary
The internal buffer of the memory access unit
Can be reduced.

[0033]

[0034]

[Brief description of the drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention.

FIG. 2 is a block diagram showing details of a main part of the embodiment of FIG. 1;

FIG. 3 is a block diagram showing details of a main part of the embodiment of FIG. 1;

FIG. 4 is a block diagram illustrating a configuration example of a conventional image decoding device.

FIG. 5 is an explanatory diagram of a macroblock in one frame of an NTSC image.

FIG. 6 is a diagram for explaining a conventional memory access procedure.

[Explanation of symbols]

Reference Signs List 1 buffer control unit 2 variable length decoder 3 scan converter 4 inverse quantizer 5 inverse DCT unit 6 motion compensation image reproduction unit 6A image reproduction unit 7 memory interface unit 30 scan converter 30A scan conversion buffer 30B buffer amount monitoring unit 40 Data 41 Data 42 Predicted frame data 44 Reconstructed image data 45 Data 46 Data bus 50 Memory 51 Buffer memory 52 Frame memory 53 Frame memory 54 Frame memory 70 Memory interface unit 70A Priority determination control unit 70B Buffer amount monitoring unit 71 Buffer A 72 Buffer B 73 Buffer C 74 Buffer D 75 Buffer E 76 Buffer F 100 Input bit stream 200 Playback image

──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Shigeru Komatsu 4-36-19 Yoyogi, Shibuya-ku, Tokyo Inside Graphics Communication Laboratories Co., Ltd. (72) Inventor Yoshikazu Kawamura Shibuya-ku, Tokyo 4-36-19 Yoyogi Inside Graphics Communication Laboratories Inc. (72) Inventor Ryuji Saito 4-36-19 Yoyogi Shibuya-ku Tokyo Metropolitan Graphics Communications Laboratories Inc. (72) Inventor Katsumi Goto 4-36-19, Yoyogi, Shibuya-ku, Tokyo Inside Graphics Communication Laboratories Co., Ltd. (72) Takayuki Kobayashi 4-36-19, Yoyogi, Shibuya-ku, Tokyo Graphics Communication Co., Ltd.・ Laboratories (56) References JP-A-6-233138 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) H04N ⁷ /24-7/68 G06T 1/00-1 / 60

Claims (1)

(57) [Claims] 1. A predicts predictive frame picture frame from the decoding to decode the encoded image data device met
A plurality of accesses to the memory are controlled by assigning a priority to each access based on the accumulated amount of the built-in buffer used for scan conversion and the accumulated amount of the built-in buffer that stores the reproduced image before writing it in the memory. Priority judgment system
Oite the image decoding equipment with control unit, the priority determination control unit, always a write access and the earliest order of the read access and the bit stream of the displayed image, the accumulation amount of the internal buffer used for scan conversion is small,
In the state quantity accumulated small internal buffer that accumulates before writing a reproduced image in the memory, the read access then in the bit stream to the earliest priority, read access of the reference image, the priority in the order of write accesses of the reproduced image The amount of storage in the internal buffer used for scan conversion is large,
When the amount of data stored in the built-in buffer for storing the reproduced image before writing it to the memory is small, the read access of the reference image and the writing of the reproduced image are performed next to the first rank.
Only accessible, prioritize the order <br/> read access bitstream, the accumulation amount of the internal buffer used for scan conversion is small,
In the state storage amount of the internal buffer that accumulates a large before writing a reproduced image in the memory, the write access of the reproduced image next to the earliest priority, read access of the bit stream, the priority order of read access reference image The amount of storage in the internal buffer used for scan conversion is large,
In a state where the storage amount of the built-in buffer for storing the reproduced image before writing it in the memory is large, the priority is set in the order of the write access of the reproduced image, the read access of the reference image, and the read access of the bit stream after the priority. An image decoding device characterized by determining.