JPH08186826A - Image decoding processing method and storage device used for it and image decoder - Google Patents

Abstract

PURPOSE: To enhance a data transfer rate in the processing method of image decoding of image data, the storage device used for it and the image data processing decoder. CONSTITUTION: In order to store image data to be coded, predicted image frame data or reproduced image frame data, an SDRAM divided into two banks A, B is used. Data in one MB or data in a block are divided into plural data, and picture element data of an odd number row and picture element data of an even number row in a same row address of the banks A, B in the SDRAM and in macro blocks adjacent to each other in the horizontal and vertical directions are stored as different banks without fail. Then the data are read and processed while accessing the banks A, B sequentially.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image decoding processing method for decoding an encoded image signal, a storage device used for the method, and an image decoding device.

[0002]

2. Description of the Related Art When transmitting or storing digitally represented image data, encoding is performed to reduce the amount of data. As an encoding method, there is a method of reducing redundancy by utilizing temporal or spatial correlation of image information (image data).

As a method of utilizing the temporal correlation, there is a method of encoding a difference between two consecutive screens (frames) or detecting a motion of an image to perform motion compensation.
As a method of utilizing spatial correlation, an image is divided into blocks of a predetermined size (for example, 8 pixels in each of the vertical and horizontal directions), the data in the blocks are orthogonally converted, and the conversion coefficient is scan-converted. However, there is one that performs variable length coding (for example, rearranges in order from low frequency components to high frequency components). An image coding method (hereinafter abbreviated as MPEG2), which is being standardized by the Moving Picture Experts Group (MPEG), uses the above two methods together.
The MPEG2 provisional recommendation is “Generic Coding of Moving P
ISO / IE entitled "ictures and Associated Audio"
It is described in C13818-2.

The present invention is applicable to the process of decoding all MPEG2 images, so the decoding process will be described.

FIG. 11 shows an example of the structure of an image decoding apparatus for decoding data coded by such a method. In FIG. 11, the decoding process is executed by the buffer control unit 1, the variable length decoder 2, the scan converter 3, the inverse quantizer 4, the inverse DCT unit 5, and the motion compensation image reproducing unit 6. Reference numeral 50 denotes a memory, which includes a buffer memory 51 and a frame memory (three I, P, and B frame memories described later) 52,
It consists of 53 and 54. In addition, reference numeral 100 denotes an input bitstream representing an encoded image, and 200 denotes a reproduced image. Further, the dotted line extending from the motion compensation image reproducing unit 6 indicates writing.

Next, the operation will be described. The input bitstream 100 is stored in the buffer memory 51 as the data 40 under the control of the buffer memory control unit 1. Data 41 read from the buffer memory 51
Is variable-length decoded by the variable-length decoder 2.

Although not all data is variable length coded, it is assumed that fixed length code is also decoded by this variable length decoder 2. Next, the order of the data is rearranged by the scan converter 3 and then dequantized by the dequantizer 4. Next, the inverse DCT unit 5 performs inverse discrete cosine transform. The motion-compensated image reproduction unit 6 performs reproduction in consideration of the movement of the image. In MPEG2, a temporally intermediate frame (here, B frame) is predicted from both a temporally previous frame (here, I frame) and a temporally later frame (here, P frame). Therefore, to reproduce the B frame, the I frame and P that have been decoded in advance are used.
It is necessary to read the predicted frame data 42 and 43 of the frame from the frame memories 52 and 53 (MPEG2
Then, the temporally later P frame is decoded before the B frame). Predicted frame data 42, 43 and inverse DC
The B frame is reproduced by the motion compensation image reproducing unit 6 by the prediction error which is the output of the T unit 5, and as the reproduced pixel data 44,
It is written in the frame memory 54. Frame memory 5
The I, P, and B frames in 2, 53, 54 are read out from each memory in a predetermined order (the B frame data 45 is read in FIG. 10), and the reproduced image 200 is output.

The present invention can be applied to an apparatus for processing all MPEG2 images as described above. As an example, consider the case of reproducing an NTSC image. NTS
One frame of the C image is composed of 720 pixels in the horizontal direction and 480 lines in the vertical direction as shown in FIG. This is divided into 16 pixels horizontally and vertically. A unit of one division is called a macroblock (hereinafter abbreviated as MB). NTSC image is 45MB in width and 3 in height
It is divided into 0 MB and 1350 MB in total. Also, MP
In EG2, a group of macroblocks closed in one horizontal row is called a slice, and an NTSC image is divided into at least 30 slices.

FIG. 13 shows details of the MB. The luminance signal (hereinafter abbreviated as Y) is 16 × as shown in FIG.
16 pixels and four 8 × 8 pixels Y ₀ , Y ₁ ,
It is divided into Y ₂ and Y ₃ . As shown in FIG. 13B, there are two types of color signals, which are blue and red (hereinafter abbreviated as Cb and Cr), and both Cb and Cr are 8 × 8 pixels. Therefore, one MB constitutes six blocks of 8 × 8 pixels. Note that Y, Cb, and Cr are all represented by 8 bits. Also, odd numbers and even numbers are used for explaining the present invention,
It will be described later.

As can be seen from FIGS. 12 and 13, the data amount of one frame is 4147200 bits. As shown in FIG. 11, in three frames of I, P and B, 124
It is 41600 bits. The maximum amount of the buffer memory 51 is set to 1835008 bits. As a result, the memory 50 has a capacity of 14276608 bits or more. 16 megabit memory device capacity is 167772
Since it is 16 bits, one 16 megabit memory element is sufficient. In the case of a PAL image, it can be calculated that one 16-megabit memory element is sufficient.

All the above-mentioned image decoding is performed in MB units. That is, to reproduce 1 MB in the B frame, the predicted frame data 42 and 43 of 1 MB are read from the I and P frames, and 1 M in the B frame after reproduction is read.
The reproduced pixel data 44 of B will be written. To be precise, the prediction from the I and P frames is possible in units of half repel (half pixel), and from the I and P frames, 17 × 17 pixels for Y and 9 × 9 pixels for Cb / Cr are provided. Must be read. Further, when field prediction is used, 17 × 9 pixels for Y are each twice (4 times for the entire I and P2 frames) and 9 for Cb / Cr.
The × 5 pixels must be read twice (four times for the entire I and P2 frames).

[0012]

As described above with reference to FIG. 11, the processing of image data requires a very high frequency of reading / writing of the memory 50.

In the conventional apparatus, in order to increase the data transfer rate with the memory 50, a method of expanding the data width by using a plurality of memories having a small capacity has been used. For example, four 4-megabit memories each having a 256K word × 16-bit structure are used to provide a data width of 64 bits in total. Therefore, there is a big drawback that the mounting area on the substrate cannot be reduced. Further, it is not economical to use four 4-megabit memories in the future, even though one 16-megabit memory is required in terms of capacity.

Further, since the data transfer rate cannot be increased by the method of using the memory as in the above-described conventional apparatus, it is difficult to process a high resolution image such as what is called HDTV.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an image decoding processing method, a storage device and an image decoding device used for the same which solve the above-mentioned drawbacks of the conventional device.

[0016]

In the image decoding processing method, the storage device and the image decoding device used therefor according to the present invention, a synchronous dynamic memory (hereinafter abbreviated as SDRAM) is used as a buffer and a memory for a frame. In a normal memory, an operation of inputting an address and outputting data is repeated, whereas in an SDRAM, after inputting a plurality of addresses, data is continuously output one after another, so that the operation speed is high. In the SDRAM, the internal operation is divided into a plurality of banks (hereinafter referred to as a bank A and a bank B assuming a 2-bank configuration),
Continuous column address data with the same row address can be accessed at high speed. Further, when accessing data having different row addresses, there is a characteristic that accessing data in different banks is faster than accessing data in the same bank. Therefore, in order to improve the access efficiency of the SDRAM, the data in the MB is divided into two and SDRA
The data is allocated to two banks A and B in M, the respective data are allocated to the same row in banks A and B, and the data stored in the different banks of the SDRAM are read out in each bank in a predetermined order. Read and process while accessing.

Further, as a method of dividing the data in the MB, each pixel data is divided into an odd row and an even row, and the banks of the odd row and the even row of adjacent MBs on the left, right, top and bottom are always assigned to different banks.

Further, the control means reads the data allocated and stored in different banks of the synchronous dynamic memory while accessing each bank in a predetermined order.

[0019]

According to the image decoding processing method of the present invention, and the storage device and the image decoding device used therefor, when the predicted data for image generation is read from the SDRAM, when the reproduced data is written in the SDRAM, and when it is displayed, SDRAM at any time when reading data from SDRAM
The A and B banks can be accessed alternately. SDRA
M has two banks A and B, but also serves as a control terminal for data such as an address and a data terminal. By alternately accessing the banks, efficient access is possible. Further, since the predicted data for image reproduction and the reproduced data are accessed in MB units, by allocating the data in the MB to the same row, continuous access is possible without changing the row address.

The data predicted by the motion vector for image generation covers a maximum of 4 MB in both frame prediction and field prediction.
As a method of dividing the data in the MB, each pixel data is divided into an odd row and an even row and allocated to two banks, and the odd row and the even row of adjacent MBs on the top, bottom, left and right are always allocated to different banks. In the case of field prediction that requires the most predictive data, at worst, the A and B banks are alternately switched, and one predictive field data for image generation is changed to S by changing the row address four times.
It can be read from the DRAM, and the efficiency is greatly improved. If the vector accuracy in the horizontal or vertical direction is an integer pixel accuracy and the predicted data spans exactly two MBs, the A and B banks are switched alternately regardless of frame prediction and field prediction. S
It can be read from the DRAM and the efficiency is greatly improved. Even when the vector accuracy in both the horizontal and vertical directions is integer pixel accuracy and the predicted data is just 1 MB, in the case of frame prediction, the A and B banks are alternately switched and the prediction data is transferred from the SDRAM. It can be read and the efficiency is greatly improved. In that case, needless to say, the efficiency of reading the reference data in the frame prediction and the efficiency of reading the display data from the SDRAM when writing the reproduced data to the SDRAM is not sacrificed. .

[0021]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 shows an embodiment of an image decoding apparatus of the present invention. In FIG. 1, 150 is a memory SD
A RAM is used and, as will be described later, a buffer memory 1
51, a frame memory 152 for I frame, a frame memory 153 for P frame, and a frame memory 154 for B frame are formed.

Reference numeral 10 is a buffer control unit, which is basically shown in FIG.
Although it is the same as the buffer control unit 1 of No. 1, it also performs control peculiar to the present invention as described later. Reference numeral 60 denotes a motion-compensated image reproducing unit, which is basically the same as the motion-compensated reproducing unit 6 in FIG. 11, but also performs control peculiar to the present invention as described later. And
The buffer control unit 10 and the motion-compensated image reproduction unit 60 are both control means, and constitute a storage device together with the memory 150. The same reference numerals as in FIG. 11 indicate the same parts.

Since the present invention is characterized by the structure of the memory 150 and its control, the memory 150 will be described below.

FIG. 2 shows a memory (FIG. 11) in this embodiment.
(Corresponding to the memory 50 of the above) 150 is a diagram showing the address allocation. A 16-megabit SDRAM having A and B banks of 2048 rows and 256 columns is assumed. 1st to 507th rows are I frames, and 508th to 1st rows in both A and B banks
Line 014 is the P frame, and lines 1015 to 1521 are the B frame areas. Remaining 1522 lines to 204
Eight lines are the buffer area.

The I, P and B frames are divided into a 338 line Y area and a 169 line Cb / Cr area.

In this embodiment, as shown in FIG. 9, odd numbered MBs
128 banks of odd-numbered Y pixels in Y
The pixels in the even rows of are assigned to bank B. Further, as shown in FIG. 10, in an even-numbered MB, 128 pixels of odd-numbered rows of Y are in bank B, and 128 pixels of even-numbered rows of Y are in bank A.
Hit

Then, the data of these respective MBs are applied to the same row as shown in FIG. 1 of memory 150
Since 16-bit data is stored in the address, one MB has 64 columns in each of A and B banks.

Similarly, for Cb / Cr, as shown in FIG. 9, in odd-numbered MBs, odd-row pixels are assigned to bank A and even-row pixels are assigned to bank B. Further, as shown in FIG. 10, in the even-numbered MBs, the pixels in the odd-numbered rows are in the bank B,
Pixels in even rows are assigned to bank A. This is shown in FIG.

The 1350 MBs shown in FIG. 12 are distributed as shown in FIG.

The image data access method is as follows.

First, the prediction frame data 4 in FIG.
The reading of Nos. 2, 43 will be described. The data predicted by the motion vector spans 4 MBs as shown in FIG. In the figure, field prediction is assumed, and the data is in the range of 17 continuous pixels in the horizontal direction and 9 pixels in every other vertical pixel. These data are stored in two areas in both A and B in the memory as shown in FIG.

Since the data in MB1 of the A bank are in the same row, they can be read continuously. The same applies to data in other MBs. In FIG. 5, the first is read continuously, without interruption,
,,,,, ... are read out.

FIG. 6 shows another read operation.
Read as ,,,,,,, ....

Next, writing of the reproduced image data 44 and reading of the display data 45 will be described with reference to FIGS. Since the reproduced pixel data 44 is all the data in one MB, it is possible to continuously write while dividing the odd and even rows into A and B banks. The data in the first row of the MB is written in the A bank as shown in FIG. 9, the data in the second row is written in the B bank as, and this is repeated in the MB. In the next MB, the data of the first row is written in the B bank as shown in FIG. 10, the row data of the second row is written in the A bank as in, and this is repeated in this MB. Subsequent MBs can be similarly written by repeating the above operation. In reading the display data, the data of the first row in the MB is read by 16 pixels from the bank A, then the data of the first row of the MB on the right is read by 16 pixels from the bank B, and then the M on the right is read.
B can be continuously read from bank A by alternately switching between banks A and B.

FIG. 7 shows that the data predicted by the motion vector for image reproduction in this embodiment has a pixel precision in the horizontal direction which is an integer pixel. Further, as shown in FIG.
This is an example over two MBs. In the figure, field prediction is assumed, and the data is in the range of 17 continuous pixels in the horizontal direction and 9 pixels in every other vertical pixel. These data are stored in one area of both A and B in the memory as shown in the figure, because the banks in which the odd-row pixels or even-row pixels of the upper and lower MBs are stored are different.

As for the reading order, in FIG. 7, it is possible to read the first one continuously and to read ,,, ..., Without interruption.

FIG. 8 shows that the data predicted by the motion vector for image reproduction in the present embodiment is frame prediction, and the pixel precision is an integer pixel in both the horizontal and vertical directions. Further, as shown in FIG. This is an example of data of one MB. In the figure, in the frame prediction, the pixel precision is an integer pixel in both the horizontal and vertical directions, so that the data is in the range of continuous 16 pixels both vertically and horizontally. These data are stored in one area for both A and B in the memory as shown in the figure.

As for the reading order, in FIG. 8, it is possible to read the first one continuously and to read ,,, ..., Without interruption.

As described above, according to the present invention, the data predicted by the motion vector for image reproduction is used as the SDR.
When reading from AM or writing a reproduced image to SDRAM, all image data can be read while alternately accessing banks A and B without continuously accessing different row addresses in the same bank. Can be efficiently accessed.

Also in the present invention, Japanese Patent Application No. 6-91
As described in No. 917, in order to speed up access to the data in the buffer area, the input bit stream 100 is divided into a certain bit amount, and written in the A and B banks alternately like the image data. By reading the reference image, reading the reproduced image, reading the display image, writing the input bit stream, and reading the input bit stream, different row addresses are continuously accessed in the same bank. Without reading, all image data can be read while alternately accessing the A and B banks,
Since it is possible to access the memory efficiently,
Even with a 16-bit width, a sufficient transfer rate can be obtained.

Further, it is needless to say that the SDRAM is not limited to one, and can be applied to a case where a plurality of SDRAMs, such as an HDTV, need to be used in a stack.

[0042]

As described above, the image decoding method according to the present invention, the storage device used therefor, and the image decoding device are divided into a plurality of banks for storing the predicted image frame data or the reproduced image frame data. At least one synchronous dynamic memory is used to divide pixel data in one macroblock into pixel data of odd rows and pixel data of even rows, and the same row of each bank of each of the synchronous dynamic memories is divided. Pixel data of odd rows and even rows of pixel data of adjacent macroblocks in the address and above, below, left, and right are always stored as different banks, and are allocated and stored in different banks of the divided synchronous dynamic memory. Read the stored data while accessing each bank in a predetermined order. Since so as to the processing, image reproduction while alternately access different banks, it is displayed, it is faster to transfer data to and from the memory.

In addition, since the pixel data in each MB is divided into odd-numbered rows and even-numbered rows, it is possible to easily divide and store the data in the SDRAM.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an image data decoding device of the present invention.

FIG. 2 is a data allocation diagram in the memory showing the first embodiment of the image data processing method of the present invention.

FIG. 3 is an allocation diagram of Y data in one MB in the first embodiment.

FIG. 4 Cb / C in one MB in the first embodiment
It is an allocation diagram of r data.

FIG. 5 is a diagram showing a first method of reading predicted image data according to the first embodiment.

FIG. 6 is a diagram illustrating a second method of reading predicted image data according to the first embodiment.

FIG. 7 is a diagram showing a third method of reading predicted image data according to the first embodiment.

FIG. 8 is a diagram showing a fourth method of reading predicted image data according to the first embodiment.

FIG. 9 is a diagram showing a method of writing reproduced image data and reading display data in the first embodiment.

FIG. 10 is a diagram showing a method of writing reproduced image data and a method of reading display data in the first embodiment.

FIG. 11 is a block diagram illustrating a configuration example of a conventional image decoding device.

FIG. 12 is a diagram showing MB division in an NTSC image.

FIG. 13 is a diagram showing an array of pixels in one MB.

[Explanation of symbols]

 1 Buffer Control Unit 2 Variable Length Decoder 3 Scan Converter 4 Inverse Quantizer 5 Inverse DCT Unit 6 Motion Compensated Image Reproduction Unit 10 Buffer Control Unit 40 Bit Stream Write Data 41 Bit Stream Read Data 42 Prediction Frame Data 43 Prediction Frame Data 44 playback pixel data 45 display data 50 memory 51 buffer memory 52 frame memory (I) 53 frame memory (P) 54 frame memory (B) 60 motion compensation image playback unit 100 input bit stream 150 memory 151 buffer memory 152 frame memory (I ) 153 frame memory (P) 154 frame memory (B) 200 playback image

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Ryuji Nishito 4-36-19 Yoyogi, Shibuya-ku, Tokyo Inside Graphics Communications Laboratories, Inc. (72) Inventor Tomoyuki Shindo 4-36 Yoyogi, Shibuya-ku, Tokyo No. 19 in the Graphics Communications Laboratories, Inc. (72) Inventor Yutaka Okada 4-36-19 Yoyogi, Shibuya-ku, Tokyo In the Graphics Communications Laboratories, Inc. (72) Inventor, Kaoru Kawamura Tokyo 4-36-19 Yoyogi, Shibuya-ku, Tokyo Within Graphics Communications Laboratories, Inc. (72) Inventor Shigeru Komatsu 4-36-19 Yoyogi, Shibuya-ku, Tokyo Graphics Communications Inc. In Laboratories

Claims (4)

[Claims] 1. A method of processing image data for predicting an image frame from a plurality of predicted frames and decoding encoded image data, wherein a plurality of banks are provided for storing predicted image frame data or reproduced image frame data. At least one divided dynamic memory is used to divide pixel data in one macroblock into pixel data of odd rows and pixel data of even rows, and each of the separate banks of the synchronous dynamic memory is divided. Within the same row address, the pixel data of the odd rows and the pixel data of the even rows of adjacent macroblocks on the left, right, top and bottom are stored as different banks without fail, and are stored in different banks of the divided synchronous dynamic memory. The allocated and stored data can be accessed in a predetermined order in each bank. An image decoding processing method characterized by reading and processing while reading. 2. A storage device used in an image data processing method for predicting an image frame from a plurality of prediction frames and decoding encoded image data, comprising: a synchronous dynamic memory divided into a plurality of banks; Pixel data in one macroblock is divided into pixel data of odd-numbered rows and pixel data of even-numbered rows, and control means is stored in the same row address of different banks of the synchronous dynamic memory. A storage device characterized by the above. 3. The storage device according to claim 2, wherein the control means reads the data assigned to and stored in different banks of the synchronous dynamic memory while accessing each bank in a predetermined order. 4. A decoding device of image data for predicting an image frame from a plurality of prediction frames and decoding encoded image data, the storage area for storing predicted image frame data or reproduced image frame data. Includes at least one synchronous dynamic memory divided into a plurality of banks, and further divides the pixel data in one macroblock into pixel data of odd rows and pixel data of even rows, and the synchronous dynamic memory Within the same row address of each different bank of
Moreover, the pixel data of the odd rows and the pixel data of the even rows of the adjacent macroblocks on the left, right, top and bottom are always stored as different banks, while they are assigned and stored in separate banks of the divided synchronous dynamic memory. An image decoding apparatus comprising a control means for reading data while accessing each bank in a predetermined order.