JPH07298264A - Image data processing method, storage device used for the method and processing unit of image data - Google Patents

Abstract

PURPOSE:To provide a processing method for image data at which a data transfer rate is set high, the storage device used for it and the processing unit for image data. CONSTITUTION:In order to store image data to be coded, prediction image frame data or reproduced image frame data, an SDRAM divided into two banks A, B is used, data in one MB or data in a block are divided into plural data and stored in the same row address of the separate banks A, B of the SDRAM and the data are read and processed while accessing the banks A, B sequentially.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image data processing method, a storage device used for the method, and an image data processing device. Image data processing means encoding and decoding of image data.

[0002]

2. Description of the Related Art When transmitting or storing digitally represented image data, encoding is performed to reduce the amount of data. As an encoding method, there is a method of reducing redundancy by utilizing temporal or spatial correlation of image information (image data).

As a method of utilizing the temporal correlation, there is a method of encoding a difference between two consecutive screens (frames) or detecting a motion of an image to perform motion compensation.
As a method of utilizing spatial correlation, an image is divided into blocks of a predetermined size (for example, 8 pixels in each of the vertical direction and the horizontal direction), the data in each block is orthogonally converted, and the conversion coefficient is scan-converted. However, there is one that performs variable length coding (for example, rearranges in order from low frequency components to high frequency components). An image coding method (hereinafter abbreviated as MPEG2), which is being standardized by MPEG (Moving Picture Expert Group), uses both of the above two methods in combination.
The MPEG2 provisional recommendation is “Generic Coding of Moving P
ISO / IE entitled "ictures and Associated Audio"
It is described in C13818-2.

The present invention can be applied to the processing of encoding and decoding all MPEG2 images, but the decoding processing will be described as an example.

FIG. 14 shows an example of the configuration of an image decoding apparatus for decoding data encoded by such a method. In FIG. 14, the decoding process is executed by the buffer control unit 1, the variable length decoder 2, the scan converter 3, the inverse quantizer 4, the inverse DCT unit 5, and the motion compensation image reproducing unit 6. Reference numeral 50 denotes a memory, which includes a buffer memory 51 and a frame memory (three I, P, and B frame memories described later) 52,
It consists of 53 and 54. In addition, reference numeral 100 denotes an input bitstream representing an encoded image, and 200 denotes a reproduced image. Further, the dotted line extending from the motion compensation image reproducing unit 6 indicates writing.

Next, the operation will be described. The input bitstream 100 is stored in the buffer memory 51 as the data 40 under the control of the buffer memory control unit 1. Data 41 read from the buffer memory 51
Is variable-length decoded by the variable-length decoder 2.

Although not all the data is variable length coded, it is assumed that the fixed length code is also decoded by the variable length decoder 22. Next, the order of the data is rearranged by the scan converter 3 and then dequantized by the dequantizer 4. Next, the inverse DCT unit 5 performs inverse discrete cosine transform. The motion-compensated image reproduction unit 6 performs reproduction in consideration of the movement of the image. In MPEG2, a temporally intermediate frame (here, B frame) is predicted from both a temporally previous frame (here, I frame) and a temporally later frame (here, P frame). Therefore, to reproduce the B frame, the I frame and P that have been decoded in advance are used.
It is necessary to read the predicted frame data 42 and 43 of a frame from the frame memories 52 and 53 (in MPEG2, a P frame that is temporally later is decoded before a B frame). Predicted frame data 42, 43 and inverse DCT
The B frame is reproduced by the motion compensation image reproducing unit 6 according to the prediction error output from the unit 5, and is written in the frame memory 54 as reproduced pixel data 44. Frame memory 52,
The I, P, and B frames in 53 and 54 are read from each memory in a predetermined order (the data 45 of the B frame is read in FIG. 14), and the reproduced image 200 is output.

The present invention can be applied to an apparatus for processing all MPEG2 images, but as an example, consider the case of reproducing an NTSC image. One frame of an NTSC image consists of horizontal 720 pixels and vertical 480 lines as shown in FIG. This is divided horizontally and vertically into 16 pixels each. 1
A unit of division is called a macroblock (hereinafter abbreviated as MB). The NTSC image is divided into 45 MB horizontally, 30 MB vertically, and 1350 MB in total.

FIG. 16 shows details of the MB. The luminance signal (hereinafter abbreviated as Y) is 16 as shown in FIG.
× 16 pixels, and further four 8 × 8 pixels Y ₀ , Y
It is divided into ₁ , Y ₂ , and Y ₃ . As shown in FIG. 16 (b), there are two types of color signals, blue and red (hereinafter, _Cb and _Cr).
Abbreviated). Both C _b and C _r are 8 × 8 pixels. In addition,
Y, C _b , and C _r are all represented by 8 bits. In addition,
The odd number and the even number are used for explaining the present invention and will be described later.

As can be seen from FIGS. 15 and 16, the data amount of one frame is 4147200 bits. As shown in FIG. 14, in three frames of I, P, and B, 124
It is 41600 bits. The maximum amount of buffer 51 is 18
It is defined as 3500 8 bits. As a result, the memory 50 has a capacity of 14276608 bits or more. 16
Since the capacity of a megabit memory element is 16777216 bits, one 16 megabit memory element is sufficient.
In the case of a PAL image, it can be calculated that one 16-megabit memory element is sufficient.

All the above-mentioned image decoding is performed in MB units. That is, to reproduce 1 MB in the B frame, the predicted frame data 42 and 43 of 1 MB each is read from the I and P frames, and 1 MB in the B frame after reproduction is read.
The reproduced pixel data 44 of is written. To be precise, the prediction from the I and P frames can be performed in half-pel (half pixel) units, and the Y from the I and P frames.
As 17 × 17 pixels and C _b / C _r as 9 × 9 pixels. Further, in the case of using the field prediction, 17 × 9 pixels for Y are each twice (4 times for all I and P2 frames) and 9 × for C _b / C _r.
Five pixels must be read twice (four times for all I and P frames).

[0012]

As described above with reference to FIG. 14, processing of image data requires a very high frequency of reading / writing of the memory 50.

In the conventional apparatus, in order to increase the data transfer rate with the memory 50, a method of expanding the data width by using a plurality of memories having a small capacity has been used. For example,
A total of 64 bits of data width was obtained by using four 4 megabit memories each having 256 K words × 16 bits. Therefore, there is a big drawback that the mounting area on the substrate cannot be reduced. Further, it is not economical in the future to use four 4 megabit memories, although the capacity is only one 16 megabit memory.

Further, since the data transfer rate cannot be increased by the method of using the memory as in the above-described conventional apparatus, it is difficult to process a high resolution image such as what is called HDTV.

It is an object of the present invention to provide an image data processing method, a storage device used for the same, and an image data processing device which solve the above-mentioned drawbacks of the conventional device.

[0016]

In the image data processing method according to the present invention, a synchronous dynamic memory (hereinafter, SD) is used as a buffer and a memory for a frame.
RAM) is used. A normal memory repeats the operation of inputting an address and outputting data, while
In the SDRAM, after inputting a plurality of addresses, data is continuously output one after another, so that the operation becomes high speed. SDR
In AM, the inside is divided into multiple banks (hereinafter,
Assuming a two-bank configuration, each of which is referred to as bank A and bank B), and column address data continuous with the same row address can be accessed at high speed. Further, when accessing data having different row addresses, there is a characteristic that accessing data in different banks is faster than accessing data in the same bank. Therefore, in order to improve the access efficiency of the SDRAM, the data in the MB is divided into two,
The banks are divided into two banks A and B in the SDRAM, the respective data are allocated to the same row in the banks A and B, and the data stored in the different banks of the SDRAM are read out in each bank in a predetermined order. Read and process while accessing.

Further, in the decoding process, the bit stream representing the encoded image data is divided into a predetermined number of bits, and the divided bit stream is distributed and stored in another bank of the SDRAM.

Further, as a method of dividing the data in the MB, the data in the MB is divided into an odd number column and an even number column and allocated to two banks.

As a method of dividing the data in the MB, Y is divided into odd columns and even columns, and C _b and C _r are divided.
Is divided into C _b and C _r and assigned to two banks.

Further, as a method of dividing the data in the MB, each 8-bit pixel data is divided into upper 4 bits and lower 4 bits.
It is divided into bits and assigned to two banks.

A storage device used in a method of predicting an image frame from a plurality of prediction frame data and encoding the image data and a method of decoding the encoded image data, the storage device comprising a plurality of banks. Divided into multiple synchronous dynamic memory and data in one macroblock or data in a block formed by a plurality of macroblocks are divided into plural, and the same row in each of different banks of the synchronous dynamic memory is divided. And a control means for storing the address.

Further, the control means reads the data assigned and stored in different banks of the synchronous dynamic memory while accessing each bank in a predetermined order.

Further, there are provided a device for predicting an image frame from a plurality of predicted frame data to encode the image data and a device for processing the image data for decoding the encoded image data. ,
A storage area for storing the predicted image frame data or the reproduced image frame data is formed by at least one synchronous dynamic memory divided into a plurality of banks and data in one macroblock or a plurality of macroblocks. The data in the block is divided into a plurality of pieces and stored in the same row address of each of the synchronous dynamic memories, respectively, while being divided and stored in different banks of the synchronous dynamic memory. A control means for reading data while accessing each bank in a predetermined order is provided.

[0024]

According to the image data processing method of the present invention, when the image data to be encoded is read from the SDRAM, when the prediction data for image generation is read from the SDRAM,
In any case, such as when the reproduced data is written in the SDRAM or when the data for display is read out from the SDRAM, the A and B banks of the SDRAM can be accessed alternately. Although the SDRAM has two banks A and B, it also serves as a control terminal for data such as an address and a data terminal. By accessing the banks alternately, efficient access is possible. Further, since the image data to be encoded, the prediction data for image reproduction, and the reproduced data are accessed in units of MB, by allocating the data in the MB to the same row, it is possible to continuously change the row address. It becomes accessible.

Furthermore, by dividing the stream into the buffer areas allocated in the same SDRAM and allocating the streams to the A and B banks, the A and B banks can be alternately accessed, and the efficiency is greatly improved. To do.

Further, the pixel data in the macroblock or in the block is divided into two by an odd number column and an even number column and is divided into SDRs.
Stored in separate banks of AM.

Further, the data in the macroblock is divided into blue and red and stored in separate banks of the SDRAM.

Further, the pixel data in the macroblock or in the block is divided into two by the upper bit and the lower bit and stored in separate banks of the SDRAM.

Further, in the memory device of the present invention, the data in one macroblock or the data in the block formed by a plurality of macroblocks is divided into a plurality of pieces and stored in different banks of the SDRAM by the control means. .

Then, at the time of reading, the control means is S
Each bank of the DRAM is accessed and read out in a predetermined order.

Further, in the image data processing apparatus of the present invention, when the image data to be encoded is read from the SDRAM, the prediction data for image generation is read from the SDRAM, the reproduced data is written to the SDRAM,
When the data for display is read from the SDRAM, the A and B banks of the SDRAM can be accessed alternately at any time. Although the SDRAM has two banks A and B, it also serves as a control terminal for data such as an address and a data terminal. By accessing the banks alternately, efficient access is possible. Further, since the image data to be encoded, the prediction data for image reproduction, and the reproduced data are accessed in units of MB, by allocating the data in the MB to the same row, it is possible to continuously change the row address. It becomes accessible.

[0032]

【Example】

[Embodiment 1] An embodiment of the image data decoding apparatus of the present invention will be described below with reference to FIG.

In FIG. 1, 150 is a memory, SD
A RAM is used and, as will be described later, a buffer memory 1
51, a frame memory 152 for I frame, a frame memory 153 for P frame, and a frame memory 154 for B frame are formed.

Reference numeral 10 is a buffer control unit, which is basically shown in FIG.
4 is the same as that of the buffer control unit 1 but also performs control peculiar to the present invention as described later. Reference numeral 60 denotes a motion-compensated image reproducing unit, which is basically the same as the motion-compensated image reproducing unit 6 in FIG. 14, but also controls the present invention as described later. And
The buffer control unit 10 and the motion-compensated image reproduction unit 60 are both control means, and constitute a storage device together with the memory 150. The same reference numerals as in FIG. 14 indicate the same parts.

Since the present invention is characterized by the structure of the memory 150 and its control, the memory 150 will be described below.

FIG. 2 shows a memory (see FIG. 14) in this embodiment.
It is a figure which shows the address allocation of the frame memory 50) 150 of this. A 16-megabit SDRAM having A, B2 banks of 2048 rows and 256 columns is assumed. A,
Rows 1 to 507 for each of the B banks are I frames, rows 508 to 1
Line 014 is the P frame, and lines 1015 to 1521 are the B frame areas. Remaining 1522 lines to 204
Eight lines are the buffer area.

Each of the I, P and B frames is divided into a 338 row Y area and a 169 row C _b / C _r area.

In the present embodiment, in the MB as shown in FIG. 16, 128 pixels of odd Y columns are assigned to bank A and 128 pixels of even columns of Y are assigned to bank B.

Then, all of these data are applied to the same row as shown in FIG. Since 16-bit data is stored in one address of the memory 150, one MB is A,
Each of the B banks has 64 columns.

Similarly, for C _b / C _r , pixels in odd columns are assigned to bank A and pixels in even columns are assigned to bank B as shown in FIG. This is shown in FIG.

The 1350 MBs shown in FIG. 15 are distributed as shown in FIG.

The image data access method is as follows.

First, the prediction frame data 4 in FIG.
The reading of Nos. 2 and 43 will be described. The data predicted by the motion vector spans 4 MBs as shown in FIG. In the figure, field prediction is assumed, and the data is in the range of 17 continuous pixels in the horizontal direction and 9 pixels in every other vertical pixel. In the memory, these data are stored in four MB areas for both A and B as shown in FIG.

Since the data in MB1 of the A bank is the same row, it can be read continuously. The same applies to data in other MBs. In FIG. 5, the first is read continuously, without interruption,
,,,,, ... are read out.

FIG. 6 shows another read operation. ,,, ... Read out.

FIG. 7 shows still another read operation. As shown in Fig. 2, the data in the same row is another MB.
It is possible to read continuously even in the area. In the figure, it looks like ,,, ....

7 to 9 show an example of reading the data in the MB in the horizontal direction, it is also possible to read in the vertical direction.

Next, writing of the reproduction pixel data 44 and reading of the display data 45 will be described with reference to FIG. Since the reproduced pixel data 44 is all the data in one MB,
It is possible to continuously write while dividing the odd and even columns into A and B banks. The data of the first row of the MB can be written in units of 8 pixels as in, and similarly, the data of the second and subsequent rows can be written.
The display data 45 can also be read by reading the data of the first row of the MB by 8 pixels at a time, and then the data of the first row of the next MB to the right can be similarly read.

As described above, all image data can be read and written while alternately accessing the A and B banks.
Even with a 6-bit width, a sufficient transfer rate can be obtained. [Second Embodiment] A second embodiment of the present invention will be described below with reference to FIG.

In this embodiment, the Y data is the same as that in the first embodiment, but the method of allocating the C _b / C _r data is different. All the data of C _b is assigned to the A bank, and all the data of C _r is assigned to the B bank. The access is performed in the same manner as in the first embodiment, and the two banks are alternately performed, so that the transfer rate can be increased. [Third Embodiment] A third embodiment of the present invention will be described below with reference to FIG.

In the present embodiment, 8-bit pixel data in MB is divided into upper 4 bits and lower 4 bits and applied to banks A and B. Also in this embodiment, the transfer rate can be increased.

Although FIG. 10 shows Y data, the same applies to C _b / C _r data. [Fourth Embodiment] A fourth embodiment of the present invention will be described below with reference to FIGS.

In this embodiment, access to the data in the buffer area is speeded up. The input bit stream 100 is divided into predetermined bit amounts and alternately assigned to banks A and B. FIG. 11 shows an example in which 16 bits are divided, and FIG. 12 shows an example in which 32 bits are divided. This division enables the A and B banks to be accessed alternately during writing and reading, thereby increasing the speed.

Although the above-mentioned embodiment is applied to the case of the image processing in the image data decoding apparatus B as shown in FIG. 13, the case of the encoding processing in the image data encoding apparatus A is also applied. Exactly the same is applicable. The reproduced image frame data in the decoding process corresponds to the image frame data to be encoded in the encoding process. The image data processing method according to the present invention may be performed by either the image data encoding device A or the image data decoding device B.

Further, it is needless to say that the SDRAM is not limited to one, and can be applied to a case where a plurality of SDRAMs, such as an HDTV, need to be used in a stack.

[0056]

As described above in detail, the image data processing method according to the present invention uses SDRAMs for buffer storage and frame storage, and further divides data in 1 MB into a plurality of SDRAMs. Since data can be distributed to another bank and image coding, reproduction, and display can be performed while alternately accessing another bank, data can be transferred at high speed to and from the memory.

Furthermore, since the bit stream is also divided and distributed to different banks and the different banks can be accessed alternately, high speed data transfer is possible.

The data is distributed to the SDRAM by dividing the pixel data into two in odd and even columns, dividing the color data into blue and red, or dividing the pixel data into upper bits and lower bits. Since it was divided into two with SDRA
It is possible to easily divide and store in M.

Further, the storage device according to the present invention is SD
It is composed of a RAM and a control means for controlling writing to the RAM, and divides data in one macroblock or data in a block into a plurality of parts and stores them in the same row address in different banks of the SDRAM. It can be suitably used in the image data processing method of the present invention.

Further, since reading is also performed while the control means accesses each bank in a predetermined order, high-speed reading is possible.

Further, in the image data processing device according to the present invention, at least one storage area for storing the image frame data to be encoded, the predicted image frame data or the reproduced image frame data is divided into a plurality of banks. Of the synchronous dynamic memory and the data in one macroblock or the data in the block formed by a plurality of macroblocks are divided into a plurality of pieces, and the same row address is provided in each of the separate synchronous dynamic memories. On the other hand, it is provided with a control means for storing and reading the data divided and assigned to the separate banks of the synchronous dynamic memory and stored while accessing each bank in a predetermined order.
Data can be divided into a plurality of pieces and distributed to different banks of the SDRAM, and image encoding, reproduction and display can be performed while alternately accessing the different banks, so that data can be transferred at high speed to the memory. There is.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an image data decoding device of the present invention.

FIG. 2 is a data allocation diagram in the memory showing the first embodiment of the image data processing method of the present invention.

FIG. 3 is an allocation diagram of Y data in one MB in the first embodiment.

FIG. 4 C _b / C in one MB in the first embodiment
It is an allocation diagram of _r data.

FIG. 5 is a diagram showing a first method of reading predicted image data according to the first embodiment.

FIG. 6 is a diagram illustrating a second method of reading predicted image data according to the first embodiment.

FIG. 7 is a diagram showing a third method of reading predicted image data according to the first embodiment.

FIG. 8 is a diagram showing a method of writing reproduced image data and a method of reading display data in the first embodiment.

FIG. 9 shows C _b / C in one MB in the second embodiment.
It is an allocation diagram of _r data.

FIG. 10 is an allocation diagram of Y data in the third embodiment.

FIG. 11 is a first allocation diagram of a bitstream according to the fourth embodiment.

FIG. 12 is a second allocation diagram of a bitstream according to the fourth embodiment.

FIG. 13 is a diagram for explaining an application part of the present invention.

FIG. 14 is a block diagram showing a configuration example of a conventional image decoding device and an image decoding device according to the present invention.

FIG. 15 is a diagram showing MB division in an NTSC image.

FIG. 16 is a diagram showing an array of pixels in one MB.

[Explanation of symbols]

 1 Buffer Control Unit 2 Variable Length Decoder 3 Scan Conversion Unit 4 Inverse Quantization Unit 5 Inverse DCT Unit 6 Motion Compensated Image Reproduction Unit 10 Buffer Control Unit 40 Bit Stream Write Data 41 Bit Stream Read Data 42 Prediction Frame Data 43 Prediction Frame Data 44 playback pixel data 45 display data 50 memory 51 buffer memory 52 frame memory (I) 53 frame memory (P) 54 frame memory (B) 60 motion compensation image playback unit 100 input bit stream 150 memory 151 buffer memory 152 frame memory (I ) 153 frame memory (P) 154 frame memory (B) 200 playback image

Claims (8)

[Claims] 1. A method of predicting an image frame from a plurality of prediction frame data to encode image data and a method of processing image data for decoding the encoded image data, the image frame data to be encoded or the prediction. Using at least one synchronous dynamic memory divided into a plurality of banks to store image frame data or reproduced image frame data, a plurality of data in one macroblock or data in blocks forming a macroblock are used. Data in each of the separate banks of the synchronous dynamic memory and stored in the same row address in each of the separate banks of the synchronous dynamic memory. In order of reading and processing An image data processing method characterized by the above. 2. The image according to claim 1, wherein a bit stream representing encoded image data is divided into a predetermined number of bits, and the divided bit stream is distributed and stored in another bank of the synchronous dynamic memory. How to process the data. 3. Pixel data in a macroblock or in a block is divided into two, an odd column and an even column, and the divided pixel data is distributed and stored in another bank of a synchronous dynamic memory. Image data processing method described. 4. The image according to claim 1 or 2, wherein the color data in the macroblock is divided into blue and red, and the divided color data is stored in another bank of the synchronous dynamic memory. How to process the data. 5. The pixel data in a macroblock or in a block is divided into two by an upper bit and a lower bit, and the divided and stored in another bank of a synchronous dynamic memory. Image data processing method described. 6. A storage device used in a method of predicting an image frame from a plurality of prediction frame data to encode image data and a method of processing image data for decoding encoded image data, the storage device comprising a plurality of banks. Divided into multiple synchronous dynamic memory and data in one macroblock or data in a block formed by a plurality of macroblocks are divided into plural, and the same row in each of different banks of the synchronous dynamic memory is divided. A storage device comprising: a control unit for storing the address. 7. The storage device according to claim 6, wherein the control means reads out data stored in different banks of the synchronous dynamic memory and stored therein, while accessing each bank in a predetermined order. 8. A method of predicting an image frame from a plurality of predicted frame data to encode the image data, and an image data processing device for decoding the encoded image data, the image frame data to be encoded or , At least one synchronous dynamic memory in which a storage area for storing predicted image frame data or reproduced image frame data is divided into a plurality of banks, and
Divide the data in one macroblock or the data in the block formed by multiple macroblocks into multiple,
Data is stored in the same row address of each of the synchronous dynamic memories, and on the other hand, the divided data assigned to and stored in different banks of the synchronous dynamic memory are accessed in a predetermined order in each bank. An image data processing device, characterized in that the image data processing device further comprises: