DE29521638U1 - Image decoding device - Google Patents

Description

Documents that are required for the registration of the utility
are to be based on

22 October 1997

Hitachi, Ltd. N 22078Gmb/Tl AL/Sn/sb

B !!(!decoding device

The present invention relates to an apparatus for decoding image data according to the preamble of claim 1; further to an image decoding apparatus for decoding and/or decompressing coded and/or compressed image signals, and in particular to an image decoding apparatus effective to reduce the memory capacity, the memory data bus width, the decoding delay time and the decoding clock frequency.

An international standard for image compression, referred to as MPEG 2, is currently being decided upon for the purpose of application in digital broadcasting and recording media (see, for example, "Journal of the Institute of Television Engineers of Japan", Volume 48, No.

1, pp. 44 to 49). In the MPEG 2 coding system, image signals are encoded by appropriately combining an intra-picture coding frame (hereinafter referred to as I-frame), an inter-picture coding frame (hereinafter referred to as P-frame) and a picture interpolation coding frame (hereinafter referred to as B-frame) to enable the coexistence of the high data compression rate and the functions of random access and editing.

In the I-frame, only image data is combined by transform coding. It is a system based on the existence of a correlation among the image data in the frame, which divides the frame into blocks of predetermined size; transforms each block; the coefficients

cient data after conversion is equivalently quantized to the frequency component; and the coded data is generated by variable length coding.

In the P frame, the data compression rate is increased by using a high correlation between the frames. The previous frame and the current frame are compared in the predetermined number of blocks, and a motion vector is obtained. And the image data of the previous frame is each extracted from the data obtained according to the

Î&ogr; motion vector shifted position is read, and a predicted value is obtained. After that, the predicted value is subtracted from the image data of the current frame to be coded, and this motion compensated prediction error is transform compensated in the same way as in the intra-frame coding system; and coded data is generated.

To further increase the data compression rate, the P-frame is used. The frame interpolation coding is also called bidirectional motion compensation inter-frame coding and uses the correlation not only in the previous frame but also in the following frame. The system compares the previous frame in display order with the current frame in the predetermined number of blocks; it obtains the motion vector; it obtains the motion vector at the same time for the following frame in the display order of the predetermined number of blocks; it reads the frame data of the previous frame and the following frame from the position shifted according to the respective motion vectors; it generates an average value; and it obtains a frame interpolation value. Then, the system subtracts the frame interpolation value from the frame data of the current frame to be encoded and obtains a motion compensated prediction error.

For example, the system performs intra-picture coding for the first frame (I-frame); it then performs inter-picture coding for the fourth frame using the first frame as a reference frame (P-frame); and then performs picture interpolation coding for the second and third frames using the first and fourth frames as a reference frame (B-frame). In this case, the display sequence of the frames is: the first -> the second -> the third -> the fourth. However, if the coding sequence is changed as follows: the first -> the fourth -> the second -> the third, the B-frame is not a reference frame for subsequent coding.

A television picture signal is an interlaced scanning signal, in which an image is structured from two fields in which the number of lines is half as large and the line position is shifted alternately.

Furthermore, there is a time delay between the fields constituting one picture or frame. On the other hand, the picture data of the two fields are combined for coding and then divided into blocks of a predetermined size and encoded as picture data of one frame. In a picture decoding device, the pictures are decoded in the coding order so that the decoding results are obtained in the sequential scanning order in block units running from the upper left to the lower right in the picture. In the picture decoding device, however, it is necessary to output an interlaced video signal.

For this reason, in an image decoding device, it is necessary to convert between sequential scanning in block units and interlace scanning in pixel units. When the P frame is included, it is necessary to rearrange the images so as to rearrange them into the normal display order.

A conventional image decoding device is described, for example, in document C-659 (Proceedings 5, p. 227) of the IEICE (Institute of Electronics, Information and Communication Engineers) Spring Conference, or in document ISSCC 94 (international Solid State Circuit Conference) 1994/Session 4/Video and Communication Single Processors/Paper WP 4.4. The conventional image decoding device provides: a buffer step for writing coded data which must be input into the coded data buffer at the input clock; a decoding step for reading and decoding the coded data from the coded data memory at the predetermined clock of the decoding clock; and writing the decoded image data into an image memory having a capacity of several simultaneous images; and it provides a display step for reading out the decoded image data from the image memory by performing scan conversion and image rearrangement and displaying

is (display) and outputting it as a digital video signal according to the predetermined display clock. Further, in case the decoding data is P-frame data or B-frame data, the decoding step reads out the reference picture data in the reference picture from the frame memory so that motion compensation is performed.

The input clock is the transmission clock for digital broadcasting. The display clock refers to the sampling frequency of the digital video signal and is set to the standard value of 13.5 MHz or 27 MHz. The decoding clock is set to a frequency at which decoding of the decoded data of each frame can always be completed within a one-frame period, even taking into account changes in the amount of processing required to decode coded data of each frame.

The processing amount required for decoding coded data of each picture generally varies according to the coding system with which the picture is coded, i.e. whether it is coded as an ε-frame, a P-frame or a B-frame. The processing amount also varies with the amount of coded data of the picture. The decoding clock may be set independently of the input clock or the display clock, and it may be set to a frequency in predetermined proportion to the display clock. In any case, the decoding step and the display step are performed independently and asynchronously. Further, a buffer step γ for the coded data is also required, and this operation is performed asynchronously to the memory access operation of the decoding step and the display step. Therefore, an arbitration function for deciding the memory access right is essential. It is generally necessary to stop the decoding step during the decision period; and it is further necessary to set the decoding clock slightly higher in advance in order to process the coded data of one picture during the one-picture period excluding the arbitration period. A conventional picture decoding apparatus is an apparatus corresponding to a common television set of the 525/60 system used in the United States and Japan, wherein the picture data of the picture consists of the data of a luminance signal and two kinds of color signals. The luminance signal consists of 720 horizontal pixels and 480 vertical lines, and the two kinds of color signals consist of 360 pixels and 240 lines, in which the resolution is half that of the luminance signal in the horizontal and vertical directions. Further, in a conventional picture decoding apparatus, four dynamic RAMs (DRAMs) with the configuration of 246k × 16 bits (4M bits) are used; and for the total capacity of 16M bits, 2 blocks can be used for storing the image data of the reference image required for decoding, and 1.5 blocks for interline conversion in the display step as the image storage area, i.e. 3.5 blocks in total (about 4M bits × 3.5 = 14M

bits); and about 2M remaining bits can be used as a buffer area for coded data. The data bus reads and writes coded data or image data of 64 (16 × 4) bits in length, and 40 MHz is selected as the decoding clock frequency.

EP 0 599 529 A2 describes a method and a device for coding/decoding frame/field-coded image data. The known method and the known device are not capable of outputting image data in accordance with television standards. Conventional television image signals cannot be transmitted, but only very small format images can be displayed at a low frame rate.

DE 38 31 277 A1 describes a method for storing and reproducing video signals, which distinguishes between intraframe and interframe coding.

EP 0 598 904 A1 describes a device for coding and decoding image signals, whereby the data transfer rate of 1.5 Mbit/s mentioned therein is only sufficient for the display of small-format moving images at a low frame rate, but not for the display of television image signals.

Based on this prior art, the invention is based on the object of creating a device for decoding image data which makes it possible to create image data in television quality.

According to the invention, this object is achieved by the decoding device defined in claim 1.

As a summary of the invention and in comparison with a conventional image decoding device, the following results:

1) An advantage of the present invention is that interlace conversion of decoded image data of a single image memory can be performed and the memory capacity can be reduced. In this way, the memory capacity can be reduced to less than 16M bits even in the 625/50 system (1 frame = about 4.7M bits) used in Europe in which the number of one-frame image data is larger than that in the U.S. and Japan. Further, the delay caused by decoding can be reduced.

2) Another advantage of the present invention is that various asynchronous operations can be performed without using the arbitration function for the memory access right. In this way, the block decoding frequency is reduced. This also has the effect of reducing the circuit size by reducing

is of power consumption and increasing the margin for circuit operation delay.

3) Still another advantage of the invention is that the memory can be efficiently accessed and the data bus width of the memory can be reduced. In this way, for example, in order to manufacture an LSI (large scale integrated circuit) for an image decoding device, the number of pins can be reduced and further the number of conductor tracks of a circuit board can be reduced.

In particular, the present invention comprises: decoding devices for obtaining decoded image data by decoding coded data of a data-compressed video signal in block units consisting of a plurality of pixels in the frame; and display devices for reading out the decoded data stored within the storage devices in field units.

ized image data based on a display synchronization signal; and obtaining interlaced display image data.

Data processing takes place as follows:

1) The data processing for decoding is performed synchronously with the operations for display, and the smallest delay time from the start of decoding the coded data of a single frame to the start of display is 0.5 frames.

2) The time slot is clocked based on the display synchronization signal, and the decoding device and the display device are accessed based on the time slot.

3) The storage device consists of two storage fields; and for the access of the decoding device and the display device to the storage device, the two storage fields are used alternately. The device according to the invention carries out the following operations.

1) Since the operations for decoding are performed synchronously with the operations for the display, and the delay time from the start of decoding to the start of display is 0.5 frames, the decoded image data of the frame is written during decoding with the capacity of at least one frame; and the reading out of the decoded image data of the previous frame as display image data is completed before the decoded image data of the previous frame is lost by rewriting of currently decoded image data; and finally an interlaced display output is obtained. The result is that with a memory capacity of

three images or less, including the memory for storing the reference image data, the operations for decoding and the operations for display can be performed.

2) The memory is accessed on the basis of the predetermined time slot; and even if there is considerable variation in the amount of processing required to decode the coded data of each frame, the coded data of one frame can always be decoded during one field period. A decision

&iacgr;&ogr; about the memory access right between different memory accesses

is not required, so that the decoding clock frequency can be reduced and the circuit size can also be reduced.

3) Since two memory fields are used alternately, it is possible to continuously read data from or write data to the memory. In this way, the memory can be accessed efficiently, and the required memory access can be performed even with a small data width.

Further features, advantages and possible applications of the present invention will become apparent from the following description of embodiments in conjunction with the drawing. In this drawing:

Fig. 1 is a block diagram of an apparatus for decoding images, which is an embodiment of the present invention;

Fig. 2 is a diagram illustrating an embodiment of the division of the memory area;

Fig. 3 is a block diagram of the clock unit shown in Fig. 1;

Fig. 4 is a block diagram of the input buffer shown in Fig. 1 and the buffer for decoded data;

Fig. 5 is a block diagram of the motion compensation unit shown in Fig. 1;

Fig. 6 is a block diagram of the display unit shown in Fig. 1;
Fig. 7 is a block diagram of the memory controller shown in Fig. 1;

Fig. 8 shows timing diagrams of the decoding operations and the display operations;

Fig. 9 is an enlarged view of the timing shown in Fig. 8;

Fig. 10 Timing diagrams of the operations for decoding and the operations for display when no B-frame is involved;

Fig. Il is a diagram for explaining the memory control system corresponding to a television signal of the 525/60 system;

Fig. 12 is a diagram for explaining the memory control system corresponding to a television signal of the 625/50 system;

Fig. 13 is a block diagram of the memory shown in Fig. 1;

Fig. 14 is a diagram illustrating an embodiment of the division of the memory area shown in Fig. 13;

- li -

Fig. 15 is a block diagram of a memory control signal generator unit for the memory shown in Fig. 13;

Fig. 16 is a diagram showing the control timing for the memory shown in Fig. 13;

Fig. 17 is a diagram illustrating the block division of a single-frame image of the present invention;

Î Fig. 18 is a diagram showing the mapping of the macroblock into the image storage area in the memory shown in Fig. 13; and

Fig. 19 is a diagram illustrating the reference image data reading order of the present invention.

Now, the embodiments of the present invention will be explained with reference to the accompanying drawings.

Fig. 1 is a diagram illustrating an image decoding apparatus of the present invention. Reference numeral 1 denotes a decoding circuit for performing the operations for decoding and the operations for displaying coded data; and Fig. 2 denotes a memory connected to the decoding circuit 1. The decoding circuit 1 is composed of an input buffer memory 11, a decoded data buffer memory 12, a variable length decoding unit 13, an IDCT (inverse discrete cosine conversion) unit 14, a motion compensation unit 15, a display unit 16, a memory controller 17, and a clock unit 18.

: i

12 -

Fig. 2 shows an area division map of the memory 2. The size of each picture is set to 4.7M bits considering the television system in Europe, and a capacity of 16M bits is divided into three pictures; and the buffer for coded data has a capacity of at least 1.8M bits.

As shown in Fig. 1, the coded data (compressed image data) is input to the buffer memory 11 of the decoding circuit 1. Further, the coded data is temporarily stored in the memory 2 from the input buffer memory 1 via the data bus γ and the memory controller 17. After temporarily storing, the coded data is read out from the memory 2 via the memory controller 17 and stored in the decoded data buffer memory 12.

is The encoded data is supplied from the buffer memory 12 and to the variable length decoded data decoding unit 13 upon request of the variable length decoding unit 13. The variable length decoding unit 13 decodes the coefficient data of the encoded data obtained by discrete cosine transform; the motion vector information and the coding type information, and sends the factor data to the IDCT unit 14, the motion vector information to the motion compensation unit 15, and the coding type information to the display unit 16. The IDCT unit 14 performs an inverse discrete cosine transform of the coefficient data, generates IDCT image data and supplies it to the motion compensation unit 15. The motion compensation unit 15 reads out reference image data from the memory 2 based on the motion vector information and generates decoded image data by adding the IDCT image data to the reference image data.

- 13 -

Further, the decoded image data is stored in the memory 2 via the memory controller 17. Thereafter, the decoded image data is read out upon request of the display unit 16 and output from the display unit 16 as display image data. The decoded image data of the I frame or the P frame is also used as reference image data.

Fig. 3 is a diagram showing the structure of the essential part of the clock unit 18. As shown in the drawing, the clock unit 18 operates in the external synchronization mode to externally

Î to provide the horizontal synchronization signal and the vertical synchronization signal of the video signal. The clock unit 18 may have a synchronization signal generator for generating a horizontal synchronization signal and a vertical synchronization signal therein and operate in the internal synchronization mode in which the synchronization signal generator is triggered in time with the decoding of the first frame of the encoded data and generates a horizontal synchronization signal and a vertical synchronization signal. In the figure, 181 denotes a horizontal clock generator circuit, 182 a vertical clock generator circuit and 183 a logic circuit. The horizontal clock generator circuit 181 is reset by a horizontal synchronization signal and forms a horizontal pixel counter for performing a counting operation in accordance with, for example, a clock signal of 13.5 MHz. The horizontal pixel counter repeats the counting operation for the total number of pixels in one line including the horizontal blanking period, i.e., for 858 pixels. The vertical clock generator circuit 182 is reset by a vertical synchronization signal and forms a vertical line counter for performing a single counting operation each time the horizontal clock generator circuit 181 finishes the counting operation of the total number of pixels constituting one line. The vertical line counter performs a counting operation for the total number of lines in one field including the horizontal

»&Ggr;:

- 14 -

blanking period, i.e. over 263 lines or 262 lines. The vertical line counter performs the counting operation alternately over 262 lines in the first field or 263 lines in the second field, which form a frame.

The logic circuit 183 outputs the input clock of 13.5 MHz as a display clock and further generates a decoding clock of about 22 MHz through a PLL circuit and outputs it. The frequency of the decoding clock is selected so that the encoded data of one frame during the one-frame period

γ can be decoded regardless of the coding type; and the decoding clock becomes a clock signal which is the basis for timing the operations for decoding in the decoded data buffer memory 12, the variable length decoding unit 13, the IDCT unit 14 and the motion compensation unit 15. The frequency of the display clock is equal to the sampling frequency of the luminance signal, and the display clock constitutes a clock signal which is the basis for timing the operations for display in the display unit 16. Further, a clock signal which is two or three times larger than the decoding clock is supplied to the memory controller 17 as a memory clock. When the memory clock is twice as large as the decoding clock, the data bus width in the decoding circuit 1 is set to be twice the data bus width in the memory 2, and when the memory clock is three times as large, the data bus width in the decoding circuit 1 is set to be three times so that the data rate supplied to the memory controller 17 via the data bus in the decoding circuit 1 is made equal to the data rate passing from the memory controller 17 to the memory 2. For the purposes of the following explanation, the memory clock is three times the decoding clock.

- 15 -

In addition, the logic circuit 183 generates and outputs various timing control signals from the horizontal pixel count value generated by the horizontal clock generating circuit 181 and a vertical line count value generated by the vertical clock generating circuit 182. It also synchronizes the operations for display with the operations for decoding.

Fig. 4 is a diagram showing the structure of the input buffer memory 11 and the buffer memory 12 for decoded data shown in Fig. 1

&iacgr;&ogr;. In the input buffer memory 11, 111 denotes a parallelization circuit, 112 a FIFO memory, 113 a calculator for determining the capacity of an empty memory area, and 114 a FIFO controller. In the decoding data buffer memory 12, 121 denotes a FIFO memory, 122 a calculator for determining the capacity of an empty memory area, and 123 a FIFO controller.

The input buffer memory 11 has the function of transferring coded data from the input unit to the memory 2. The coded data has a configuration of, for example, 8 bits and is input to the parallelizing circuit 111 of the input buffer memory 11 according to an input clock. The parallelizing circuit 111 parallelizes the input data with 48-bit data (the bus width of the memory is set to 16 bits) having the same data width as the data bus, and outputs the data to the FIFO memory 112. The FIFO memory 112 is the well-known first-in/first-out memory and is controlled by the FIFO controller 114. The control of the FIFO controller 114 is based on the result that occurs when the capacity of the empty storage area of the FIFO memory 112 is calculated by the computer for the capacity of the empty storage area 113, and it is based on a data request signal. If the data request signal is a data permission

- 16 -

sensing status, and when the calculation result of the calculator for the capacity of the empty storage area 113 indicates that the predetermined number of data is stored in the FIFO memory 112, the encoded data is read out from the FIFO memory 112 to the data bus. The read clock is the decoding clock, and the asynchronization of the input clock and the decoding clock is released from the FIFO memory.

The buffer memory 12 for decoded data transmits the decoded data with a width of 48 bits from the memory 2 via the FIFO memory 12 to the

‘Variable length decoding unit 13, which is equivalent to the first stage of the image decoding processing shown in Fig. 1. The FIFO memory 121 is controlled by the FIFO controller 123. Namely, in order to supply data to the FIFO memory 121, the empty memory area capacity calculator 122 calculates the empty area of the FIFO memory 121; outputs a data request signal to the memory controller 17 when there is an empty area in the FIFO memory 121; receives a data acknowledge signal output from the memory controller 17; and writes the predetermined number of coded data read out from the memory 2 into the FIFO memory 121. Further, the FIFO memory 121 outputs coded data upon request of the variable length decoding unit 13.

Fig. 5 is a diagram showing the structure of the motion compensation unit 15. In the figure, 150 denotes a motion vector decoder, 151 an adder, 152 a serializing circuit, 153 and 154 reference picture memories, 155 a timing controller for the reference picture memories 153 and 154, 156 a parallelizing circuit, 157 and 158 decoded picture memories, and 159 a timing controller for the decoded picture memories 157 and 158.

5 '

The motion vector decoder 150 decodes differentially encoded motion vector information input from the variable length decoding unit 13 and sends it to the memory controller 17.

Reference picture data read out for motion compensation is input from the memory 2 to the reference picture memories 153 and 154 via the data bus; and the two reference picture memories 153 and 154 are read and written alternately in macroblock units, which will be explained later. Namely, when one of them is in the reference picture data write mode, the other is in the read mode. The reference picture data depends on the picture coding type, with P frame data being data from the previous picture only, while B frame data being data from the previous and subsequent pictures. The E frame does not require reference picture data, and no data is read from the memory 2.

The reference image data read out from the reference image memories 153 and 154 is supplied as one of the inputs of the adder 151 through the serializing circuit 152, which converts data of 48-bit width into data of pixel units. In order to decode data encoded in B frames, the serializing circuit 152 calculates the average value of the reference image data of the previous image and the subsequent image if necessary, and outputs the average value. The other input of the adder 151 is the IDCD image data subjected to the inverse discrete cosine transformation by the IDCD unit 14. Motion compensation is then performed by the adder 151, and decoded image data is generated. Of the decoded image data, some pixels are parallelized by the parallelizing circuit 156, and the data width thereof is again set to 48 bits. Then, the data is supplied to the decoded picture memories 157 and 158. The decoded picture memories 157 and 158 also operate in macroblock units in bank format; and

- 18 -

when one memory is in the decoded image data write mode, the other is in the read mode. Further, the decoded image data read from the decoded image memories 157 and 158 are written into the memory 12 via the data bus.

Fig. 6 shows the detailed structure of the display unit 16. In the display unit 16, 161 denotes a brightness line memory, 162 an interpolation circuit, 163 and 164 two kinds of color signal line memories, 165 a serializing circuit, 166 a clock controller, 167 an OSD generating circuit, and 168 a multiplexer. The serializing circuit 165 converts display image data inputted through the data bus with a 48-bit data width into data in the form of 8-bit pixel units and outputs them sequentially. The display image data of the brightness signal is written into the brightness signal line memory 161, and the display image data of the two kinds of color signals are written into the color signal line memories 163 and 164, respectively. For example, when data is read out from the memory 2 three times during one horizontal scanning period, the luminance signal line memory 161 is a FIFO memory with a capacity of 240 bytes, and the color signal line memories 163 and 164 are FIFO memories each with a capacity of 120 bytes.

The display image data of the luminance signal is sequentially read from the luminance signal line memory 161 according to a display clock of 13.5 MHz during the display period, excluding the horizontal blanking period and the vertical blanking period. At the same time, the display image data of the two types of color signals is sequentially read from the color signal line memories 162 and 163 according to a clock of 6.75 MHz, which is half the frequency of the display clock. The clock for writing and reading each of the line memories 161, 162 and 163 is

•&igr;·♦ ·· ·

controlled by the timing controller 166 according to a timing control signal supplied from the timing unit 18.

The interpolation circuit 162 performs the operations for interpolating in the vertical direction for the display image data of the two kinds of color signals, respectively, and makes the number of vertical lines equal to that of the luminance signal. Thereafter, the interpolation circuit performs time multiplexing alternately for the display image data of the two kinds of color signals in pixel units. In each field, the color signal ω of 120 lines decoded by the operations for decoding is converted into a signal of 240 lines, which is doubled. For this reason, a line memory for storing the decoded image data of the two kinds of color signals of the previous line is included in the interpolation circuit 162.

The display unit 16 can perform the function of setting an OSD area for storing OSD (on screen display) data in the memory 2 by reading out the OSD data from the OSD area as part of the display image data; generating bit map image data of characters and graphics; and superimposing it on the display image data.

The OSD generator circuit 167 first stores and holds the OSD data read out from the memory 2 internally and generates bit map image data of the characters and graphics according to the output timing of the display image data. The multiplexer 168 superimposes the bit map image data output from the OSD generator circuit 167 on the display image data output from the line memories 161, 163 and 164 and outputs it as output image data.

Fig. 7 is a diagram showing the structure of the memory controller 17. In the diagram, 171 denotes a serialization circuit, 172 a

Parallelization circuit, 173 a calculator for determining the capacity of an empty memory area, 174 a write address generator unit for coded data, 170 a read address generator unit for coded data, 176 a read address generator unit for motion-compensated reference image data, 177 a write address generator unit for decoded image data and 178 a read address generator unit for display image data.

The serializing circuit 171 converts data input via the data bus from 48-bit data into three 16-bit serial data that constitute the input/output bus width of the memory 2, and outputs it to the memory 2.

The parallelizing circuit 172 converts data of 16-bit width input from the memory 2 into three serial-continuous data in parallel and outputs them as 48-bit data to the data bus.

Reference numerals 174 to 178 denote generators which generate an address signal and a control signal of the memory 2. Depending on the type of data output to the memory 2 or input from the memory 2, one of the generator units operates, the output of the operating address generator unit being supplied to the memory 2 as an address signal and a control signal.

The coded data write address generator unit 174 generates address and control signals for sequentially writing coded data and controls the writing of the coded data into the memory 2. The coded data read address generator unit 175 generates address and control signals for sequentially reading out the coded data and controls the reading out of the coded data from the memory 2. When no data request signal is output from the decoded data buffer memory 12, even

- 21 -

not during a period in which coded data can be read out, reading out of the coded data from the memory 2 is stopped. The empty memory area calculation calculator 173 calculates the capacity of the empty area of the buffer memory for coded data allocated to the memory 2 from the write address generated by the coded data write address generating unit 174 and for coded data allocated to the memory 2 from the read address generated by the read address generating unit 175, and outputs a data request signal to the input buffer memory 11 when there is a free area for storing the coded data.

The motion-compensated reference image data read address generator unit 176 generates address and control signals for sequentially reading out the reference image data, and it controls the reading out of the reference image data from the

is memory 2. An offset value is added to the reference picture data read address according to the motion vector value supplied from the motion compensation unit 15. The type of reference picture data to be read out depends on the coding type of the picture during decoding; however, there may be no need to read the data. Therefore, the number of data required for reading as reference picture signals depends on the case. Therefore, even during a period in which the reference picture data can be read out, the reading out of the reference picture data from the memory 2 may end halfway. The decoded image data write address generator unit 177 generates address and control signals for sequentially writing decoded image data and controls the writing of the decoded image data into the memory 2. The display image data read address generator unit 178 generates address and control signals for sequentially reading out display image data (there is a case where OSD data is included), and controls the reading out of the display image data from the memory 2. In this case, the unit 178 selects one of the three image memories in the memory

2 according to the coding type output from the variable length decoding unit 2 and generates an address accordingly.

Figs. 8, 9 and 10 show a control system of the memory 2 for synchronizing the operations for decoding and the operations for display. This memory control is performed by the memory controller 17 on the basis of a clock control signal generated by the clock unit 18 from the horizontal synchronization signal and the vertical synchronization signal.

Fig. 8 shows diagrams to explain the sequence and timing of the decoding process and the display process.

Figure 8(a) shows the frame order of coded data to be decoded, while Figure 8(a) shows the frame order of display image data to be displayed. Figures 8(b) to 8(d) show memory images of the three image memories (hereinafter referred to as FM 1 to FM 3) in the memory 2, assuming that each image memory consists of two field memories. The arrows pointing downward from (a) to (b) to (d) indicate writing of the decoded image data, and the arrows pointing upward from (b) to (d) to (a) indicate reading of the reference image data. The arrows pointing downward from (b) to (d) to (e) indicate reading of display image data.

According to the present invention, the decoding operations are carried out so that the operations for decoding coded data of each picture always end within the field period, and the operations for decoding coded data are stopped only for the predetermined period of time during the operations for decoding each picture. The timing diagrams can be applied to the decoding operations and to the

Operations for the display of coded data from two current television video signals of the 525/60 and 625/50 systems.

The decoded image data is written into one of the memories FM 1 to FM 3. The decoded image data of the I frame or the P frame, which is to be used to predict the P frame and the B frame, is alternately written into the memories FM 1 and FM 2. The decoded image data of the B frame is written into the memory FM 3. The dense lines shown in Figs. 8 (b) to 8 (d), which are made slightly wider, indicate the writing situation of the decoded image data.

To decode the coded data of the P frame, the decoded image data of the previous frame is read out as reference image data from FM 1 or FM 2. The selection of FM I or FM 2 for reading the reference image data is controlled by selecting the frame memory in which a frame adjacent to the frame currently decoded on a time basis is stored. As the frame memory in which the decoded image data is written, a frame memory different from the frame memory from which the reference image data is read is selected, and the decoded image data is written therein. To decode the coded data of the B frame, the decoded image data of the previous frame and the subsequent frame are read out as reference image data from the memories FM 1 and FM 2. The thin hatched lines shown in Figs. 8(b) to 8(c), which are wider, indicate the reading of the reference image data.

As shown in the drawing, in the operations for decoding each image, the operations for decoding the coded data are stopped for a predetermined period of time. The blocks 13, 14 and 15

each of the decoding operations includes a decoding stopping means for stopping the decoding operations based on a clock signal received from the clock unit 18.

Display image data can be obtained by reading decoded image data of an image stored during the one-frame period in any of the memories FM I to FM 3 in the display order. The frame period for decoding and the frame period for display are shifted from each other by 0.5 frame, i.e., one field period.

Reading of the display image data of the B frame for display is started one field period after the frame period from which it was decoded and since its writing into the memory FM 3 was started. In the I frame and the P frame, there is still a delay caused by the image rearrangement operations. In Figures 8(b) and 8(d), each of the thick solid lines shows the situation of reading the display image data. The choice of the image memory for display processing is decided by observing the coding type of the frame during decoding. As for the I frame and the P frame, they are the same as the image memory from which the reference image data of FM 1 or FM 2 is read out, except for the one field delay. The B frame is the same as the memory FM 3.

As shown in Figure 8 by the designations B2 and B3, when a plurality of B frames occur, the decoded image data of the B frames obtained by decoding them are written into the memory FM 3 during the continuous image period. Before the decoded image data of the previous B2 frame is written again, because newly decoded image data of the B3 frame is written,

· &Ggr;·, '.&igr; y

j ;

- 25 -

Therefore, it is necessary to display and read out the decoded image data of the B2 frame. To do this, a delay of one field period is provided between the decoding and writing of the B frame and the display and reading thereof; and at the same time, the predetermined hold period is provided for performing the operations for decoding the coded data. The hold period for the operations for decoding is provided not only for the B frame, but also for the I frames and P frames. In this way, the timing for decoding is made equal regardless of the image coding type, and the writing of the decoded image data is generalized regardless of the coding type.

Figure 9 is an enlarged view of the portion enclosed in an ellipse in Figure 1, showing the situation of the memory control of FM 3 when the B frames continue, that is, a view for explaining the pause period in the operations for decoding. The shape of the small rectangles shown continuously and stepwise from the upper left to the lower right in Figure 9 represents the situation of decoding write address changes. The addresses of FM 3 are allocated in the order of the interlaced display scanning, such as from left pixels to right pixels, upper lines to lower lines, and also from the first field to the second field. Therefore, the write addresses for the decode write command in each block in the B frame are not continuously increased but are half-omitted. For the decode write command in a frame, the addresses are gradually increased as a whole, although the write addresses are broken off.

It is assumed that a set of all blocks that are arranged horizontally in a frame with the same vertical position is called a block line; and that write addresses for the decode write command and

··
• * ··
• : ■ *· ·· •
• m
···· »·· ·· ·· - 26

Read addresses for the display read command are converted into block column units. This is equivalent to a pixel set of 16 lines. Therefore, after the blocks are decoded sequentially in each block line, the decode write command addresses are positioned at least within the range of addresses corresponding to each pixel in the block line. Namely, there are addresses for 16 lines. The address range corresponding to each block line is represented by the rectangles shown in Fig. 9. The height of the rectangles is equal to half the number of vertical lines that form the block lines in the frame, i.e., in the address range,

&iacgr;&ogr; which corresponds to the number of vertical lines in the field, i.e. the addresses for 8 lines.

In Figure 9, the thick line running from top left to bottom right indicates the reading of display image data.

In the area shown in the drawing, the B frame indicated by B2 is read. For display reading in two fields, the read addresses in this B frame are continuously increased. During the vertical blanking period that exists between field and field, the display read command is temporarily stopped. The slope of the thick line indicating a change in the display read addresses is twice the slope of the steps of the lined-up rectangles and indicates a change in the decoding write addresses.

It is necessary to execute the display read command for sequentially reading out each pixel data of B2 from FM 3 after the decoded image data of pixels B2 are written into FM 3 by decoding the written one, and before they are written into FM 3 by the decode write command and before they are written again by the decode write command of B3 which is the next B frame. It is

namely, the shape of the stepwise connected rectangles indicating a change in the write addresses of the decoded image data is prevented from intersecting the thick solid line indicating a change in the display addresses of the display image data. To this end, in the present embodiment, a delay of the field period is provided between the decoding write commands of the B frames and the display read commands, and the predetermined stop period is provided for simultaneously performing the decoding of the coded data in each frame. Namely, the decoding write command of the B2 frame is executed before the display read command of the first field of the B2 frame ends, while the display read command of the second field of the B2 frame starts before the decoding write command of the B3 frame starts.

The length of the hold period provided between picture periods for decoding each picture is the sum of the display periods of all the lines of the lowest block line in each field, the vertical blanking period between the first field and the second field, and the display period of all the lines of the highest block line in the second field. For example, in the 625/50 system, the vertical blanking period between the first field and the second field is equal to the display period of about 25 lines, so that the length of the hold period of the operations for decoding is the length of time corresponding to the display period of 8 + 25 + 8 = 41 lines. To shorten the hold period and keep the period of the operations for decoding each block as long as possible, the hold period is shortened to the smallest length.

According to the above-mentioned memory control system of the present invention, the operations for decoding and the operations for display can be carried out by 3 image memories. The storage capacity of one image required in the system 625/50 is about 4.7 Mbits, so

that the total image storage capacity is about 14 Mbits. The delay time from the start of decoding the coded data to the start of display of the display image data can be set to a period of 1.5 frames.

Figure 10 is a diagram showing the sequence and timing of the operations for decoding and the operation for display when no image interpolation coding is used. In this example, only a single image memory (FM la) is used. Its size is limited to

× is set to a slightly larger capacity than the capacity for one frame. Figure 10 (a) shows the frame order of the coded data to be decoded, while Figure 10 (c) shows the frame order of the display image data to be displayed. Figure 10 (b) shows the situation of memory access of FM la; and two field memories whose size is larger than that of one field in the predetermined size are shown separately. The two field memories are separated by a thick hatched line. The arrow pointing downward from (a) to (b) indicates the situation of the decode write command, and the arrow pointing upward from (b) to (a) indicates the reading of the reference image data, while the arrow pointing downward from (b) to (c) indicates the reading of the display image data. In the drawing, when decoded image data is written into the memory FM la, the write address for writing into each field memory is decided as follows. An offset of one field is added to the write address into each field memory for each frame period, and then a modulo operation is performed according to the capacity of each field memory, the size of which is larger than that of one field by a predetermined amount. Namely, each field memory is used as a ring buffer. The densely hatched lines shown in Figure 10 (b), which are slightly wider, indicate the write situation of the decoded image data.

In the coded inter-picture blocks in P frames, the decoded picture data of the previous picture stored in FM la is read out as reference picture data. In this case, a positive or negative offset is added to the read address according to the size of the motion vector. The decoded picture data decoded in the previous frame is read out in a reference-rate manner. Therefore, the same offset as that of the previous picture is added to the read address from each field memory, and then a modulo operation is performed according to the field memory capacity. In Figure 10 (b), the thin hatched lines, which are wider, indicate this situation.

The display read command of the decoded image data is executed during each frame period by reading the decoded image data from each frame stored in the FM la. In the same manner as in Figure 8, the frame period for decoding and the frame period for display are shifted from each other by one field period. The decoded image data for which the decode write command is initiated by one field before is read out in a display-oriented manner so that the field memory adds the same offset as that of the decode write address to the read address, and then a modulo operation is performed according to the field memory capacity. In Figure 10 (b), the thick solid lines show the situation of this display read step.

The capacity of each field memory is made larger than that of a field by the predetermined amount because it is necessary to finish reading the reference image data of the previous image and reading the display image data before the decoded image data of the image memory FM la are rewritten by the decoding write command of a new frame. The capacity is namely increased by an amount corresponding to the maximum

- 30 -

mum corresponds to the number of vertical lines, which is the range by which the block is shifted in accordance with the motion vector in the field, so that the densely hatched lines indicating a change in the decoding write address and the thinly hatched lines indicating a change in the reference read address do not intersect each other. For example, in a television signal of the 625/50 system, the capacity of each field memory is set to a capacity which is 64 lines larger than that of a field in the luminance signal, the total capacity of the frame memory being about 5.8 Mbits (M bits = 1024 x 1024 bits). A hold period is provided between the frame periods for decoding each frame, the length of the hold period being the same as that shown in Figure 8.

As mentioned above, the device shown in the example of Fig. 8 may be composed of a frame memory whose size is slightly larger than that of one frame. The delay time between the start of decoding of the coded data is 0.5 frame period, so that the frame memory capacity can be reduced and at the same time the delay time from the start of decoding can be shortened by a single frame, i.e., 0.5 frames. As mentioned above, the present memory control system is effective for efficiently performing decoding and displaying decoded data with a small number of memories and a short delay time by the coding system in which non-use of the B frame is specified.

If the memory has a capacity of 16 Mbits, and if the encoded data further includes information about the type of the encoded data in the form of a multiplexed flag, it is possible to automatically switch the memory control system shown in Fig. 8 as well as the memory control system shown in Fig. 10 by this flag. By using

This system allows a device for image decoding to cooperate with a two-way communication system in which a short delay time is essential and only δ and P are encoded, both from a broadcast reception system and a recording media playback system in which high image quality is desired and I, P and B are all encoded.

Fig. 11 is a diagram showing the control system of the memory 2 for performing the operation for synchronizing the operations of the

Î decoding and display operations are performed during the frame period; and further, Fig. 11 is a diagram for explaining an example of the system 525/60. In this example, in the decoding and display operations, the access to the memory 2 is made through the fixed time slot which is based on a

is a horizontal synchronization signal and a vertical synchronization signal.

In the 525/30 system, the frame rate is 30 Hz, while the sampling frequency of the brightness signal is 13.5 MHz. A frame consists of 525 lines in total, and the first field consists of 262 lines, while the second field consists of 263 lines. For example, if a clock of 65.25 MHz is used as the memory clock, the one-line period extends over a time of 858 x 29/6 = 4147 clocks. The one-line period is divided into three macroblock time slots of 1380 clocks each, with the remaining seven clocks being dummy slots. During the dummy slots, data access to memory 2 is halted.

A number of 1458 time slots allocated from the 93rd line to the 524th line and from the 0th line to the 253rd line are used to decode the coded data of one picture. From the 285th

From the 524th line to the 524th line, the image data in the second field of the image which is already decoded is displayed, and from the 22nd line to the 261st line, the image data in the first field of the image which is currently being decoded is displayed. In each macroblock time slot, various data accesses to the memory 2 relating to the operations for decoding a macroblock and reading out the decoded image data to be displayed from the memory 2 are carried out on a time-division basis. The procedure for decoding the coded data of a macroblock is also carried out according to the macroblock time slots.

The macroblock is a set of image data in an area of 16 pixels × 16 lines for a brightness signal, or 8 pixels × 8 lines for two color signals. The block size is 8x8 pixels and consists of four

is blocks for the luminance signal or a single block for each two kinds of color signals. Therefore, a macroblock consists of six blocks in total. One frame consists of 720 pixels x 480 lines, so that one frame consists of (720/16) x (480/16) = 1350 macroblocks. In order to decode coded data of one frame, 1458 time slots are allocated to one frame period, so that if the operations for decoding one macroblock are performed substantially in each time slot, the operations for decoding one frame can be performed during one frame period.

As shown in Fig. 11, in each macroblock time slot, with respect to the memory access required for the operations of decoding and display, three kinds of time slots are provided for reading out (a) the display image data read command, (b) the reference image data read command and (c) the coded data read command from the memory 2. Next, a time slot is provided for (d) the memory refresh. In the memory

- 33 -

2, which is composed of a dynamic memory device (DRAM), cyclic refresh is required, so that in the (d) memory refresh command, blind reading of the memory 2 is performed by sequentially increasing the address. Further, two kinds of periods for writing into the memory 2 are provided, namely, for the (e) write command for coded data and the (f) write command for decoded image data.

In addition, although not shown in the drawing, in case the device has an OSD function, an (g) OSD data read command and an (h) OSD data write command are additionally provided.

In the decoding circuit 1 shown in Fig. 1, the input buffer memory 11 writes coded data from the internal FIFO memory into the memory 2 during the slot period of the (e) write command. The buffer memory

[5 rather 12 reads out coded data from the memory 2 during the slot period of the (c) coded data read command and writes it into the internal FIFO memory. The motion compensation unit 15 reads out reference picture data of a single macroblock from the memory 2 during the slot period and writes it into the internal reference picture memory, and writes decoded picture data from the internal decoded picture memory into the memory 2 during the slot period of the picture data command. Further, the display unit 16 reads out display picture data from the memory 2 during the slot period of the (a) display picture data read command and writes it into the internal line memory.

Fig. 12 is a diagram for explaining the memory control system corresponding to a television signal of the system 625/50. It corresponds to Fig. 11 which shows the case of the system 525/60.

• 1 11 1 ··

- 34 -

In the 625/50 system, the frame rate is 25 frames/second and the sampling frequency of the luminance signal is 13.5 MHz. The memory clock is generated from this sampling frequency and is the same as that in the 525/60 system. A frame consists of 625 lines and the first field of a frame consists of 312 lines while the second field consists of 313 lines.

The online period extends over a duration of 864 x 29/6 = 4176 cycles, and the online period is divided into three time slots of 1380 cycles each

&iacgr;&ogr; while the remaining 36 clocks are dummy slots. The number of clocks chosen during the online period is the same as that in the 525/60 system to emphasize commonalities in decoding and display, with any difference between the two systems being absorbed by the number of clocks of dummy slots. During the

&ugr; Blind slot period the data access to memory 2 is stopped.

A total of 1752 macroblock time slots allocated to the 345th line to the 624th line and the 0th line to the 303rd line are used for decoding the coded data of one picture. From the 337th line to the 624th line, the picture data of the second field of the picture which has already been decoded is displayed on the display, and from the 24th line to the 311th line, the picture data of the first field of the picture which is currently being decoded is displayed. In each macroblock time slot, various data accesses to the memory 2, which consist of the operations for decoding a macroblock and for reading out the decoded picture data to be displayed from the memory 2, are performed on a time-division basis. The operations for decoding a macroblock are also performed according to the macroblock time slots.

- 35 -

A picture consists of 720 pixels x 576 lines, so that a single picture consists of (720/16) x (576/16) = 1620 macroblocks. To decode coded data of a picture, 1752 time slots are allocated to a field, so that if the operations for decoding a macroblock are performed substantially in each time slot, the operations of decoding a picture can be completed during a single picture period.

In each macroblock time slot, a plurality of types of time slots are provided in the same way as in the system 525/60 shown in Fig. 11.

βββ are provided, namely a (a) display image data read command; a (b) reference image data read command; a (c) read command for coded data; a (d) memory refresh command; a (e) write command for coded data; and a (f) write command for decoded image data. The remaining time slots after the end of the above operations form a marginal area, and the memory access

is essentially stopped.

In Figs. 11 and 12, the time slots for the operations for decoding and the operations for displaying one macroblock are allocated accordingly. However, time slots may be allocated in a smaller unit such as a two-block unit, for example. In this case, the switching frequency of the memory access process is increased, so that the operations of the clock unit and the memory controller become somewhat complicated. However, the size of a working memory required for each switching operation for performing decoding and display can be reduced from the value corresponding to one macroblock to the value corresponding to two blocks.

Fig. 13 shows an example of the structure of the memory 2, which enables efficient access to the memory 2 from the decoding unit 1. The reference numerals: 21 denote a bank selector; 22 a

Row address buffer; 231 a column address buffer; 232 a column address counter; 241 and 242 row address decoders; 251 and 252 column address decoders; 261 and 262 sense amplifiers and I/O buses; 271 and 272 memory arrays; 281 an input data buffer; and 282 an output data buffer.

The memory 2 includes two memory arrays, and each memory array includes address control circuits such as a row address decoder and a column address decoder.

An address and a control signal are applied to the bank selector 21, the row address buffer 22 and the column address buffer 231. The bank selector 21 determines the bank for which the address is effective and controls the row address decoders 241 and 242 and the column address buffer 231.

When the address is a row address, the bank selector 21 supplies it to the row address decoders 241 and 242 via the row address buffer 22, and activates the memory array corresponding to the specified row of the memory array 271 or the memory array 272 according to the decoding result of the row address decoder 241 for the memory array 271 (hereinafter referred to as bank 0) or the row address decoder 242 for the memory array 272 (hereinafter referred to as bank 1). When the address is a column address, the bank selector 21 holds it in the column address buffer 231 once and regenerates a column address by the column address counter 232 based on the held value. In this way, a column address in the same row can be automatically generated without continuously supplying it. The column address is decoded after regeneration by the column address decoder 251 or by the column address decoder 252. When the memory is in the write mode, the bank selector 21 writes data to be output into the specified address of the memory array 271 or 272 via the input data buffer

- 37 -

281 and the sense amplifier and the I/O bus 261 or 262. When the memory is in the read mode, the bank selector 21 reads the data into the specified address of the memory arrays 271 or 272 through the sense amplifier and the I/O bus 261 or 262 and outputs it through the output data buffer 282.

Fig. 14 is a diagram showing a map of the data arrangement of the memory 2. The reference numerals indicated in the drawing correspond to those of the system 525/60, and an example of a case is shown in which the image data of one frame consists of 720 horizontal pixels and 480 vertical lines. The memory arrays 271 and 272 of the bank 0 and the bank 1 are mapped into three image memories each consisting of 507 rows including the 528 rows in the buffer area for coded data.

Fig. 15 is a diagram showing the structure of the various address generator units, starting from the write address generator unit 174 for coded data to the read address generator unit 178 for display image data in the memory controller 17, the structure having a bank switching control function in accordance with the memory 2 shown in Fig. 13. In the read address generator unit 176 for the motion-compensated reference image data, the portion in which the offset values of the row and column addresses are specified by the motion vector is not shown. In Fig. 15: 71 denotes a row address generator circuit; 72 a column address generator circuit; 73 a multiplexer; 74 a bank selector; and 75 a clock controller.

The row address generator circuit 71 generates a row address of the memory 2, and the column address generator circuit 72 also generates

•&iacgr;

- 38 -

a column address of the memory 2. The row and column addresses are multiplexed by the multiplexer 73 and output to the address bus. The bank selector 74 generates a bank select signal (Bank_sel) of the memory 2, and the clock controller controls the operations of the row and column address generating circuits 71 and 72. The bank selector 74 also generates a control signal such as a write enable clock signal (WE).

Fig. 16 (a) is a diagram showing the control timing of the memory controller

17 for the memory 12. The sections T0, T1, T3, ... are more finely detailed operation timings of a single time slot (hereinafter, a section of T0, T1, T2, ... is referred to as a bank access slot) corresponding to an operation for decoding and display shown in Figs. 11 and 12. The memory control such as the (a) display image data read command; (b) reference image data read command; (c) coded data read command; (d) memory refresh command; (e) coded data write command; and (f) decoded image data write command, all shown in Figs. 11 and 12, causes the alternate write or read access to the bank 0 and the bank 1 in the units of this bank access slot. Namely, in the same bank access slot, the row address is not changed. The row address is changed during the preceding period of a bank access slot when another bank is accessed. In this way, the waiting period accompanying the change of row address is obviously eliminated, so that the effective memory bandwidth is improved. The bank access slots in processes (a) to (f) may have different lengths.

The memory control operation for alternately accessing different banks is even performed with a section extending over different operation times, such as from (a) display image data

- 39 -

Read command to (b) reference image data read command; from (b) reference image data read command to (c) coded data read command; and further from (e) coded data write command to (f) coded image data write command, or from (f) decoded image data write command to (a) display image data read command. It is therefore necessary to control or select the arrangement of the coded data and the decoded image data in the memory so that access to memory 2 in each process starts at bank 0 and ends at bank 1, or vice versa.

× As for the (c) coded data read command and the (e) coded data write command, a single row of the bank 0 and the bank 1 in the memory 2 contains a total of 256 column addresses, and it is necessary to prevent the row address from changing in the bank access slot, so that the length of the bank access slot period is set to 8 words (8 column addresses). Each time slot of the (c) coded data read command and the (e) coded data write command can initiate access from the bank 0 and terminate access at the bank 1 if the bank access slots are even. Therefore, when coded data is written from the input buffer memory 11 into the memory 2, the device makes sure that there is coded data of at least 16 words in the input buffer memory 1. When encoded data is read out from the memory 2 and written into the decoded data buffer memory 12, the device confirms in advance that a free area of at least 16 words exists in the decoded data buffer memory 12.

Fig. 17 is a diagram showing the situation of block division of a one-frame image. In this example, it is assumed that the luminance signal in an image of 720 pixels x 480 lines, and the color signal (expressed by Cb or Cr in the drawing) has a pixel density,

which accounts for half of the luminance signal (expressed by Y in the drawing) in the horizontal and vertical directions. As shown in the drawing, the macroblock is defined as a set of 6 blocks, with the luminance signal blocks almost equal in area to the color signal blocks. Using this macroblock, the one-frame image consists of 45 macroblocks in the horizontal direction and 30 macroblocks in the vertical direction, for a total of 45 × 30 = 1350 macroblocks.

Î Fig. 18 is a diagram showing the mapping of the image data in units of the aforementioned macroblock in the image storage area of the memory 2. As shown in the drawing, the image data of a single macroblock is stored in accordance with the position of a single row address of the single bank when the image signals

is and the color signals are mapped in different banks. Furthermore, the image data corresponding to the macroblock adjacent to the horizontal position on the image is stored in another bank.

Based on the above arrangement, with respect to the memory access of the (f) decoded image write command, the bank 0 and the bank 1 can always be accessed alternately when a bank access slot belongs to the image data of the luminance signal in the macroblock and a bank access slot belongs to the image data of the color signal in the macroblock; and the decoded image data is written in the order of luminance signal first and color signal next in the case of an even-numbered macroblock; and the image data is written in the order of luminance signal first and color signal next in the case of an odd-numbered macroblock.

- 41 -

Fig. 19 is a diagram showing the existing position of the reference image data to be read out by the (b) reference image data read command according to the macroblock. The range of the reference image data to be read out generally extends over four macroblocks as shown in the drawing. In view of the fact that the banks in which the image data of the luminance signal and the image data of the color signal are stored differ from each other with respect to the macroblocks and that the banks for the adjacent macroblocks also differ from each other, the banks can be alternatively accessed. Based on the method shown in Fig.

Namely, in the example of image data mapping shown in Fig. 18, it is decided whether the macroblock number at the top left to which a number i is assigned in Fig. 19 is odd or even, and the bank in which the image data of the luminance signal of the macroblock e is stored is checked. The image data of the color signal for the same macroblock is stored in a different bank from that for the luminance signal. For example, if i is even, the luminance signal of the macroblock i is stored in bank 0, while the color signal is stored in bank 1. When the image data is read out as reference image data in the order: brightness signal of macroblock i -> color signal of macroblock i -> color signal of macroblock i + 1 -> brightness signal of macroblock i + 1 -> brightness signal of macroblock i + 46 -» color signal of macroblock i + 46 -> color signal of macroblock i + 45 -* brightness signal of macroblock i + 45, or when it is read out in the order: brightness signal of macroblock i -» brightness signal of macroblock i + 1 -* brightness signal of macroblock i + 46 -» brightness signal of macroblock i + 45 -* color signal of macroblock i + 45 -> color signal of macroblock i + 46 -> color signal of macroblock i + 1 -* color signal of macroblock i, it is possible to start the access from bank 0 and end the access at bank 1.

Even for the (a) display image data read command, it is possible to access the bank O and the bank 1 alternately due to the fact that it is necessary to display the luminance signals and the color signals together, namely, by accessing in the order of luminance signal and then color signal at every even-numbered macroblock, and in the order of color signal and then luminance signal at every odd-numbered macroblock.

In the above-mentioned memory control system, as time slots for the (a) display image data read command, the (b) reference image data read command, the (c) coded data read command, the (e) coded data write command, and the (f) decoded image data write command, fixed time slots are allocated regardless of the content of the coded data. Each time slot gives access to bank 0 and bank 1 alternately, and a certain time slot can perform an access operation at the predetermined timing regardless of the operation of the just previous time slot. In this way, the arbitration function for deciding the data bus access right between the write address generator unit 174 for coded data, the read address generator unit 175 for coded data, the read address generator unit 176 for motion-compensated reference image data, the write address generator unit 177 for decoded image data, and the read address generator unit 178 for display image data can be avoided, and each circuit can be simplified significantly.

In explaining the embodiments of the present invention, the correspondence of the coded data encoded in frames by combining intra-picture coding, inter-picture coding using motion compensation and picture interpolation coding is used. However, the present invention can also be applied to coded data encoded by, for example, only intra-picture coding.

- 43 -

coding. Data can be coded not only in picture units but also in field units, and even if the two coded data exist simultaneously, the present invention can be applied. Moreover, as the coding system, instead of the system using the DCT described in the embodiments, another system for processing in block units of the predetermined size, for example, a system using the vector quantization, can be used.

&iacgr;&ogr; As for the coded data input to the image decoding device, it can accommodate not only the case where coded data is continuously input at a fixed bit rate, but also the case where the coded data is input at a variable bit rate or in a burst. In any of these cases, the

is the present invention can be applied.

Of course, the present invention can also be applied to an apparatus for decoding digital video signals corresponding to the HDTV which is different in resolution from the conventional television system. The image decoding apparatus can also be used for switching the processing according to a plurality of video signals of the conventional television system 525/60, the conventional television system 625/50 and the HDTV system. Further, the present invention can also be applied to an image decoding apparatus which enables not only the display output of the interlaced scanning but also the display output of the sequential scanning.

As for the image decoding apparatus, in case it is so constructed as to carry out both coding and decoding as well as display and output of decoded image data, the invention can

- 44 -

also be applied in an image decoding circuit incorporated in the image decoding device.

Claims (16)

Expectations 1. Device for decoding image data, comprising: Decoding devices (13, 14, 15) for obtaining decoded image data by decoding coded data of a video signal which is coded &iacgr;&ogr; and by an intra-frame coding (I-frame) using the same frame ends, an inter-frame coding (P-frame) referring to the previous frame, and a frame interpolation coding (B-frame) referring to the previous frame and the following frame, in frame units consisting of is two fields and block units, consisting of a multitude of pixels in the frame; Storage devices (2) for storing the decoded image data; and display devices (16) for reading out the decoded image data stored in the storage devices (2) in field units on the basis of a display synchronization signal and for obtaining display image data in an interlaced manner,
marked in that the memory devices (2) comprise a first frame memory (FM1), a second frame memory (FM2) and a third frame memory (FM3) which are jointly connected to a data bus and to an address bus, each frame memory (FM1, FM2, FM3) being intended for storing decoded image data obtained by the decoding devices (13, 14, 15);
by timing means (17, 18) for reading the decoded image data from the first and second frame memories as a reference frame for the B frame; and for controlling the time difference between a time of writing decoded image data into the third frame memory and a timing of reading decoded image data from the third frame memory for display by the display devices to achieve simultaneous reading/writing of the third frame memory in frame units; by means (166) for delaying the start of a display of a frame of one of the I-frame and the P-frame from a start of decoding encoded data for the frame to be displayed by at least 1.5 frames; and &iacgr;&ogr; by means (166) for delaying the start of a display of a B- frame in the third frame from a start of decoding of coded data for the frame to be displayed by at least 0.5 frames. 2. An apparatus for decoding image data according to claim 1, wherein the storage devices store decoded image data of a plurality of scanning systems, the plurality of scanning systems comprising a scanning system with a frame rate of 30 Hz and 525 scanning lines and a scanning system with a frame rate of 25 Hz and 625 scanning lines, and wherein the storage devices (2) have a storage capacity of at most 16,777,216 bits for storing encoded data before decoding the same. 3. An apparatus for decoding image data according to claim 1 for decoding two kinds of coded data, namely first coded data including the I frame, the P frame and the B frame and second coded data including the I frame and the P frame, wherein the smallest delay time from the start of decoding the coded data of one frame to the start of display is 1.5 frames when the frame is divided by the I frame, the P frame and the B frame. frame or by the P frame in the first coded data, and which is 0.5 frames for the B frame in the first coded data and 0.5 frames for the second coded data.
s 4. An apparatus for decoding image data according to claim 1, wherein the decoding means (13, 14, 15) decode coded data of a single frame during the one-frame period in synchronism with the display synchronization signal. 5. An apparatus for decoding image data according to claim 1, wherein the decoding devices (13, 14, 15) comprise: means for setting a decoding hold period at the time at which a frame of coded data to be decoded is started; is and at the the timing means (17, 18) comprise: means for starting reading of the decoded image a certain time after starting reading for the display which begins in the hold period; and Means for terminating writing of the decoded image a certain time before terminating writing for the display which ends in the hold period. 6. An apparatus for decoding image data according to claim 1, wherein the timing control means (17, 18) comprises means for setting the time slots in synchronization with the display synchronization signal for writing and reading decoded image data at least to and from the third frame memory. 7. An apparatus for decoding image data according to claim 6, wherein the time slots are set so that a plurality of blocks can be decoded during one horizontal scanning period of the display synchronization signal. 8. An apparatus for decoding image data according to claim 7, wherein the storage devices store decoded image data of a plurality of scanning systems, the plurality of scanning systems comprising a scanning system having a frame frequency of 30 Hz and 125 scanning lines and a scanning system having a frame frequency of 25 Hz and 625 scanning lines, and wherein the time slots are set so that the number of block units that can be decoded during one horizontal scanning period of the display synchronization signal is the same for both scanning systems. 9. An apparatus for decoding image data according to claim 6, wherein the storage devices (2) comprise a buffer memory for temporarily storing the coded data, and wherein the time control means (17, 18) further comprise: Coded data writing devices; coded data reading devices; decoded image data writing devices; reference image data reading devices for reading decoded image data as reference image data; and display image data reading devices;
and wherein the time slots are set according to access by each of the coded data writing devices, the coded data reading devices, the decoded image data writing devices, the reference image data reading devices, and the display image data reading devices. 10. Device for decoding image data according to claim 1, in which the storage devices comprise a first memory field and a second memory field, the first frame memory (FM1), the second frame memory (FM2) and the third frame memory (FM3) being mapped into the first memory field and the second memory field, and the timing control means (17, 18) comprise decoded image data writing devices for storing decoded image data, reference image data reading devices for reading the decoded image data as reference image data and display image data reading devices for obtaining &iacgr;&ogr; interlaced scanned display image data, each of the Decoded image data writing devices, reference image data reading devices and display image data reading devices supply a memory control signal to the second memory array in preparation for the next reading/writing, while the decoding devices is read from the first memory field or write data to the first memory field, and provide a memory control signal to prepare for the next read/write to the first memory field, while the decoding devices (13, 14, 15) read data from the second memory field or write data to the second memory field. 11. An apparatus for decoding image data according to claim 10, wherein the storage means (2) further comprise a buffer memory for temporarily storing the encoded data; and wherein the timing control means (17, 18) further comprise: Coded data writing devices and coded data reading devices; wherein the coded data writing devices and the coded data reading devices supply a memory control signal to the second memory array while reading data from the first memory array or writing data into the first memory array; and that they supply a memory control signal to the first memory array while while reading data from or writing data into the second memory array; and writing or reading encoded data into or from the memory devices by pairing the accesses to the first and second memory arrays. 12. An apparatus for decoding image data according to claim 10, wherein the decoded image data writing means, the reference image data reading means and the display image data reading means &iacgr;&ogr; during the operating period of the writing devices for decoded image data or the reference image data reading devices or the display image data reading devices which previously read data from or write data into the memory devices, supply a memory control signal to the first memory array to start reading data from the first memory array or writing data into the first memory array. 13. An image data decoding apparatus according to claim 11, wherein the decoded image data writing devices, the reference image data reading devices, the display image data reading devices, the coded data writing devices and the coded data reading devices supply a memory control signal to the first memory array during the operating period of the decoded image data writing devices, the reference image data reading devices, the display image data reading devices, the coded data writing devices or the coded data reading devices which previously read data from or write data into the memory devices to start reading data from or writing data into the first memory array. 14. An image data decoding apparatus according to claim 10, wherein the decoded image data writing means, the reference image data reading means and the display image data reading means read data from or write data into the storage means (2) according to time slots set in synchronization with a display synchronization signal. 15. An image data decoding apparatus according to claim 11, wherein the decoded image data writing means, the reference image data reading means, the display image data reading means, the coded data writing means and the coded data reading means read data from or write data into the storage means (2) in accordance with the time slots set in synchronization with a display synchronization signal. 16. An apparatus for decoding image data according to claim 10, wherein the storage devices (2) further comprise buffer devices; and wherein the timing control means (17, 18) further comprise: Means for storing control commands in the buffer devices; and Means for reading the control commands from the buffer devices for controlling the simultaneous reading/writing of the first and second memory arrays.