JPH0877345A - Image data processor - Google Patents

Abstract

PURPOSE: To execute a correct display by exactly operating a frame buffer for storing image data and a transforming circuit for transforming the form of the image data on the poststage while simplifying the configuration. CONSTITUTION: The image data processor for decoding the encoded digital image data is provided with a DRAM being the frame buffer for storing the decoded pixel data for the unit of a frame, ring buffer (Y-FIFO) 31 for storing a luminance signal Y in the image data read from this DRAM, line buffers 32 and 33 for storing color difference signals Cb and Cr in the image data read from the DRAM, and a YCbCr/RGB converting part 20 for converting the signals to RGB (three primary color) signals while using the stored contents of these ring buffer 31 and line buffers 32 and 33.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image data processing device for decoding image data read from a recording medium by a multimedia device and outputting the decoded image data to a necessary external device.

[0002]

2. Description of the Related Art Recently, a plurality of types of digital data, for example, moving image data and audio data, are combined into a CD-ROM.
A multimedia personal information device that records the data on a recording medium such as the above and reads and outputs these data as necessary is planned and developed. When displaying moving image data read from a recording medium with this type of device, the bit string of the coded moving image data to be read is decoded and the decoded moving image data is displayed on one screen (hereinafter referred to as "frame"). Stored in an external memory (usually composed of DRAM; hereinafter referred to as “frame buffer”) for each frame, rearranged each frame for display in the original order, and then output to a device for display. It uses a decoder circuit.

MPEG (Moving Picture coding image), which is an international standardization organization for multimedia mainly relating to coded display of moving images and related audio stored in a digital recording medium.
Experts Group) defines the structure of image data as follows in a standard generally called MPEG1.

That is, as shown in FIG. 5, a macroblock, which is a unit for transferring image data, is a luminance signal (Y) 4 block (Y0 to Y3) and a color difference signal (Cb,
Cr) Two blocks, one block each, for a total of 6 blocks.

Each block consists of 8 × 8 pixels. Since the color difference signals Cb and Cr are both sub-sampled, four blocks of the luminance signal Y and one block of the color difference signals Cb and Cr have the same size on the screen.

In the moving picture coding used in MPEG,
MC (Motion Compensation) for reducing temporal information volume
: DCT (Dis-compensation) and spatial information reduction
creteCosine Transform (discrete cosine transform) is a basic element, and MC performs processing in units of 16 × 16 pixel macroblocks and DCT in units of 8 × 8 pixel blocks.

[0007] Further, macroblocks are Y0, Y1, Y2, Y3, Cb, Cr in block units in the frame buffer.
Written in order. FIG. 6 shows a frame buffer FB in which image data is sequentially stored in macroblock units for one frame.
Since the ratio of the luminance signal Y and the color difference signals Cb and Cr in the macroblock is 4: 1: 1, the ratio of the data amount stored in one frame is the same.

After the writing is performed in this manner, the corresponding blocks of the luminance signal Y and the color difference signals Cb and Cr are read out at appropriate timings, converted into RGB primary color signals as needed, and then, are transferred to the video unit in the subsequent stage. It will be output.

[0009]

In the decoder circuit as described above, the image data and the difference data of the previous or previous frames, which are called inter-frame prediction, are used, and they are positioned after or in between. A technique for decoding and reproducing frame image data is used.

When this inter-frame prediction is carried out, there is no problem if a frame buffer for storing the image data of the previous or previous frame and a frame buffer for storing the image data of the frame located after or in between are provided. Although it does not occur, it is difficult to use a large number of frame buffers in terms of circuit scale, etc.
In one frame buffer, the stored image data is read and displayed, and at the same time, the image data of a new frame is written in the same buffer.

In this case, new image data may be overwritten even though the image data has not yet been read out for display, resulting in incorrect image data being displayed. .

Further, a conversion circuit, which is located at the subsequent stage of the frame buffer, converts the image data consisting of the luminance signal and the color difference signal read from the frame buffer into RGB primary color signals and outputs the RGB primary color signals as needed. In this case, a RG is provided by including a buffer memory that appropriately reads out and stores the luminance signal and the color difference signal, and executes an operation using the luminance signal and the color difference signal stored in the buffer memory.
Although each of the B primary color signals is generated, the time and bus width required for transfer between the frame buffer and the conversion circuit and the buffer memory capacity of the conversion circuit increase, which imposes a heavy burden on the decoder circuit. Has become.

The present invention has been made in view of the above situation, and an object thereof is to provide a frame buffer for storing image data and a conversion circuit for converting the form of the image data in the subsequent stage with a simple structure. It is an object of the present invention to provide an image data processing device capable of operating correctly and performing correct display.

[0014]

That is, according to the present invention, in an image data processing device for decoding coded digital image data, a frame buffer for storing the decoded pixel data in frame units, and a frame buffer read out from the frame buffer. Using a FIFO (first-in first-out) memory for storing the luminance signal in the image data, a line buffer for storing the color difference signal in the image data read from the frame buffer, and the contents stored in the FIFO memory and the line buffer. And a conversion means for converting into RGB (three primary color) signals.

[0015]

With the above structure, the conversion circuit for converting the form of the image data from the luminance signal and the color difference signal to the RGB signal in the subsequent stage of the frame buffer has a simple structure, and between the frame buffer and the conversion circuit. The load of data transfer can be reduced.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the present invention is applied to an image decoder for moving images conforming to the MPEG1 standard will be described below with reference to the drawings. In addition, here MPEG
The hierarchical structure of the image data will be described. That is, the hierarchical structure has sequence header, GO
P, picture, slice, macroblock and block.

The sequence header is a quantization table,
This is the highest layer in which basic MPEG parameters such as picture rate can be set. GOP (Group of Pictures)
Is a group of a plurality of pictures including at least one I (Inter flame) picture and serves as a unit of random access.

A picture is one picture (screen), and is an I picture (intra-frame (inter flame) coded screen) that is independently coded without performing temporal prediction, and is mainly an error due to forward prediction. P picture (inter-frame (Predictive) predictive coding screen) in which a signal is coded, and B picture (Bidirectional predictive coding screen) in which an error signal mainly due to forward prediction and backward prediction is coded There are types of pictures. The I picture can be an access point for random access, and the P picture can be an error signal due to the reference point of the next P picture. B pictures do not serve as reference points for prediction to other B pictures or P pictures.

A slice is a picture obtained by dividing one picture into several pieces in the row scanning direction.
Says a line. The macroblock is 16x1 as described above.
It is composed of 6 pixels and serves as a calculation unit of MC, and is composed of a total of 6 blocks of 4 blocks of luminance signal (Y) and 2 blocks of color difference signal (Cb, Cr). Since the color difference signal is sub-sampled, 4 blocks of luminance signal and 1 color difference signal
The blocks are the same size on the screen. Blocks
As described above, it is composed of a total of 64 pixels of 8 × 8 pixels, and serves as a DCT calculation unit.

FIG. 1 exemplifies a circuit configuration of an image decoder for decoding moving image data. Moving image data read from an MPEG compliant recording medium and separated from audio data by a system decoder (not shown) is stored in a host bus. 11
Is input to the host interface 13 in the image decoder 12 via.

The image decoder 12 is for receiving and decoding a bit string of coded image data sent from the system decoder and reproducing the image data.

The bit string of coded image data input to the host interface 13 is buffered by the DRAM interface 14 and then sent to a variable length coding decoder (shown as "VLC decoder" in the figure) 15.

The variable length coding decoder 15 decodes the bit string of the image data sent via the DRAM interface 14 and separates it into various data. For example, the variable-length coding decoder 15 decodes a picture type, a macroblock type, a motion vector (a vector used for inter-frame prediction, which represents a motion direction and size of an image of each macroblock), and the like. A bit string of data is decoded for each block which is a basic decoding unit to obtain a quantized DCT coefficient for I, P and B pictures, and the quantized DCT coefficient is sent to the inverse DCT circuit 16.

The inverse DCT circuit 16 is a variable length coding decoder.
Inverse quantization and inverse D for quantized DCT coefficients from 15
The CT processing is performed to obtain pixel value data, which is sent to the motion compensation unit 17.

The motion compensator 17 adds the motion vector to the pixel value data from the inverse DCT circuit 16 to reproduce the image data, and the obtained image data is the above DRAM.
It is sent to the interface 14.

The DRAM interface 14 is a motion compensation section.
The image data sent from 17 is stored via the DRAM bus 18 in the DRAM 19 which appropriately constitutes a frame buffer in frame units, and is read out to be read as YCbCr / RGB.
It is sent to the conversion unit 20, and has priority registers 14a and D for internally defining the priority of reading from the DRAM 19.
A write protection register 14b that defines the write timing to the RAM 19 is provided.

The DRAM 19 has a capacity of, for example, 3 frames, and stores, for example, image data of I and P pictures for 2 frames and image data of B pictures for the remaining 1 frame.

The YCbCr / RGB converter 20 is a DRAM
The image data read from 19 is stored in an internal buffer memory, which will be described later, and then the luminance signal Y
And the image data composed of the color difference signals Cb and Cr are converted into image data composed of RGB primary color signals by using calculation. The image data that has been converted as necessary is
It is sent back to the host interface 13 and then the host bus
It is supplied to the system decoder via 11 or is output to a video interface 21 outside the image decoder 12 and is displayed via a video bus 22 on a display device (not shown) such as a CRT.

However, the internal controller 23 integrally controls the operation control in the image decoder 12. Next, FIG. 2 shows a structure of a buffer memory provided in the YCbCr / RGB conversion unit 20.
As shown in the figure, the YCbCr / RGB conversion unit 20 has a D
DR via RAM bus 18 and DRAM interface 14
A ring buffer (indicated as "Y-FIFO" in the figure) 31, which is a FIFO (first in, first out) memory for holding the luminance signal Y read from the AM 19, holds the color difference signals Cb and Cr which are also read. Line buffer for
32 and 33 are provided. In the figure, "IP" and "OP" in the ring buffer 31 are the positions of the input pointer and the output pointer, respectively, and "W" in the line buffers 32 and 33.
The positions of the write pointer and the read pointer are indicated by P "and" RP ".

Although not shown in the figure, the ring buffer 31 is provided with a status bit indicating that the data input holding status is empty, full, or almost empty. And

In the configuration of the image decoder 12 as described above, its operation is as follows. First, the order of bit strings of image data input to the image decoder 12 will be described. FIG. 3 shows GO defined in the MPEG standard.
The three types of pictures that make up P, namely I, P,
It shows the mutual relationship of B pictures, for example, "I
"I" of "0" is an I picture, and "0" is GO
It indicates that the order in the original image is "0" in the series of pictures in P.

In the figure, as shown by the arrows,
Only the I picture exists independently, the P picture depends on the preceding I or P picture, and the B picture depends on the preceding or following I or P picture.

Therefore, the order of the image data when it is read from a recording medium (not shown) and input to the image decoder 12 through the system decoder is different from the original order of pictures shown in FIG. Depends on
Alternatively, it is necessary to arrange the P picture prior to the B picture.

FIG. 4A shows the order of the original image data as in FIG.
(2) shows the order of coded image data written in the recording medium, read out, and sent to the image decoder 12 via the system decoder.

The bit string of the coded image data thus input to the image decoder 12 is sequentially separated into various data by the variable length coding decoder 15 via the host interface 13 and the DRAM interface 14 and quantized D
After the CT coefficient is extracted, the inverse DCT circuit 16 performs inverse quantization and inverse DCT processing to obtain pixel value data, and the motion compensation unit 17 adds motion vectors to create original image data, The data is sequentially stored in the DRAM 19 in macro block units via the DRAM interface 14.

FIG. 4 (3) is decoded in this way, and DR
This shows the order of image data written in AM 19 in macroblock units. The DRAM 19 has a capacity of, for example, 3 frames as described above, and stores, for example, image data of I pictures and P pictures for 2 frames and image data of B pictures for the remaining 1 frame.

Therefore, for example, the image data of "I0" and "P3" is decoded and written in the first and second frame portions of the DRAM 19, and the subsequent image data of "B1" is decoded and stored in the DRAM 19. At the time when the writing is started in the third frame portion, "I0" written in the first frame portion of the DRAM 19 is simultaneously delayed by two frames from the decoding as shown in FIG. 4 (4). The image data of is started to be read.

The read image data of "I0" is DRA.
YCbCr / RGB converter via M interface 14
After being converted into RGB primary color signals at 20 in accordance with need, they are output from the video interface 21 to the outside via the video bus 22.

After that, the decoding of the image data of "B1" and the writing to the DRAM 19 are completed, and at the same time, the DRAM 19
When the reading and display of the image data of “I0” from the device is completed, the decoding and DR of the image data of “B2” are performed next.
Writing to the AM 19 is started, and at the same time, reading and display of image data of "B1" from the DRAM 19 is started in the display order as shown in FIG.

At this time, "B1" image data and "B1"
The image data of "2" will share the third frame portion of the DRAM 19. Therefore, if the image data of "B2" is decoded and written and then the image data of "B1" is read and provided for display, the image data of "B2" is erroneously read as the image data of "B1". As a result, accurate display cannot be performed.

Therefore, when writing and reading are performed by sharing one frame portion of the DRAM 19, the DRAM interface 14 sets a flag in the write protection register 14b to read and display the stored image data. Then, the decoded image data is overwritten and written.

Next, the image data read from the DRAM 19 is transferred to the YCbCr / RG via the DRAM interface 14.
An operation when RGB conversion is performed by the B conversion unit 20 as necessary and output from the video interface 21 or the host interface 13 to the outside will be described.

In the DRAM interface 14, the DRAM
The image data of the luminance signal Y and the color difference signals Cb and Cr corresponding to the luminance signal Y stored in the corresponding frame portion of 19 are read out and transferred to the YCbCr / RGB conversion unit 20. The YCbCr / RGB conversion unit 20 Then, the image data of the luminance signal Y is input to the point position according to the input pointer IP of the ring buffer 31, and at the same time, sequentially output from the point position according to the output pointer OP. Also, the color difference signal C
The image data of b and Cr are line buffer 32,
It is written in the point position according to the write pointer WP 33, and the written contents are sequentially read from the point position according to the read RP. The image data of the luminance signal Y output from the ring buffer 31 Line buffer 32, 33
The RGB primary color signals are obtained by calculating the corresponding read-out image data of the color difference signals Cb and Cr as necessary.

At this time, the image data of the color difference signals Cb and Cr has a data amount of 1 in both the vertical and horizontal directions by sub-sampling with respect to the image data of the luminance signal Y as shown in FIGS. 5 and 6.
Since it is compressed to / 2, two horizontal lines of the image data horizontal line of the luminance signal Y are sequentially read and transferred to the ring buffer 31, while one horizontal line of the image data horizontal lines of the corresponding color difference signals Cb and Cr are transferred. Read and line buffer
It will be transferred to 32, 33.

Therefore, the YCbCr / RGB conversion unit 20 uses the image data of the luminance signal Y for two horizontal lines which are sequentially stored in the ring buffer 31, and uses the RGB primary color signals adapted to the size of the image data to be displayed. When performing the conversion calculation into, the color difference signals Cb and Cr stored in the line buffers 32 and 33 are respectively twice (3 × 2 if the calculation for obtaining the R, G and B signals is counted as 3 times). 6 in total
It will be used for calculation.

The state of the ring buffer 31 is taken into consideration during the conversion operation into the RGB primary color signals.
As described above, the ring buffer 31 is usually set with a status bit indicating that the data input holding status is empty, full, or almost empty. The conversion operation is controlled, and when the ring buffer 31 is empty, it indicates that the output processing of the image data of the line has already been completed.

Then, the data is read from the DRAM 19,
YCbCr / RGB via DRAM interface 14
The order of the image data of the color difference signals Cb and Cr corresponding to the image data of the luminance signal Y stored in the conversion unit 20 is DRAM.
It is determined based on the contents of the flag set in the priority register 14a in the interface 14.

The contents of the priority register 14a are transferred from the YCbCr / RGB conversion section 20 to the video interface 2
1. Set according to the size of one horizontal line and the size of the ring buffer 31 of image data externally output for display via the video bus 22.

Here, the contents of the priority register 14a are
The color difference signal C corresponding to the image data of the luminance signal Y
When it is set to transfer the image data of b and Cr, the transfer of the image data of the color difference signals Cb and Cr must be already completed when the transfer of the image data of the luminance signal Y is completed.

However, the brightness signal Y for one horizontal line of the image data displayed by the size of the ring buffer 31 is displayed.
If it is larger than the data amount of, the read image data of the luminance signal Y for one horizontal line can be stored in the ring buffer 31 as it is. At this time, if the image data of the luminance signal Y is read out before the image data of the corresponding color difference signals Cb and Cr and transferred to the YCbCr / RGB conversion section 20, the overwriting condition becomes effective, and therefore the color difference The image data of the signals Cb and Cr are all line buffers 32 and 33.
The rewrite operation is performed before it is read out more,
Data that has not yet been output may be lost and display may be hindered.

When the size of the ring buffer 31 is smaller than the data amount of the luminance signal Y for one horizontal line of the image data to be displayed, one line of image data of the Y signal is transferred to the ring buffer 31 all at once. Cannot be stored. As described above, the image data of the color difference signals Cb and Cr for one horizontal line corresponds to the image data of the luminance signal Y for two horizontal lines.
In the conversion into the primary color signal of the above, it is read twice and used for the calculation with the image data of the corresponding luminance signal Y.
Therefore, when the image data for the last two horizontal lines of the one frame is output and provided for display, this
Since the image data of the luminance signal Y for each line cannot be transferred to and stored in the ring buffer 31 at one time as described above, the color difference signals Cb and C are transferred last.
It must be the image data of the luminance signal Y, not the image data of r.

Considering each of the above points, the image data of the luminance signal Y and the corresponding color difference signals Cb and Cr from the DRAM 19 to the YCbCr / RGB conversion section 20 and the YCbCr / RGB conversion section 20 to the outside are transferred. The order of outputting the image data is such that the image data of the luminance signal Y is transferred to the color difference signal Cb only when the size of the ring buffer 31 is smaller than the data amount of the image data of the luminance signal Y for one horizontal line of the displayed image data. , Cr, the image data of the color difference signals Cb, Cr is transferred before the image data of the luminance signal Y. The priority order is controlled by the DRAM interface 14 based on the content of the priority order register 14a as described above, and is output from the YCbCr / RGB conversion section 20 to the external display device via the video interface 21 and the video bus 22. Whether in the case or sent back to the system decoder via the host interface 13, accurate display operations can be performed without partial loss of data.

By setting such a transfer order,
Even if the data amount of the image data of the luminance signal Y for one horizontal line of the DRAM 19 and the size of the ring buffer 31 are both large, there is no problem and D
Transfer from RAM 19 to YCbCr / RGB converter 20,
The data amount of the color difference signals Cb, Cr and the number of times of transfer stored in the YCbCr / RGB conversion unit 20 can be minimized in accordance with the ratio of the data amount in the macro block of the luminance signal Y and the color difference signals Cb, Cr.

Further, as described above, the DRAM 19 is provided in accordance with the contents of the priority register 14a of the DRAM interface 14.
To YCbCr / RGB conversion unit 20 to YCbCr / RG
Rather than controlling the transfer sequence of the image data of the luminance signal Y and the corresponding color difference signals Cb and Cr from the B conversion unit 20 to the outside, the write protection register 14b is used and the luminance signal Y is based on the contents thereof. Image data of the corresponding color difference signal C
YCbCr / RGB always precedes the image data of b and Cr
The conversion unit 20 may transfer and output the data to the outside.

In this case, the size of the ring buffer 31 represents the luminance signal Y for one horizontal line of the image data to be displayed.
When the amount of image data is larger than the color difference signal C
Until the image data of r is completely stored in the line buffer 33 from the DRAM 19, the corresponding image data of the luminance signal Y is not stored in the ring buffer 31, and conversely, the size of the ring buffer 31 displays the image data of the displayed image data. When the image data of the luminance signal Y for one horizontal line is smaller than the data amount of the image data of the luminance signal Y, the corresponding color difference signals Cb, Cr until the image data of the luminance signal Y is completely stored in the ring buffer 31.
The image data of is not stored in the line buffers 32 and 33.

Therefore, when the size of the ring buffer 31 is larger than the data amount of the image data of the luminance signal Y for one horizontal line of the image data to be displayed, the color difference signal C
A flag is set in the write protection register 14b as a signal for performing write protection for the line buffer 33 of r, and conversely, the size of the ring buffer 31 displays the image data of the luminance signal Y for one horizontal line of the displayed image data. When it is smaller than the data amount, the write protection register 14b is also used as a signal for protecting the write of the luminance signal Y to the ring buffer 31.
Is set to the flag.

As described above, the image data of the luminance signal Y and the color difference signal Cr stored in the ring buffer 31 and the line buffer 33 are not rewritten by overwriting before being output, respectively. It will be output.

In the above embodiment, the case where the ratio of the data amount of the luminance signal Y to the color difference signal Cb and the color difference signal Cr is 4: 1: 1 according to the MPEG1 standard has been described.
The present invention is not limited to such a ratio. In short, the ratio of the data amount of each signal is fixed in advance, and at least one of the signals is data-compressed compared to the other signals. If it has been used, it is similarly applicable.

[0059]

As described above in detail, according to the present invention, the conversion circuit for converting the form of the image data from the luminance signal and the color difference signal into the RGB signal in the subsequent stage of the frame buffer has a simple structure, and the frame buffer and the frame buffer are provided. It is possible to provide an image data processing device capable of executing correct display while reducing the load of data transfer between conversion circuits.

[Brief description of drawings]

FIG. 1 is a block diagram showing a circuit configuration according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a circuit configuration in a YCbCr / RGB conversion unit of FIG.

FIG. 3 is a diagram exemplifying a dependency relationship of image data according to the embodiment.

FIG. 4 is a diagram exemplifying a processing order of image data according to the embodiment.

FIG. 5 is a diagram showing the structure of a macroblock in MPEG.

FIG. 6 is a diagram showing a storage configuration of a frame buffer in MPEG.

[Description of Codes] 11 ... Host bus, 12 ... Image decoder, 13 ... Host interface, 14 ... DRAM interface, 14a ... Priority register, 14b ... Write protection register, 15 ... Variable length coding decoder, 16 ... Inverse DCT Circuit, 17 ... Motion compensation unit, 18 ...
DRAM bus, 19 ... DRAM, 20 ... YCbCr / RGB
Converter, 21 ... Video interface, 22 ... Video bus, 31 ...
Ring buffer (Y-FIFO), 32, 33 ... Line buffer.

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification number Office reference number FI technical display location H04N 9/808 G06F 15/66 330 B H04N 9/80 B

Claims (4)

[Claims] 1. An image data processing device for decoding digital image data composed of coded luminance signal and color difference signal, and a frame storage means for storing the decoded pixel data in frame units, and a frame storage means for reading the decoded pixel data. A first-in first-out (FIFO) memory for storing a luminance signal in the image data; a line buffer for storing color difference signals in the image data read from the frame storage means; and a memory content of the FIFO memory and the line buffer. An image data processing device, comprising: a conversion unit that converts the signals into RGB (three primary color) signals. 2. The contents of the frame storage means are stored in the FI.
Holding means for holding priority information when transferring to the FO memory and the line buffer is further provided, and the transfer of the luminance signal from the frame storing means to the FIFO memory is transferred from the frame storing means to the FIFO memory according to the contents held by the holding means. 2. The image data processing apparatus according to claim 1, wherein whether or not to execute the color difference signal is determined prior to being transferred to the line buffer. 3. A holding means for storing definition information for protecting the image data stored in the frame storage means is further provided, and the image data is written into the frame storage means according to the contents held by the holding means. 2. The image data processing apparatus according to claim 1, wherein it is determined whether or not to transfer the image data already stored to the FIFO memory or the line buffer in advance. 4. The contents of the frame storage means are stored in the FI.
A holding means for storing definition information for protecting the contents of the FIFO memory or the line buffer when transferring to the FO memory and the line buffer is further provided, and the frame memory means stores the definition information in the FIFO memory or the storage means according to the contents held by the holding means. The method according to claim 1, wherein it is determined whether or not the transfer to the line buffer is executed.
The image data processing device described.