JPH09261630A - Compressed image data processing method and device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress deterioration in a reproduced image even when a bit stream with a different image size is inputted by an MPEG decoder. SOLUTION: An input bit stream is stored in a DRAM 16 and read and decoded by a variable length decoding section 20 and an inverse quantization section 24 or the like and stored in the DRAM 16. The DRAM 16 is divided into a buffer area to store a bit stream and a frame area to store decoded image data and the size of the buffer area and the size of the frame area are changed depending on the size of image. In the case of a bit stream whose image size differs, after header data required for reproducing the image are decoded, the buffer is once cleared, and then the remaining succeeding bit stream is stored and decoding is restarted from an I picture.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for processing compressed image data, and more particularly to a method and an apparatus for decoding bit stream data compliant with the MPEG standard and outputting the data in the order of display screens.

[0002]

2. Description of the Related Art In MPEG, which is a moving picture coding standard, compressed data of an I picture consisting only of intra-frame predictive-coded macro blocks (intra macro blocks),
Compressed data of a P picture in which intra macroblocks and forward-direction interframe predictive-coded macroblocks coexist, and intra macroblocks and forward-direction interframe predictive-coded macroblocks and backward predictive-coded macroblocks Compressed data of a B picture, in which macroblocks that are both bidirectionally and predictively coded are mixed, are arranged such that the pictures referred to when decoding each predictively coded macroblock in the forward direction, the backward direction, or the bidirectional direction precede in time. It is considered as a bit stream. In the MPEG decoder, the image data of each picture is decoded from this bit stream and displayed in the display order. Since image data is encoded through DCT processing, quantization processing, and variable length encoding processing of image signals, decoding of compressed data from a storage medium or a broadcasting station is performed by variable length decoding processing,
Inverse quantization processing, inverse DCT processing, and motion compensation processing are performed.

As a specific processing method, an input bit stream is sequentially stored in, for example, a 16 Mbit DRAM, the bit stream is sequentially read and output to a variable length decoding unit, an inverse quantization unit, an inverse DCT unit, and In the case of a macroblock, the reference image and the motion compensating unit are subjected to addition processing and stored again in the DRAM. Therefore, since the bit stream and the image data are stored in the same DRAM, the address of the DRAM is divided into the buffer area for the bit stream and the frame area for the image data for writing / reading. That is, the same DRAM functions not only as a bitstream buffer memory but also as a frame memory.

FIG. 5 schematically shows the structure of the DRAM. FIG. 5A shows a configuration for processing an image A (computer image or NTSC image) having a certain image size, in which 12 Mbits of 16 Mbits are allocated to the frame memory and the remaining 4 Mbits are bitwise. Allocated to the stream buffer memory. On the other hand, FIG. 5B shows a configuration for processing an image B (another computer image or a PAL image) having a size larger than the image A. 14.25 Mbits out of 16 Mbits are allocated to the frame memory, and the remaining 1.75 Mbits are allocated to the bitstream buffer memory.

[0005]

When the image size of the input bit stream is constant, there is no problem if either of the configurations of FIG. 5A or 5B is used, but bit streams having different image sizes can be used. In case of continuous input, the memory allocation must be switched accordingly, so the capacity of the bitstream buffer changes under the influence of the image size, which causes an error in the reproduced image and makes it unsightly. there were.

FIG. 6 shows a change in the bit stream buffer memory when the bit stream of the image A having a relatively small image size is continuously changed to the bit stream of the image B having a relatively large image size. There is.
In FIG. 6A, 4 Mbits are allocated as a buffer memory for the bit stream (first stream) of the image A, and the first bit stream is stored therein. Then, the bit stream of the image B (second
When a stream) is input, it is similarly stored in a 4 Mbit buffer memory. However, since the size of this image B is larger than the size of the image A, D detected this
As shown in FIG. 6B, the RAM controller moves the upper limit position of the write pointer into the buffer memory (lower end in the figure) to reduce the capacity of the buffer memory from 4 Mbits to 1.75 Mbits (frame memory). Capacity is increased from 12 Mbits to 14.25 Mbits).

Therefore, among the second streams stored in the buffer memory while 4 Mbits are allocated to the buffer memory, the stream corresponding to the movement of the upper limit position of the write pointer cannot be read, and eventually this stream cannot be read. An error occurs because the part of the image cannot be reproduced.

Contrary to FIG. 6, FIG. 7 shows changes in the bit stream buffer memory when the bit stream of the image B having a large image size is continuously changed to the bit stream of the image A having a small image size. It is shown. In FIG. 7A, 1.75 Mbits are allocated as a buffer memory for the bit stream (first stream) of the image B, and the first bit stream is stored here. Then, when the bit stream (second stream) of the image A is continuously input, similarly, 1.7
It is stored in a 5 Mbit buffer memory. When the end of 1.75 Mbits is reached when the second stream is stored, the first stream that has already been read is overwritten and stored. However, since the size of the image A is smaller than the size of the image B, the DRAM controller that detects this has the following size, as shown in FIG.
Move the upper limit position of the write pointer into the frame memory (downward in the figure) to increase the buffer memory capacity to 1.75M.
Increase from 4 Mbits to 4 Mbits (reduce the frame memory capacity from 14.75 Mbits to 12 Mbits).

Therefore, of the second stream data stored in the buffer memory while the buffer memory is allocated for 1.75 Mbits, the data corresponding to the movement of the upper limit position of the write pointer is in an undefined state. When the end of 1.75 Mbits is reached, the address should be returned to the first address, but the end of 4 Mbits is read.) In this case, a reproduction error also occurs.

Of course, if the capacity of the DRAM is sufficiently large and the areas of the buffer memory and the frame memory can be fixed, such a problem does not occur, but the increase of the DRAM capacity directly causes the cost increase of the MPEG decoder.
It is not desirable to increase the capacity.

The present invention has been made in view of the above problems of the prior art, and an object thereof is to suppress deterioration of a reproduced image even when a bit stream having a different image size is input without increasing the capacity of the DRAM. It is an object of the present invention to provide a compressed image data processing method and apparatus capable of processing the same.

[0012]

In order to achieve the above object, the first aspect of the present invention is to decode each compressed image data of an I picture, a P picture and a B picture, each of which includes a predetermined macroblock, when decoding each macroblock. A compressed image data processing method that decodes image data of each picture from a bitstream in which reference macroblocks are arranged so as to precede in time and outputs the decoded image data in a display order. In this case, the decoding process of the compressed image data immediately after the image size change is interrupted.

A second invention is characterized in that, in the first invention, after the decoding process is interrupted, the decoding process is restarted from the first I-picture image data of a bit stream having a different image size.

In a third aspect of the invention, compressed image data of I picture, P picture and B picture each including a predetermined macroblock are arranged such that a reference macroblock at the time of decoding each macroblock precedes in time. Is a compressed image data processing method for sequentially storing the bit stream formed by the above in a memory, sequentially reading these, decoding the image data of each picture, storing in the memory, and outputting in the display order. When the input is continuously input, the bitstream already stored in the memory is cleared to interrupt the decoding process of the bitstream, and then the input bitstream is sequentially stored in the memory. .

A fourth aspect of the invention is characterized in that, in the third aspect of the invention, the clearing process is executed after decoding the data up to immediately before the picture data in the bit streams having different image sizes.

In a fifth aspect based on the fourth aspect, the bit stream storage area of the memory and the decoded image data storage are performed after decoding the data up to immediately before the picture data in the bit streams having different image sizes. It is characterized in that the allocation of areas is changed and then the clearing process is executed.

A sixth aspect of the invention is the computer system according to any one of the third, fourth, and fifth aspects of the invention, in which the first I-picture image data in the bitstream stored in the memory after execution of the clearing process is searched. Then, the decoding process after the I picture image data is restarted.

In a seventh aspect of the invention, the compressed image data of I picture, P picture and B picture each including a predetermined macroblock are arranged so that the reference macroblock at the time of decoding each macroblock precedes in time. A compressed image data processing device having a memory for storing decoded image data, the detecting means for detecting input of bit streams having different image sizes; It has a memory control means for clearing the bit stream stored in the memory after the change.

According to an eighth aspect of the present invention, in the seventh aspect, the memory control means stores the data in the memory after the decoded data up to immediately before the picture data in the bit stream is input to the variable length decoding unit. Characterized by clearing the stored bitstream.

A ninth invention is the search method according to any one of the seventh and eighth inventions, wherein the first I picture image data included in the bitstream stored in the memory after the memory is cleared is searched. It is characterized by having means.

Further, the tenth invention is a compressed image data processing method for temporarily storing a plurality of image data required at the time of decoding in the case of decoding encoded compressed image data, in the image size processing method. When different compressed image data are continuously input, allocation of storage areas of the plurality of image data in the memory is changed and set according to the image size, and the decoding process is interrupted. .

[0022]

Embodiments of the present invention will be described below with reference to the drawings. Image data encoded by MPEG has a hierarchical structure from a sequence layer to a block layer, and a predetermined number of I pictures, P pictures, and B pictures are grouped together to form a group (GOP layer). A picture (picture layer) is formed from slices formed by arbitrarily collecting macroblocks (slice layer). Each macroblock has 16 × 16 pixels and includes four 8 × 8 luminance signal blocks and two color difference signal blocks. In this embodiment, it is premised that such compressed data is input from a storage medium or a broadcasting station.

FIG. 1 is a block diagram showing the configuration of the MPEG decoder according to this embodiment. The bit stream is input to the CPU interface 10 and FIFO
It is output to the DRAM controller 14 via the memory 12. DRAM controller is CPU interface 1
The input bit stream is written to the DRAM 16 based on the write request signal from 0. DRAM 16 is 16M
It is a bit and functions as a bit stream buffer memory by allocating its address as in the conventional case, and also as a frame memory for storing decoded image data. The bit stream stored in the DRAM 16 is sequentially read by the DRAM controller 14 which receives a transfer request from the variable length decoding unit VLD 20, and is read via the FIFO memory 18 by the variable length decoding unit VL.
It is output to D20. The internal data bus is a 64-bit bus, and 1-line 8-pixel 64-bit data of a block forming image data is read from the DRAM 16 by designating a row address and a column address. The VLD 20 performs variable length decoding on the header data located at the beginning of the input bitstream, and outputs it as header information to the main control unit 22. This header information includes information such as the horizontal size and vertical size of the image, the number of macroblocks in the image, the image rate, and the bit rate. The main controller 22 uses the header information to determine the image of the bitstream. It detects the size and the presence of non-intra macroblocks. The VLD 20 also performs variable-length decoding on the image data following the header data and outputs the image data to the inverse quantization unit IQ24. In the IQ 24, the variable-length decoded image data is inversely quantized and output to the inverse DCT unit IDCT 26. When the inversely quantized image data is an intra macroblock, it is written as it is to the frame memory of the DRAM 16. However, when it is a non-intra macroblock, the DRAM controller 14 that has received a command from the motion compensation unit MC30 causes the DRAM 16 to operate. The reference macroblock stored in the memory is read and output to the motion compensation unit MC30 via the FIFO memory 28. In the MC 30, the image data that has been subjected to the inverse DCT processing and the reference macroblock are subjected to addition processing to perform motion compensation, and then output to the DRAM 16. The I picture, P picture, and B picture thus stored in the frame memory of the DRAM 16 are video-outputted in the display order by the display interface 32.

In such a configuration, the present embodiment is characterized in that the main control unit 22 outputs a clear request for the bit stream buffer of the DRAM 16 to the DRAM controller 14 to clear the bit stream buffer. That is, in the conventional technology,
When bit streams having different image sizes are continuously input, the main control unit 22 detects a change in the image size based on the decoded header information and simply allocates the memory of the DRAM 16 according to the image size. An error occurred in the reproduced image due to the switching, but in the present embodiment, the main control unit 2
In the case of detecting a change in the image size, 2 decodes up to the data immediately before the picture data necessary for image reproduction in the bitstream after the change, then clears the bitstream buffer once, and reads out in accordance with the change in the buffer capacity. The error is removed to suppress the deterioration of the reproduced image.

FIG. 2 shows the format of an input bit stream. The bit stream starts with a sequence header (Sequence Header), followed by a sequence extension (Sequence Extension).
After that, a GOP header, a picture header, and picture data follow. In this embodiment, after decoding the data up to immediately before the picture data indicated by the arrow in the drawing (for example, the slice header that is the beginning of the picture data), the buffer is cleared and the stored bitstream decoding process is prohibited. As a result, the image part that causes an error can be removed and deterioration of the reproduced image can be suppressed, and the bitstream stored in the buffer (the buffer whose capacity has changed according to the change in the image size) can be reproduced after clearing the buffer. Basic data needed to do this, such as the presence of intra / non-intra macroblocks and user data,
Since the GOP data and the like have already been decoded and supplied to the main control unit 22, the image data after the I picture can be immediately decoded.

3 and 4 show detailed processing flowcharts of this embodiment. First, the DRAM 16
The bitstream stored in the bitstream buffer is read, and the process proceeds to the search process for the header (S
101), it is determined whether a sequence header has been detected (S102). When the header data decoded by the VLD 20 is output to the main control unit 22, the sequence header is detected, and at this time, the main control unit 22 resets the bitstream buffer clear flag BSBUFCLR to 0 (S103). Then, it is determined whether the image size has changed (S10).
4). This determination is performed based on the image size data included in the header information from the VLD 20, and when the image size of the newly input bitstream is different from that of the previous bitstream (when it becomes larger than before or when it becomes smaller than before). (Including both), the flag BS
Set BUFCLR to 1 to prepare for clearing (S
105). On the other hand, when the image sizes are the same, the value of the flag BSBUFCLR is maintained at 0.

After the change of the image size is detected by the above process and the preparation for the buffer clear is made, the process shifts to the decoding process up to the data immediately before the picture data of the newly input bit stream. In this embodiment, the slice header, which is the head of the picture data, is used as the data immediately before the picture data. Specifically, the header search is performed again (S108), and the extension user data is detected (S109). After detecting the extension user data, the GOP header is detected next (S110). When the VOP 20 decodes the GOP header and outputs it to the main control unit 22, G
The OP header detection is completed, and the header search is started again (S111). This header search is for detecting extension user data in the GOP layer (S1
12) When this header data is detected, the process proceeds to the picture header detection of the picture layer (S113).

If the bitstream is from a broadcast medium, it may be possible that there is no GOP layer. If the GOP header is not detected in S110, the process immediately shifts to the picture header detection process. Then, if a picture header is detected in the picture header detection processing, the process moves to the next processing shown in FIG. 4, and if not detected, whether or not the header of the GOP layer in the upper layer has already been detected. Is confirmed (S114), and when the GOP header has already been detected, the process returns to S111 and the picture header detection process is repeated. On the other hand, G
If no OP header has been detected yet, S108.
Then, the process returns to the process No. and the GOP header detection process is repeated.

When the picture header is detected, the process shifts to the process of FIG. 4 and the header search is performed again (S11).
5), the picture coding extension is detected (S116), and a header search is further executed (S11).
7), the extension user data and slice header of the picture layer are detected (S118, S119).
By the above processing, sequence header → sequence extension → extension user data → (G
OP header) → (extension user data) →
The data is decoded in the order of picture header-> picture coding extension-> extension user data-> slice header. When the data up to the slice header of the input bit stream is decoded and supplied to the main control unit 22, it is next determined whether or not the flag BSBUFCLR is set to 1 (S120). If the image size has changed, this flag is set to 1 (see S104 and S105 in FIG. 3), so the main control unit 22 outputs a buffer clear request to the DRAM controller 14 to clear the buffer. Yes (S12
1). By this clearing process, among the bitstreams having different image sizes, the data already stored in the buffer of the DRAM 16 disappears, and the remaining bitstreams following the buffer newly allocated according to the image size are newly stored. It By this clearing process, the bitstream data before the different image size is also erased at the same time, but this bitstream has already been decoded, so there is no effect, and bitstreams with different image sizes do not affect the image playback. Since the decoding up to the required slice header data has already been completed, the image data for several frames is only lost and the subsequent image reproduction is possible.

However, when decoding a newly stored bitstream after clearing the buffer, P pictures and B pictures are reproduced based on I pictures, so it is necessary to search for I pictures first after clearing the buffer. . The process of S122 is for this reason, and after detecting the I picture, the process proceeds to the decoding process of the image data as in the conventional case (S123). On the other hand, when the image size has not changed, the flag BSBUFCLR remains 0, so the buffer is not cleared, and the I picture is GO.
Since it always comes first in P, an I picture search process is unnecessary (simply decoding from the beginning). After that, the process shifts to a process corresponding to the header detected by the header search (S124 to S128).

As described above, the main control unit 22 and the DRAM controller 14 (1) detect a change in the image size (2) decode the data up to immediately before the picture data of the bitstream in which the image size has changed (3) Clear the bitstream stored in the DRAM buffer to interrupt the decoding process of the image data. (4) Switch the DRAM buffer capacity according to the new image size to store the bitstream. (5) Stored bitstream. By executing each process of searching I picture data from the image data of No. 1 and reproducing the subsequent image data by using the already decoded header information, deterioration of the reproduced image can be achieved without increasing the memory capacity of the DRAM. Can be suppressed.

In order to make the difference between this embodiment and the prior art clearer, the processing of this embodiment will be described with reference to the examples of FIGS. 6 and 7 as follows. That is, in the example of FIG. 6, in the prior art, after decoding halfway through the second stream, the data that should originally be read is not read, and the image cannot be reproduced correctly, and an error is output. Then, only the data up to immediately before the picture data of the second stream is decoded and the data after it is cleared, the second stream following from the beginning of the buffer is stored, and the decoding is performed from the I picture that appears first. Since it starts, the error portion does not exist in the first place, and therefore, the image quality is not deteriorated (the portion where an error has occurred in the past is cleared). Further, in the example of FIG. 7, in the conventional technique, an error occurs because the data in the indefinite area is read out after decoding halfway through the second stream.
In the present embodiment, the undefined area is cleared, and the remaining portion of the second stream is newly stored and read from the beginning of the buffer. Therefore, the undefined area does not exist and the image quality does not deteriorate. The effectiveness of this embodiment will be clear.

In the present embodiment, the image data for several frames after the image size is changed as described above is cleared and is not reproduced, so that an error (disorder of image quality) occurs.
However, the so-called “image jump” occurs instead of the above, but since the jump is extremely short, the user does not feel any inconvenience.

[0034]

As described above, according to the present invention,
Even if bit streams having different image sizes are continuously input, it is possible to suppress deterioration of reproduced image quality without increasing the capacity of the DRAM, and it is possible to obtain an MPEG decoder with high cost performance.

[Brief description of drawings]

FIG. 1 is a configuration block diagram of an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing a flow of a bitstream of the same embodiment.

FIG. 3 is a processing flowchart of the same embodiment.

FIG. 4 is a processing flowchart of the same embodiment.

FIG. 5 is an explanatory diagram showing memory allocation according to image size.

FIG. 6 is an explanatory diagram when the image size is switched from small to large.

FIG. 7 is an explanatory diagram when the image size is switched from large to small.

[Explanation of symbols]

10 CPU interface, 14 DRAM controller, 16 DRAM, 20 VLD, 22 main control unit, 24 inverse quantizer (IQ), 2
6 Inverse DCT unit (IDCT), 30 Motion compensation unit (M
C).

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Anpachi Hamamoto 2-5-5 Keihanhondori, Moriguchi-shi, Osaka Sanyo Electric Co., Ltd.

Claims (10)

[Claims] 1. I including each predetermined macroblock
Decompresses image data of each picture from a bitstream formed by arranging compressed image data of a picture, P picture, and B picture so that reference macroblocks at the time of decoding each macroblock are temporally preceded, and outputs the display order. A compressed image data processing method, wherein the decoding process of the compressed image data immediately after the image size change is interrupted when bit streams having different image sizes are continuously input. 2. The compressed image data processing method according to claim 1, wherein after the decoding process is interrupted, the decoding process is restarted from the first I picture image data of a bitstream having a different image size. 3. I including each predetermined macroblock
A bit stream formed by arranging compressed image data of a picture, P picture, and B picture so that reference macroblocks at the time of decoding each macroblock temporally precede is stored in a memory, and these are sequentially read to read each picture. Is a compressed image data processing method of decoding the image data of, storing it in the memory, and outputting it in the order of display. When bit streams of different image sizes are continuously input, the bit stream already stored in the memory By canceling the bitstream decoding process, and then sequentially storing the input bitstream in the memory. 4. The compressed image data processing method according to claim 3, wherein the clearing process is executed after decoding the data up to immediately before the picture data in the bit streams having different image sizes. 5. The bitstream storage area and the decoded image data storage area of the memory are changed after decoding the data up to immediately before the picture data in the bitstreams having different image sizes, and then the clearing process is performed. The compressed image data processing method according to claim 4, which is executed. 6. The first I picture image data is searched from the bitstream stored in the memory after the clear processing is executed, and the decoding processing after the I picture image data is restarted. The compressed image data processing method according to any one of 3, 4, and 5. 7. I containing each predetermined macroblock
The compressed image data of pictures, P pictures, and B pictures are sequentially stored in a bit stream in which reference macro blocks at the time of decoding each macro block are sequentially stored, and the decoded image data is stored. A compressed image data processing device having a memory, comprising: detection means for detecting input of bitstreams having different image sizes; and memory control means for clearing the bitstreams stored in the memory after the image size has changed. A compressed image data processing device characterized by the above. 8. The memory control means clears the bitstream stored in the memory after a time point when the decoded data up to immediately before the picture data in the bitstream is input to the variable length decoding unit. The compressed image data processing device according to claim 7. 9. The method according to claim 7, further comprising search means for searching the first I picture image data included in the bit stream stored in the memory after the memory is cleared. Compressed image data processing device. 10. A compressed image data processing method for temporarily storing a plurality of image data required at the time of decoding in the case of decoding encoded compressed image data, wherein compressed image data having different image sizes are consecutive. When the data is input as described above, the storage area allocation of the plurality of image data in the memory is changed and set according to the image size, and the decoding processing is interrupted.