JPH07298239A - Device and method for decoding image - Google Patents

Abstract

PURPOSE:To decode efficiently plural (multi-point) sets of image information with a small sized hardware. CONSTITUTION:Coded image data coming from different points are given to each of buffers 12-18. A storage quantity monitor circuit 20 always monitors data of each of the buffers 12-18 and informs the monitor result to a decoded image selection circuit 24. Furthermore, the stored data content (mainly header) in each of the buffers 12-18 and the monitor result to the decoding image selection circuit 24. The selection circuit 24 selects the output of each of the buffers 12-18 based on the monitor result of the monitor circuits 20, 22 to send the output to a decoding circuit 26. The decoding circuit 26 decodes coded image data selected by the decoding image selection circuit 24.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image decoding apparatus and method, and more specifically, to an image decoding apparatus and method for decoding encoded image information in a moving image transmission system such as a TV conference system. Regarding

[0002]

2. Description of the Related Art In recent years, the development of image compression coding technology and the widespread use of digital communication lines have been remarkable, and various communication protocols for videophones or videoconferences have been prepared as recommendations, and along with this, videophone devices and Various video and audio terminal devices such as a TV conference system have been proposed. Among them, a multi-point video conference (or video conference) system, which conducts a conference by connecting multi-point video conference terminals of three or more points to each other, has attracted attention.

As is well known, in order to directly digitally transmit moving image information, a transmission rate of several hundred Mbps is required. Therefore, various compression coding methods for reducing the transmission rate and the transmission cost are proposed. Has been done. Regarding transmission and display of moving image information in a multipoint video conference system, typically, a method of selectively encoding and transmitting a single point image such as a speaker image and displaying it, and a method of encoding images at a plurality of points. There is a method of converting and transmitting and displaying.

[0004]

The former method of moving image information, that is, the method of switching and transmitting only a single-point image of a speaker or the like, simplifies the decoding process and makes the device inexpensive and compact. However, since only a single (one-point) image can be transmitted at any time, there is a disadvantage that even when there are a plurality of moving images of interest at the same time, they cannot be displayed simultaneously.

In the latter method, that is, the method of transmitting a plurality of (multipoint) images simultaneously or in a time-division manner, if the number of image decoding devices corresponding to a plurality of encoded images is provided, the hardware becomes large in scale. . Further, when a plurality of encoded images are decoded by one decoding device, the decoding device
The decryption process has to be performed very fast, which is impractical.

As a means for solving these drawbacks of the latter method, a multipoint image to be received is treated as an apparently single image by adding or changing header information and the like, and a single decoding device of normal speed can be used. A configuration capable of performing a decoding process has been proposed. However, with this configuration, the frame rate of each point image must be made uniform, which imposes a strong restriction on image encoding, which tends to hinder actual operation.

An object of the present invention is to provide an image decoding apparatus and method capable of decoding images from a plurality of points with a small hardware.

Another object of the present invention is to provide an image decoding apparatus and method for decoding a plurality of (multipoint) image information without imposing a large restriction on the image encoding.

[0009]

In the present invention, a plurality of coded image information from a plurality of terminals or lines are temporarily stored in a plurality of storage means. The storage amount monitoring means monitors the information storage amount of each of the plurality of storage means, and / or the storage information monitoring means monitors the storage information of each of the plurality of storage means. The selection means is the accumulated amount monitoring means and / or
Alternatively, the output of the plurality of storage means is selected based on the monitoring result of the accumulated information monitoring means, and the decoding means decodes the selected output of the selecting means.

[0010]

With the above-mentioned means, one of the plurality of encoded image information is adaptively selected and decoded according to the storage amount and / or storage information of the storage means. By selecting according to the accumulated amount, overflow or underflow of the storage means can be prevented. The coded image data to be decoded can be selected according to the frame accumulation state in the storage means, and the decoding process can be made efficient.

[0011]

Embodiments of the present invention will now be described in detail with reference to the drawings.

FIG. 1 shows a schematic block diagram of an embodiment of the present invention. Here, ITU-T Recommendation H.264 is used. An image decoding device compliant with H.261 will be described as an example.

In FIG. 1, reference numeral 10 denotes an overall control circuit for comprehensively controlling the whole, that is, each unit described later. 1
2, 14, 16 and 18 are buffer memories for temporarily storing, for example, coded image data to be decoded from different terminals, and 20 are respectively stored in the buffers 12 to 18. A storage amount monitoring circuit that sequentially monitors the data amount, and 22 is a storage information monitoring circuit that monitors the content of the data stored in each of the buffers 12 to 18.

Reference numeral 24 is a monitoring result of the monitoring circuits 20 and 22,
And a decoded image selection circuit for selecting data stored in the buffer memories 12 to 18 under the control of the overall control circuit 10, and 26 for decoding the encoded image data selected by the selection circuit 24, ITU-T Recommendation H ． It is a decoding circuit compliant with H.261.

The coded image data input to the buffers 12 to 18 is ITU-T Recommendation H.264. Common Intermediate Format (CIF: Common Interface)
eFormat). The structure of the CIF format is shown in FIGS. The CIF format has a hierarchical structure, and one frame includes 12 GOBs (Group Of Blocks) as shown in FIG.
Composed of. One GOB is, as shown in FIG.
It consists of 33 macro blocks. Further, the macro block is composed of 6 blocks, as shown in FIG.
That is, it is composed of four Y blocks, one Cb block, and one Cr block. As shown in FIG. 5, one block consists of 8 pixels × 8 lines. A frame start code, frame number, GO
Various header information such as B start code, GOB number and macroblock address are inserted.

[0016] Next, ITU-T Recommendation H.264. A moving image compression encoding method in H.261 will be briefly described.

According to this recommendation, temporal redundancy is removed by motion-compensated interframe coding utilizing correlation in the time direction,
A hybrid coding method that removes spatial redundancy is used in orthogonal transform coding that uses correlation in the spatial direction. Discrete cosine transform (DCT) is adopted as the orthogonal transform. Further, as an encoding mode, an INTER (inter-frame encoding) mode in which a difference from a predicted value of a previous encoded frame is encoded, and an INT in which the difference is not encoded and is encoded as it is
There is an RA (Intra-frame coding) mode. Since the current frame and the previous frame are very similar to each other in an image with little movement or change, the temporal redundancy can be greatly reduced by using the INTER mode in which the difference from the previous frame is encoded. Further, by performing motion compensation (MC), efficient coding can be realized even for a moving image. For images with large movements and scene changes,
Since the correlation between frames is small, the coding within the same frame is more effective, and the INTRA mode that does not take the difference from the previous frame is used. These three coding modes, that is, the INTER mode with motion compensation, the INTER mode without motion compensation, and the INTRA mode are adaptively switched in macroblock units.

The basic operation of the embodiment shown in FIG. 1 will be described.
First, four pieces of encoded image data having different encoding sources (for example, encoded image data from different TV conference terminals) are input to the buffers 12 to 18, respectively. The storage amount monitoring circuit 20 constantly monitors the data storage amount of each of the buffers 12 to 18, and notifies the decoded image selection circuit 24 of the monitoring result. The accumulated information monitoring circuit 22 constantly monitors the accumulated data contents (mainly headers) of the buffers 12 to 18, and notifies the decoded image selection circuit 24 of the monitoring result.

The decoded image selection circuit 24 selects the outputs of the buffers 12 to 18 on the basis of the monitoring results from the storage amount monitoring circuit 20 and the storage information monitoring circuit 22, and selects the decoding circuit 2.
Send to 6. Details of the selection operation in the decoded image selection circuit 24 will be described later. The decoding circuit 26 has a plurality of frame memories for enabling the decoding processing of a plurality of encoded image data, and the encoded image data selected and input by the decoded image selection circuit 24 is followed by the above recommendation. Decrypt.

The overall control circuit 10 controls the timing of the entire apparatus shown in FIG. 1 and, at the same time, constantly monitors control signals from the entire apparatus and from the outside, and based on this, controls the entire image decoding. To do.

The selection operation in the decoded image selection circuit 24 will be described in detail. The selection circuit 24 of the present embodiment is
The decoded image data is switched in frame units.

First, the selection processing according to the monitoring result of the storage amount monitoring circuit 20 will be described. As described above, the storage amount monitoring circuit 20 constantly monitors the encoded data storage amount of each of the buffers 12 to 18, and the monitoring result is notified to the decoded image selection circuit 24. The decoded image selection circuit 24 adaptively selects one of the outputs of the buffers 12 to 18 so that the buffers 12 to 18 do not overflow or underflow, and supplies it to the decoding circuit 26. For example, the overflow of the buffers 12 to 18 means that the encoded image data stored in the buffers 12 to 18 is lost, and normal decoding processing becomes impossible. The decoded image selection circuit 24 executes the selection according to the data storage amount of the buffers 12 to 18, so that
Overflow and underflow of each buffer 12-18 can be avoided.

The selection processing according to the monitoring result of the stored information monitoring circuit 22 will be described. As described above, the stored information monitoring circuit 22 constantly monitors the header information in the encoded data stored in the buffers 12 to 18. In particular,
From the data stored in the buffers 12 to 18, the number and position of frame headers are detected to determine whether or not one frame data has been stored, and how many frame data have been stored. The decoded image selection circuit 24 is notified of frame accumulation states such as. Each buffer 1
2 to 18, the code amount per frame of the coded image data is unequal, so the decoded image selection circuit 24
One of the outputs of the buffers 12 to 18 is adaptively selected and supplied to the decoding circuit 26 so that the decoding process is efficiently executed according to the frame accumulation state of each of the buffers 12 to 18.

For example, since the switching of the decoding process in the decoding circuit 26 is performed on a frame-by-frame basis, if the storage of the frame data in the buffers 12 to 18 is not completed during the decoding process, the decoding process is performed. The circuit 26 has to wait for the reception of the encoded data (storage in the buffers 12 to 18), which causes unnecessary waiting time. The decoded image selection circuit 24 selects the output of the buffers 12 to 18 according to the contents of the stored data of the buffers 12 to 18, particularly the stored header information, in order to avoid such inefficient processing.

As described above, in this embodiment, each buffer 1
By selecting the coded image data to be decoded according to the data storage amount of 2 to 18, overflow and underflow of the buffers 12 to 18 are avoided. Also, the decoding process is made efficient by selecting the coded image data to be decoded according to the frame accumulation state in each of the buffers 12 to 18.

ITU-T Recommendation H.264 Although the image decoding apparatus according to H.261 has been described as an example, it is obvious that the present invention is not limited to this coding method and can be applied to other coding methods.

In the above embodiment, the selection circuit 24 selects
Although it has been described as a frame unit, it is also clear that it may be a GOB unit or a plurality of frame units.

In the above embodiment, four buffers 12-1
It is possible to handle up to 4 lines by providing 8, but it is possible to handle coded images of more lines by providing 5 or more buffers, and it is also applicable to providing 2 or 3 buffers. it is obvious. Further, in the above-described embodiment, the buffers 12 to 18 are described as being composed of separate hardware, but it is also clear that physically one buffer may be used as long as it can be logically used as another buffer. It goes without saying that even if one buffer memory is physically used, it may belong to the technical scope of the present invention.

In the above embodiment, the case has been described in which the selection control method according to the buffer storage amount and the selection control method according to the storage information are combined, but it can be easily realized by combining with another control method.

The present invention may be applied to either a system composed of a plurality of devices or an apparatus composed of a single device. Part of the present invention can be implemented by software.

[0031]

As can be easily understood from the above description, according to the present invention, a plurality of encoded image data can be decoded without causing overflow or underflow of each buffer. Moreover, a plurality of encoded image data can be efficiently decoded.

[Brief description of drawings]

FIG. 1 is a schematic block diagram of an embodiment of the present invention.

FIG. 2 is a layout diagram of 12 GOBs forming one frame in CIF format.

FIG. 3 is a configuration diagram of GOB.

FIG. 4 is a configuration diagram of a macro block.

FIG. 5 is a configuration diagram of a block including a minimum unit of 8 pixels × 8 lines.

[Explanation of Codes] 10: Overall control circuit 12, 14, 16, 18: Buffer memory 20: Storage amount monitoring circuit 22: Storage information monitoring circuit 24: Decoded image selection circuit 26: Decoding circuit

Claims (8)

[Claims] 1. An image decoding apparatus for decoding encoded image information, comprising: a plurality of storage means for temporarily storing a plurality of encoded image information; and information storage of each of the plurality of storage means. A storage amount monitoring unit for monitoring the amount, a selection unit for selecting the output of the plurality of storage units based on the monitoring result of the storage amount monitoring unit, and a decoding unit for decoding the selected output of the selection unit are provided. An image decoding device characterized by the above. 2. An image decoding device for decoding encoded image information, comprising: a decoding means for decoding the encoded image information; and a plurality of storages for temporarily accumulating a plurality of encoded image information. Means, accumulated information monitoring means for monitoring accumulated information in each of the plurality of storage means, selecting means for selecting output of the plurality of storage means based on the monitoring result of the accumulated information monitoring means, and the selecting means. An image decoding apparatus comprising: a decoding unit that decodes a selected output. 3. The image decoding apparatus according to claim 2, wherein the accumulated information monitoring unit monitors accumulated header information of accumulated information in the plurality of storage units. 4. A storage amount monitoring means for monitoring the information storage amount of each of the plurality of storage means, wherein the selection means is based on the monitoring result of the storage amount monitoring means. 4. The image decoding apparatus according to claim 2, wherein the output of is selected and transferred to the decoding means. 5. An image decoding method for decoding encoded image information, comprising: a temporary storage step of temporarily storing a plurality of pieces of encoded image information separately; An accumulation amount monitoring step of monitoring the information accumulation amount of the encoded image information, and a selection step of selecting one of the plurality of encoded image information accumulated in the temporary accumulation step based on the monitoring result of the accumulation amount monitoring step. And a decoding step of decoding the coded image information selected in the selecting step. 6. An image decoding method for decoding encoded image information, comprising: a temporary storage step of temporarily storing a plurality of pieces of encoded image information separately; A stored information monitoring step of monitoring the information content of the encoded image information, and a selection step of selecting one of the plurality of encoded image information stored in the temporary storage step based on the monitoring result of the stored information monitoring step. And a decoding step of decoding the coded image information selected in the selecting step. 7. The image decoding method according to claim 6, wherein said accumulated information monitoring step monitors accumulated header information of each encoded image information accumulated and stored in said temporary accumulating step. 8. The method further comprises a storage amount monitoring step of monitoring the information storage amount of each encoded image information stored and stored in the temporary storage step, wherein the selection step is a monitoring result of the storage amount monitoring step. The image decoding method according to claim 6 or 7, wherein one of the plurality of pieces of encoded image information accumulated in the temporary accumulation step is selected based on the above.