KR100933331B1 - Video decoding device - Google Patents

Abstract

The video decoding apparatus of the present invention includes a decoder unit for decoding an input bitstream to restore an image, an internal memory and an external memory for storing data, and an internal memory and an external memory based on a memory access request from the decoder unit. Control to distinguish the data stored in the internal memory and the external memory based on at least one of a memory control unit transmitting predetermined data to each other, a bitstream information from the decoder unit, an internal memory size, and a bandwidth of the external memory. It includes a system control unit. This makes it possible to efficiently use the internal memory and the external memory when decoding a video signal with high resolution.

Description

Video decoding apparatus {Apparatus for decoding video}

The present invention relates to a video decoding apparatus, and more particularly, to a video decoding apparatus for controlling the efficient use of an internal memory and an external memory when decoding a high resolution video signal.

Technology development and discussion on Digital Cinema and Ultra-High Definition Television (UHDTV), which are digital imaging technologies that surpass the current 1920x1080, 2K-class High Definition broadcasting It is becoming. Digital cinema has a maximum image resolution of 4096x2096, 4K, and ultra-high definition TV has a resolution of up to 7680x4320, 8K at 4K resolution.

Ultra-high resolution video contains more image information than existing video, which makes viewers feel more real and thus it is an important element to realize realistic broadcasting that digital broadcasting ultimately pursues.

Since such ultra-high resolution video has a throughput of 3 to 8 Gbps per second without data compression and 100 to 600 Mbps during compression, data compression is indispensable for processing, transmitting, and storing images.

With the recent development of communication technology, electronic products such as mobile phones, PDAs, digital TVs, and DMBs have been developed. Data used in such electronic products include audio, still images, video, etc. Among them, the amount of data of the video is very large as described above.

Therefore, video compression standards have been developed to reduce the size of data, starting with H.261, VC-1, the standard of the Society of Motion Picture and Television Engineers (SMPTE), and ITU-T and ISO / IEC. Much progress has been made to H.264 / AVC. Among them, H.264 / AVC adds several features compared to the previous video compression standard to improve the compression efficiency. On the other hand, in the case of Level 4 dealing with HD-class applications among the levels of H.264 / AVC, the number of reference pictures is set to a maximum of four.

1 is a diagram illustrating a frame memory in a conventional decoding apparatus.

Referring to FIG. 1, in a conventional decoding apparatus, that is, a decoding apparatus of H.264 / AVC, 4 corresponding to a reference frame size is used to decode an encoded bitstream. A total of six HD-quality image memory spaces are required, including one memory space, one memory space for a frame to be currently decoded, and one memory space for a frame not used as a reference.

Since the decoder must display after decoding the input bitstream, it is necessary to distinguish the case of the image which is not used as a reference so that decoding or display does not interfere.

For example, suppose that the previous four pictures are used as a reference during decoding, and the pictures decoded thereafter are not used as references, and the display order is the same as the input order of the bitstream. . After decoding up to the fifth input image, decoding the sixth input image requires the previous four reference images and memory space for decoding the sixth input image. Since the memory space stored after the fifth input image is decoded needs to be maintained for the display of the fifth image, the space of the required frame memory is increased by adding the previous four reference spaces and one currently decoded space. Will be the size of.

As described above, a high-definition application requires a large memory space. In particular, when decoding a bitstream compressed using a compression standard such as H.264 / AVC or VC-1, the 6-tap or 4-tap filtering tab can be used for motion compensation. As a result, it requires more pixels than the size of a block to be motion compensated, and the size of the blocks varies, requiring more memory bandwidth than the existing video compression standard.

In particular, among the elements of video decoding, the motion compensation unit requires the largest amount of data, and as the video compression standard having better compression efficiency and image quality increases, the degree of data demand increases rapidly. Increasing the memory bandwidth of the motion compensator causes difficulty in implementing integration with other applications such as an image scaler due to the bandwidth limitation of the memory module. In addition, when ultra-high-resolution images require super-capacity data access, high-performance memory specifications and a plurality of memory modules are required.

In applications with small video sizes, the frame memory size is not relatively large, so using an external memory such as SDRAM does not limit the memory bandwidth, but in applications where the frame memory size is very large, such as HD, The use of external memory causes difficulties in decoder implementation due to memory bandwidth limitations.

This problem of memory configuration in the HD class poses a further obstacle to constructing an ultra-high resolution imaging system. The use of high specification memory to solve the increase in memory bandwidth leads to an increase in cost, and the inefficiency of the memory use is increased. In addition, to reduce the dependence on the use of external memory, it can be considered to put the entire frame memory into the internal memory, such as SRAM, but this is because the integration is relatively weak in the hardware configuration, the gate size increases, so There is a weak point in economics due to the high cost increase compared to memory use.

An object of the present invention is to provide a video decoding apparatus for efficiently configuring and using an internal memory and an external memory when decoding a high resolution video signal.

According to an aspect of the present invention, there is provided a video decoding apparatus comprising: a decoder for decoding an input bitstream to reconstruct an image, an internal memory and an external memory for storing data, and a memory from a decoder. An internal memory based on at least one of a memory controller for transmitting predetermined data to an internal memory and an external memory based on an access request, and at least one of bitstream information from the decoder unit, an internal memory size, and a bandwidth of the external memory; And a system controller for controlling data stored in the external memory to be distinguished.

According to the present invention, when decoding a video signal having a high resolution, the video decoding apparatus efficiently stores the internal memory and the external memory based on at least one of bitstream information, the size of the internal memory, and the bandwidth of the external memory. Will be configured and used. Accordingly, by fully utilizing the internal memory, it is possible to alleviate the memory bandwidth limitation due to external memory access with high data throughput.

In addition, the present invention can suppress the increase factor of the gate size according to the increase in the size of the internal memory by appropriately utilizing the internal memory and the external memory instead of using only either internal or external memory for frame storage.

In addition, it is possible to use a low-end external memory that can reduce the cost instead of a high-end external memory.

In addition, the present invention can efficiently utilize the bitstream information by the system control unit.

In addition, the mitigation of memory bandwidth allows the decoder and other additional devices to share the external memory, thereby enabling the effective use of the device.

Hereinafter, with reference to the accompanying drawings looks at a preferred embodiment according to the present invention.

2 is a diagram illustrating an example of a super high resolution video configuration.

Referring to the drawings, an ultra-high definition image (UHD-class image) can be compressed and reconstructed using a video compression algorithm for a single image as shown in FIG. 2 (a), and as shown in FIG. After dividing the video (sub picture 0 ~ 3, HD quality video), it can be compressed and restored using a multiprocessor.

Both choices can be determined by the skill level at the time of system implementation. Hereinafter, it looks at the basis of Figure 2 (b) that can utilize the existing system.

3 is a block diagram illustrating a video decoding apparatus according to an embodiment of the present invention.

Referring to the drawings, the video decoding apparatus 300 includes a decoder 310, an internal memory 320, an external memory 330, a memory controller 340, and a system controller 350.

The decoder 310 restores an image displayed from an input bitstream. In more detail, the entropy decoding unit (not shown) receives the bitstream parsed from the previous stage of the decoder unit 310, performs the entropy decoding, and de-quantizes / converts ( A residual signal obtained by inverse quantization and inverse transformation of the residual signal encoded in the drawing is obtained. Meanwhile, the predictive picture based on the motion vector is calculated by the motion compensator (not shown), and the predictive picture and the residual signal are synthesized by the pixel reconstructor (not shown). In order to remove blocking artifacts of the synthesized image, the deblocking filter unit (not shown) performs deblocking filtering.

Inverse quantization and inverse transformation may be performed through one process, but may be separately performed through separate devices.

The internal memory 320 and the external memory 330 respectively store corresponding data. For example, the internal memory 320 may store some regions of the target frame to be decoded, some or all regions of the reference frame, parameters necessary for decoding, and peripheral information of the target macroblock.

In addition, the external memory 330 may store a current frame to be decoded and a frame not to be used as a reference frame.

The internal memory 320 and the external memory 330 are controlled by the memory controller 340 and the system controller 350 which will be described later.

The memory controller 340 transmits predetermined data to the internal memory 320 and the external memory 330 based on the memory access request from the decoder 310. The memory controller 340 controls data transmission between the internal memory 320 and the external memory 330 based on the information received from the system controller 350.

The system controller 350 may use the internal memory 320 and the external memory based on at least one of bitstream information from the decoder 310, the size of the internal memory 320, and the bandwidth of the external memory 330. The data stored at 330 is controlled to be distinguished. That is, the purpose of the memory is determined.

In addition, the system controller 350 controls parameter management and overall decoding situation necessary for decoding the video bitstream. In addition, considering the characteristics of the bitstream has the authority to change the role of the devices constituting the system.

The system controller 350 may control to store a partial region of the target frame to be decoded in the internal memory 320. This will be described later with reference to FIG. 4.

In addition, the system controller 350 may control to store a part or all of a reference frame in the internal memory 320. This will be described later with reference to FIG. 6.

In addition, the system controller 350 may control such that parameters necessary for decoding and peripheral information of the macro block are stored in the internal memory 320. If the external memory access is not performed separately to obtain necessary pixel information in the same frame, the memory bandwidth is reduced.

In order to decode the current macroblock in this manner, the parameter information of the macroblocks such as the surrounding motion vector, the format of the surrounding macroblock, and the surrounding pixel information is stored in the internal memory 320 instead of the external memory 330. This reduces the amount of external memory access.

In this case, unlike the pixel information related to the frame, the parameter information may be continuously stored in the internal memory 320 by overwriting each time new information is updated.

Meanwhile, the memory access order in the present invention is as follows.

First, when the operation of the decoding apparatus 300 starts, the system controller 350 grasps information about the internal memory 320 and the external memory 330 as described above. In this case, if there is not enough space in the internal memory 320 in consideration of the size of the internal memory 320 or the like, or if the bandwidth of the external memory 330 does not have a problem in configuring a video application system, it is the same as the conventional method. For example, video decoding is mainly performed through an external memory 330.

On the contrary, if it is determined that the space in the internal memory 320 is free and the external memory 330 has a limited bandwidth, the macroblock currently decoded is stored in the internal memory 320.

The conventional method also temporarily stores the currently decoded macroblock in the internal memory and transmits frame information to the external memory. However, in the present invention, the external memory is accessed during post-processing using pixels of a block adjacent to the current macroblock, such as deblocking filtering. The difference is that some areas of the current frame being decoded are stored in the internal memory so as to minimize the error.

4 is a view showing the processing contents of macro blocks in a frame according to the present invention.

Referring to the drawings, in the case of H.264 / AVC using Macroblock-Adaptive Frame / Field decoding (MBAFF), an area corresponding to six macroblock rows in total. 420 is stored in the internal memory 320. The region 410 above the row of six macroblocks has already been deblocked filtered, and the transfer to the external memory 330, especially the frame region, is completed. In the drawing, reference numeral 430 denotes an area to be decoded.

In the drawings, an example of using H.264 / AVC in the MBAFF in the video compression standard is given, but is not limited to this. In the case of using another compression standard or even in case of H.264 / AVC compression, the MBAFF is used. If not, the amount of data of the decoded target frame stored in the internal memory 320 may vary. That is, the size and the like of the partial region of the decoded target frame may be changed according to an encoding method or a purpose.

On the other hand, in a standard such as H.264 / AVC, the data for the current macroblock is read from the external memory 330 for deblocking filtering, rather than the amount of data for recording the currently decoded macroblock in the external memory 330. The amount of data movement occurring in the process of writing the data to the external memory 330 again after the deblocking filtering is performed may be greater.

Accordingly, when post-processing such as deblocking filtering, which is a level required to display the decoded target frame, is completed, at least some regions of the target frame whose restoration is completed by post-processing or the like are transmitted to the external memory 330. desirable. Further, even if the target frame for which restoration has been completed may be used in decoding and post-processing another frame, it is preferable that the target frame is continuously stored in the internal memory 320.

Meanwhile, the target frame that has been decoded but has not yet been subjected to post-processing such as deblocking filtering may be stored in the internal memory 320, but may be stored in macroblock units as shown in the figure.

In this case, the currently decoded macroblock may sequentially overwrite the internal memory 320, and thus, the number of decoded target frames stored in the internal memory 320 or the size of some region of the target frame may be constant.

5 is a diagram illustrating an example of transmitting a decoding macroblock of an internal memory to an external memory.

Referring to the drawings, the macro block data stored in the internal memory 320 after the processing for the current macro block is completed using information of the neighboring macroblocks and the like, is used as a display and a reference, the external memory 330 It is stored as a frame for decoding. This decoding frame is used as reference data when decoding another macroblock.

6 is a diagram illustrating an example of transmitting a reference frame in an external memory to the internal memory.

Referring to the drawings, if it is determined that the frame to be decoded by the system controller 350 is an inter frame, the reference frames of some of the reference frames stored in the external memory 350 are stored in the internal memory 320. do. In the drawing, reference frames 0 and 1 in the external memory 330 are stored in reference areas 0 and 1 in the internal memory 320, respectively. Here, the reference frame moved to and stored in the internal memory 320 may be part or all of the reference frame.

Meanwhile, when a reference frame in the external memory 330 is transmitted to and stored in the internal memory 320, the reference frame may be stored in macroblock units. In addition, the size of the reference frame area stored in the internal memory 320 may vary based on the size of the motion vector of the input bitstream.

7 is a diagram illustrating an example of transmitting a reference frame in an external memory to the decoder unit.

Referring to the drawings, the memory controller 340, while decoding by the decoder 310, if the data required by the current frame to be decoded does not exist in the internal memory 320, the external memory 330 Get the data you need from.

In case of bringing data from the external memory 330, the motion vector of the target macroblock indicates an area which is not stored in the internal memory 320 and a reference index different from that of the reference data stored in the internal memory 320. includes cases with (index).

FIG. 8 is a diagram illustrating an example of a super high resolution image configuration, and FIG. 9 is a diagram illustrating segmentation processing of the super high resolution image of FIG. 8, and FIG. 10 is a block diagram illustrating a video decoding apparatus according to an embodiment of the present invention. to be.

Referring to FIG. 8, an ultra high resolution (UHD) image may be an 8K class image including four 4K class images including four 2K class images.

Referring to FIG. 9, an ultra high resolution image (UHD) may be divided into a plurality of lower images (sub pictures 0 to 3).

Such a high resolution image may be divided into four subpictures, and may be divided for each subpicture. Of course, it may be processed as a single image as shown in FIG. In the case of processing a single image, as shown in FIG. 3, one decoder 310 and one internal memory 320 may be used. However, the external memory 330 may use more than one.

Meanwhile, unlike the video decoding apparatus 300 of FIG. 3, the video decoding apparatus 1000 of FIG. 10 uses a plurality of decoders 1010-0,. A plurality of internal memories 1020-0, ..., 1020-3 and a plurality of external memories 1030-0, ..., 1030-3 may be provided. It goes without saying that it includes a system controller 1050 and a memory controller 1040.

In the drawing, four decoder units, four internal memories, and four external memories corresponding to four subpictures of FIG. 9 are illustrated, but various embodiments are not limited thereto. For example, four 2K-class images of FIG. 8 may have respective external memories. In addition, in order to decode the 4K-class video of FIG. 8, one 4K-class decoder and four external memories corresponding to the 2K-class may be used. In such a case, four 4K class decoder units and 16 2K corresponding external memories can be used to decode 8K class images.

On the other hand, when a plurality of external memories 1030-0, ..., 1030-3 are used, and there is no reference frame in the external memory corresponding to any one of the decoder units, the memory control unit 1040 does not support the reference frames. Data can be transmitted to the required reference frame data through another external memory. In such a case, it may occur frequently near the boundary of the divided super high resolution image.

Meanwhile, in the above-described embodiments, the content stored in the internal memory may be changed according to the size of the internal memory and the bandwidth of the external memory. In addition, the size of the macroblock of the decoded target frame or the number of reference frames stored in the internal memory may be changed according to a coding method or a characteristic of the bitstream.

While the above has been shown and described with respect to preferred embodiments of the present invention, the present invention is not limited to the specific embodiments described above, it is usually in the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

1 is a diagram illustrating a frame memory in a conventional decoding apparatus.

2 is a diagram illustrating an example of a super high resolution video configuration.

3 is a block diagram illustrating a video decoding apparatus according to an embodiment of the present invention.

4 is a view showing the processing contents of macro blocks in a frame according to the present invention.

5 is a diagram illustrating an example of transmitting a decoding macroblock of an internal memory to an external memory.

6 is a diagram illustrating an example of transmitting a reference frame in an external memory to the internal memory.

7 is a diagram illustrating an example of transmitting a reference frame in an external memory to the decoder unit.

8 is a diagram illustrating an ultra-high resolution video configuration.

FIG. 9 is a diagram illustrating processing of the super high resolution image of FIG. 8.

10 is a block diagram illustrating a video decoding apparatus according to an embodiment of the present invention.

Claims (12)

A decoder to decode the input bitstream to reconstruct an image; An internal memory and an external memory for storing data; A memory controller which transmits predetermined data to the internal memory and the external memory based on a memory access request from the decoder; And And a system controller for controlling data stored in the internal memory and the external memory to be distinguished based on at least one of bitstream information from the decoder unit, the size of the internal memory, and the bandwidth of the external memory. A video decoding apparatus. The method of claim 1, The system control unit, And controlling a partial region of a target frame to be decoded to be stored in the internal memory. The method of claim 2, The system control unit, And controlling the number of decoded target frames or the size of a partial region stored in the internal memory to be constant. The method of claim 3, The size of the partial region of the decoded target frame stored in the internal memory is variable according to an encoding scheme. The method of claim 2, The partial region of the target frame stored in the internal memory is stored in the external memory after the deblocking filtering is performed. The method of claim 2, The partial region of the decoded target frame is stored in units of macro blocks. The method of claim 1, The system control unit, And controlling some or all of a reference frame to be stored in the internal memory. The method of claim 7, wherein The system control unit And when the target frame is determined to be an inter frame based on the bitstream information, controlling to transmit a part or all of a reference frame stored in the external memory to the internal memory. The method of claim 7, wherein Part or all of the reference frame is stored in units of macro blocks. The method of claim 7, wherein The system control unit, And determining the number of reference frames stored in the internal memory based on at least one of the bitstream information, the size of the internal memory, and the bandwidth of the external memory. The method of claim 1, The system control unit, And a parameter necessary for decoding from the decoder unit and peripheral information of the target macroblock to be decoded by the decoder unit to be stored in the internal memory. The method of claim 1, The system control unit, And storing the reference frame, a current frame to be decoded, and a frame not to be used as a reference frame in the external memory.

Images