KR100891116B1 - Apparatus and method for bandwidth aware motion compensation - Google Patents

Abstract

The present invention relates to a bandwidth-aware motion compensation device, comprising: a bandwidth determination unit for determining a maximum bus bandwidth allowed for motion compensation, and determining the quantity and range of reference data to be stored in the variable cache buffer based on the determined bus bandwidth. Specify the reference data and transfer the reference data from the external memory to the variable cache buffer to update the variable cache buffer, and check whether the data to perform motion compensation is stored in at least one of the variable cache buffer or the external memory. In accordance with the results of the cache checker and the cache checker to include at least one of the variable cache buffer or the external memory to read the data to perform the data to perform the motion compensation, thereby reducing the read cycle of the motion compensation and system bandwidth efficient It allows for motion compensation.

Motion Compensation, HD, Memory Bus Bandwidth, Bandwidth Aware, Variable Cache Buffer

Description

Apparatus and method for bandwidth aware motion compensation

 1 is a diagram illustrating bandwidth analysis in a critical case in an HD class 1088-60p environment.

2 is a block diagram of a bandwidth-aware motion compensation apparatus according to an embodiment of the present invention.

3 illustrates the structure of a variable cache buffer.

4 illustrates the function of the cache controller.

5 illustrates a data transmission and update method of the cache controller.

6 illustrates a data update method in a variable cache buffer.

7 is a flowchart of a method of a bandwidth aware motion compensation apparatus according to an embodiment of the present invention.

The present invention relates to a bandwidth-aware motion compensation device and method, and more particularly, to a motion compensation device and method for compensating for motion of a macro block in consideration of a bandwidth allowed to a motion compensation module in a video codec device.

H.264 / AVC video coding was developed in 2001 by the JVC (Joint Video Team), which forms the ITU-T Video Coding Experts Group (VCEG) and Moving Picture Experts Group (MPEG). H.264 / AVC can reduce the bit rate by 39% ~ 64% compared to MPEG-4, H.263, and MPEG-2, and these performance improvements are mainly based on motion estimation using quarter pixel and variable block size. It comes from several new features, including sizes (VBS), multiple reference frames (MRF), de-blocking filters, integer transforms, and advanced entropy coding tools.

In terms of motion compensation, variable block size and multiple reference frames are the major factors that make bus traffic and bandwidth insufficient. The motion compensation process can use more than 50% of the total data bandwidth in critical cases, but this depends on how many different components it has. According to the operating time analysis of the H.264 / AVC decoder software, the motion compensation process uses up to 55% of the total decoding time.

Motion compensation is two methods for real time processing of high definition (HD) systems, especially in an environment in which an image sequence is input at 1920 * 1088 resolution and a transmission rate of 60 frames per second (hereinafter, referred to as '1088-60p'). Has a very important problem. One is the bus bandwidth problem and the other is the processing cycle problem of motion compensation. The processing cycle problem of motion compensation can be solved by core utilization, parallelization of interpolation processing, optimized decoding flow, hierarchical separation of motion compensation data reader, and the like.

For video over HD resolution, the bus bandwidth problem is more serious. In addition, almost all systems cannot make the bus 100% efficient because of random master requests, short burst efficiency issues due to DRAM-assisted bursts, initial delays and bus traffic. The efficiency of the bus depends on whether the data response comes directly from the bus. Various methods have been developed to solve this bandwidth problem, but these methods cannot be applied to various systems having different bandwidths. In addition, the caching method is not suitable for variable block sizes and multiple reference frames, as well as in the environment of HD-class 1088-60p.

The motion compensation module is the module that consumes the most bus bandwidth in the decoder. In an H.264 / AVC system, 5 pixels are needed because the data is interpolated with a 6-tap filter. In addition, there are some restrictions for high resolution images of high quality, which means that the maximum number of motion vectors (MaxMVsPer-2Mb) per two consecutive macroblocks (MB) is 16. In this case, the critical case of motion compensation means that all macro blocks are encoded by bidirectional prediction of 8 × 8 (13 × 13) size blocks or unidirectional prediction of 4 × 8 (9 × 13) size blocks.

Considering the bus data bits, the burst size for 8x8 mode is 4 bytes or 3 bytes in 32 or 64 bit bus systems where double date rate (DDR) is already taken into account, respectively. The size is 4 bytes or 3 bytes and 3 bytes or 2 bytes. However, DDR1 only supports 2-, 4- and 8-bursts, and DDR2 supports 4-, 8-bursts, so virtually only 4-bursts are adopted for all cases for both bus data bits. Therefore, the maximum data bandwidth is 576 Hz per MB.

Figure 1 shows the bandwidth analysis of the critical case in HD class 1088-60p environment.

YCbCr is 4: 2: 0 and considered a single bus system. Fixed bursts represent actual values calculated with bursts applicable to DDR1 or DDR2.

As shown in FIG. 1, the threshold bandwidth of motion compensation exceeds 280 MHz at 1088-60p. In this case, the system bandwidth and clock will exceed 400 MHz for real-time processing. This is not common, but it is possible in an H.264 / AVC environment. And if every system considers any arbitrary case, there must be a solution to this critical case.

In the prior art, there have been various solutions to the problem of insufficient bandwidth, but it does not take into account what the maximum bandwidth the motion compensation module can use. In addition, some assumptions that the interpolation window should be located in the same place are necessary, and thus are not appropriate for variable block sizes and multiple reference frames, and cannot be handled until each block has a different motion vector.

Accordingly, an object of the present invention is to provide a motion compensation apparatus and method considering a bandwidth of a decoder of a compressed image, the motion compensation apparatus controlling transmission and storage of data in consideration of a bandwidth required for performing motion compensation; To present a method. A bandwidth aware motion compensation structure for an H.264 / AVC decoder is proposed. By determining the maximum bandwidth information available in the motion compensation module, the variable cache buffer is used to determine the quantity and range of data to be managed, and to control the storage and update of data transferred from the external memory to the variable cache buffer.

In addition, in order to reduce the data read cycle, the data transfer and update process to the variable cache buffer may be performed separately from the motion compensation, thereby updating data in parallel with the motion compensation. In addition, the data for motion compensation can be read from the variable cache buffer or the external memory, and the method can be read from both the variable cache buffer and the external memory.

Therefore, in order to achieve the above technical problem, the bandwidth-aware motion compensation apparatus according to an embodiment of the present invention is a bandwidth determination unit for determining the maximum bus bandwidth allowed for motion compensation, and stored in a variable cache buffer based on the determined bus bandwidth A cache controller which determines the quantity and range of reference data to be used, specifies the reference data, and transfers the reference data from the external memory to the variable cache buffer to update the variable cache buffer; Or a cache checker for checking whether the memory is stored in at least one of the external memories, and a data readout for reading data to perform motion compensation by accessing at least one of the variable cache buffer or the external memory according to the result of the cache checker. Include.

In one embodiment of the invention, the bandwidth determiner, the first bandwidth is the total bus bandwidth of the decoder, the second bandwidth is the bus bandwidth required for the performance of other components except the motion compensation device, the third bandwidth is a variable cache When the bus bandwidth required for motion compensation when no data is stored in the buffer, the maximum bus bandwidth is determined by subtracting the second bandwidth and the third bandwidth from the first bandwidth.

In a preferred embodiment, the variable cache buffer uses at least one cache buffer, the first cache buffer having a width greater than the number of bytes in one line of the block, and the second cache buffer less than the number of bytes in one line of the block. When having a width, it is comprised of at least one of a first cache buffer and a plurality of said second cache buffers, such that the total width of one line of the variable cache buffer is greater than the number of bytes in one line of the block.

In one embodiment, the cache controller classifies the mass data and the update data among the reference data and transmits them to the variable cache buffer, and determines an area for searching the mass data on the variable cache buffer.

In order to achieve the above technical problem, the bandwidth-aware motion compensation method according to an embodiment of the present invention, determining the maximum bus bandwidth allowed for motion compensation, the reference to be stored in the variable cache buffer based on the determined bus bandwidth Determining the quantity and range of data, specifying the reference data and transferring the reference data from the external memory to the variable cache buffer to update the variable cache buffer, and at least one of the variable cache buffer or the external memory to perform motion compensation Determining whether the data is stored in one of the variable cache buffers or the external memory according to the result of the checking, and reading data to perform motion compensation.

Hereinafter, a bandwidth aware motion compensation apparatus and method according to an embodiment of the present invention will be described with reference to FIGS. 2 to 7.

2 is a block diagram of a bandwidth-aware motion compensation apparatus according to an embodiment of the present invention.

The bandwidth aware motion compensation apparatus 200 includes a bandwidth determiner 202, a cache controller 204, a variable cache buffer 206, a data bus 208, an external memory 210, a cache checker 212, and a data readout. Part 214 is included.

The bandwidth determiner 202 calculates a bandwidth that can be used by the motion compensation device among the total bandwidth of the data bus 210 and outputs it to the cache selector 204.

The cache controller 204 receives the bandwidth obtained by the bandwidth determiner 202, determines the quantity and range of reference data to be stored in the variable cache buffer 206 based on the bandwidth, and stores the variable in the variable cache buffer 206. The reference data to be transmitted is specified and the specified reference data is transmitted and updated from the external memory 210 to the variable cache buffer 206.

The variable cache buffer 206 receives and stores the reference data from the external memory 210 through the data bus 208 and is updated as indicated by the cache controller 204, and outputs the data to the data reading unit 214. do.

The data bus 208 is a data transmission path of the external memory 210 and the bandwidth aware motion compensation device 200.

The external memory 210 stores data to be used in the bandwidth-aware motion compensation device 200, and outputs the data to the motion compensation processing module according to an instruction of the cache controller 204 or the cache checker 212. Data is output to the bus 208.

The cache checking unit 212 checks whether the data required by the data reading unit 214 is stored in at least one of the variable cache buffer 206 and the external memory 210, and the result of the data reading unit 214. Output to

The data reading unit 214 reads data from the variable cache buffer 206, the external memory 210, or both based on the result of the cache checking unit 212, and thus the bandwidth according to the embodiment of the present invention. Recognition motion compensation is performed.

Hereinafter, each component of the bandwidth aware motion compensation device 200 will be described in detail.

Every system can have multiple components and data buses. As the image size and frame rate increase, the bandwidth usage of each component is limited. This is determined by how many different components the system uses.

The bandwidth determiner 202 considers the bus bandwidth required to perform other components except the motion compensation device in the total bandwidth available to the decoder and the bus bandwidth necessary for motion compensation when there is no data stored in the variable cache buffer. The bandwidth that the bandwidth motion compensation device 200 can use is calculated.

BW   = TotalBW  - OtherBW  - nohitBW

BW denotes a bus bandwidth to a bandwidth recognized motion compensator 200 is available, TotalBW is the total bus bandwidth, which is decoding system can be used, OtherBW bus bandwidth used by all other components and parallel devices other than the motion compensator Indicates.

nohitBW represents the bus bandwidth used by the motion compensation device when there is no data stored in the variable cache buffer. nohitBW is for critical cases where data should only be sent from external memory without storing it in the variable cache buffer. In an embodiment of the present invention, it is assumed that nohitBW is 50 clocks per macroblock.

Thus bandwidth recognition bus bandwidth to be considered in the motion compensation unit is the value obtained by subtracting the sum of the bandwidth nohitBW of the absence of data on a variable bandwidth OtherBW cache buffers of the other components in the bandwidth of the total TotalBW bus.

The variable cache buffer 206 should be large enough to access one line of a macroblock in one clock as long as motion compensation is performed. The variable cache buffer 206 may use a cache buffer having a width larger than that of the macroblock. When a small width cache buffer is used, a plurality of parallel cache buffers having a small width are used so that one line of a macroblock can be accessed at one clock.

3 illustrates the structure of the variable cache buffer 206 of FIG. 2.

According to one embodiment, the variable cache buffer 206 uses 6TSRAM or 1TSRAM. Since the maximum width of the macroblock used in the motion compensation is 21 bytes in consideration of the 6-tap filter in 16 × 16 mode, the width of the variable cache buffer must be large enough to include 21 bytes. Therefore, although a cache buffer having a width of 21 bytes or more may be used, a cache buffer having a width smaller than 21 bytes may be used by connecting a plurality of capper buffers in parallel.

That is, assume a critical case when a 16-byte wide cache buffer is used as the variable cache buffer 206 as shown in FIG. 3. In order to include 21 bytes of data 308, one byte may be called in the cache buffer 302, 16 bytes in the cache buffer 304, and 4 bytes in the cache buffer 306. Therefore, when a 16-byte cache buffer is used to form the variable cache buffer 206 according to an embodiment of the present invention, at least three 16-byte wide cache buffers are required.

Similarly, when an 8 byte wide cache buffer is used, the four cache buffers must be allowed until a critical case including 1, 8, 8, and 4 bytes, respectively, and thus the variable cache buffer 206 according to an embodiment of the present invention. At least four 8-byte cache buffers are required when using an 8-byte cache buffer.

4 illustrates the function of the cache control unit 204.

In operation 402, the quantity and range of reference data to be stored in the variable cache buffer 206 are determined using the bandwidth generated by the bandwidth determiner 202.

In step 404, reference data to be stored in the variable cache buffer 206 is specified by the cache selection algorithm based on the quantity and range of the reference data determined in step 402. The cache selection algorithm according to an embodiment of the present invention uses a closest picture order count (POC) and adds the same parity weight to the field.

In step 406, the reference data specified in step 404 is transferred from the external memory 210 to the variable cache buffer 206 to store massdata and update data.

Since the data transmission of the cache controller 204 is performed in a separate step from the operation of the motion compensation processing module, data can be transmitted and stored in parallel with the motion compensation process.

5 illustrates a data transmission and update method of the cache controller 204. The vertical search area 502 is a search area for mass data among reference data, and the vertical search area 504 after the vertical search area 502 is a search area for update data.

씩은 가변 캐쉬 버퍼의 수직 범위 504에 업데이트된다.

는 캐쉬 버퍼에서의 수직 검색 영역이다. The cache controller 204 obtains mass data from the reference data selected to be stored in the variable cache buffer 206 at the start of the slice, and stores the mass data in the variable cache buffer 206, where the vertical search area 502 of the mass data is obtained. Also determined by the cache control unit 204. Also, 2- or 4-burst × at the beginning of each macroblock

Each update to the vertical range 504 of the variable cache buffer.

Is the vertical search area in the cache buffer.

When the address of the data of the macroblock is changed, the access time of the data becomes long, so that update data is stored in advance in the vertical search area 504 of the variable cache buffer in burst units for more efficient call of the data.

6 illustrates a data update method in the variable cache buffer 206.

의 값은 각각 5, 5, 6 일 수 있다. 이는 대역폭 트래픽과, DRAM의 프로그램할 수 있는 버스트 길이로부터 야기되는 비효율성, 즉 1-, 2-, 3-버스트와 같이 작은 버스트로부터 오는 비효율성 문제를 감소시키기 위함이다.For example, in one embodiment of the present invention, if multiple cache buffers of size 16 bytes × 3 are used, the vertical search area of the cache buffers for macroblocks 1, 2, and 3

The values of may be 5, 5, and 6, respectively. This is to reduce the bandwidth traffic and the inefficiency resulting from the programmable burst length of the DRAM, i.e. the inefficiency problems resulting from small bursts such as 1-, 2- and 3-bursts.

According to an embodiment of the present invention, the quantity of reference data to be stored in the variable cache buffer 206 is found from the cache controller 204 using the maximum bandwidth of the motion compensation apparatus 200. In another embodiment, the amount of reference data to be stored in the variable cache buffer 206 may be directly limited by the user for the low power design of the motion compensation device 200.

When the operation of the cache controller 204 is completed, the motion compensation apparatus 200 determines which reference data and which range of data are already stored in the variable cache buffer 206. Since the bus data 208 may respond only after the operation of the cache controller 204 is completed, it is difficult to perform motion compensation and interpolation without the variable cache buffer 206. Moreover, weighted prediction or bidirectional prediction requires more processing cycles.

The preferred embodiment of the present invention separates the cache checker 212 and the cache reader 214 from the motion compensation processing module. As such, the motion compensation processing module can process the motion compensation task using the variable cache buffer 206 of the motion compensation device 200 from beginning to end, and there is no delay.

The cache checking unit 212 checks whether reference data necessary for motion compensation of the current block is stored in the variable cache buffer 206 or the external memory 210, and outputs the result to the data reading unit 214. The necessary reference data may be stored in either the variable cache buffer 206 or the external memory 210, or both. The cache checking unit 212 checks the position of the necessary reference data while checking the index of the mode, the motion vector, and the reference data.

The data reading unit 214 accesses the external memory 210 through the bus 208, accesses the variable cache buffer 206, or is variable with the external memory 210 according to the instruction of the cache checking unit 212. Both cache buffers 206 can be accessed to read data. The motion compensation processing module processes the motion compensation task using the data read by the data reader.

7 shows a flowchart of a method of the bandwidth aware motion compensation apparatus 200 according to an embodiment of the present invention.

In step 702, determine the maximum bus bandwidth allowed for motion compensation.

In step 704, the quantity and range of reference data to be stored in the variable cache buffer is determined based on the bus bandwidth determined in step 702.

In step 706, the reference data is specified according to the information determined in step 704, the reference data is transferred from the external memory to the variable cache buffer, stored in the variable cache buffer, and updated.

In step 708, it is determined whether the data required for motion compensation of the current block is located in the variable cache buffer or the external memory.

In step 710, as determined in step 708, the variable cache buffer or external memory or both are accessed to read data to perform motion compensation.

One embodiment of the present invention uses the H.264 / AVC reference software JM10.2 codec and motion compensation hardware module. Every 30th frame is used as an intra period, and 4 reference frames and 3 B frames are used. Both 32-bit and 64-bit bus systems are considered for DDR.

It can save about 60 to 80% of the readout cycle of motion compensation under 1088-60p environment. The used data bandwidth of DDR1-32-bit and DDR2-64-bit systems saves 30-50% in sequences with I and P only and 50-80% in sequences with I, P, and B, respectively. . More than 80% of the frames in each bus data bit and structure are stored in the variable cache buffer, and if more bandwidth is allowed for the cache control, the probability of storing the data in the variable cache buffer can be increased.

Preferred embodiments of the present invention can save 60-80% of MC read cycles and 50-85% of total data bandwidth. It can also be applied in various systems with different maximum data bandwidths. Since the data update process of the variable cache buffer has a certain pattern, the calling form of the bus is almost normalized. Therefore, it is much more convenient to access the variable cache buffer than the conventional technique, and it is useful for data bus scheduling.

Meanwhile, the above-described embodiments of the present invention can be written as a program that can be executed in a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium.

The computer-readable recording medium may be a magnetic storage medium (for example, a ROM, a floppy disk, a hard disk, etc.), an optical reading medium (for example, a CD-ROM, a DVD, etc.) and a carrier wave (for example, the Internet). Storage medium).

So far I looked at the center of the preferred embodiment for the present invention. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

The present invention provides a motion compensation apparatus and method for controlling a movement and storage of data in consideration of a bandwidth required for performing motion compensation in a motion compensation apparatus and method considering a bandwidth of a decoder of a compressed image. In particular, the present invention proposes a bandwidth-aware motion compensation apparatus and method for H.264 / AVC that supports a decoder in a HD class environment with a transmission speed of 60 frames per second at 1920 × 1088 resolution.

This hardware module can save 60-80% of the data read cycle of motion compensation in both DDR1-32bits and DDR2-64bits systems. In addition, it has the effect of reducing 50 to 85% of the data bandwidth in the sequence of the 'IBBBP' structure, and 30 to 50% in the 'IPPPP' sequence. This difference occurs because the 'IPPPP' sequence has relatively fewer buses to access than the 'IBBBP' sequence.

Since the number and range of the variable cache buffers can be controlled by identifying the maximum bus bandwidth that the motion compensation module can use, embodiments of the present invention can be applied to various systems. It is also possible to build a system that operates under 266MHz data bandwidth in a 1088-60p environment. Bandwidth-aware motion compensation can reduce both bus bandwidth and read cycles and perform in real time even in HD-grade 1088-60p and higher environments.

Claims (5)

A bandwidth determining unit determining a maximum bus bandwidth allowed for motion compensation; The quantity and search range of the reference data to be stored in the variable cache buffer are determined based on the determined bus bandwidth, and the reference data is specified to transfer the reference data from an external memory to the variable cache buffer to update the variable cache buffer. A cache control unit; A cache checking unit for checking whether data necessary for motion compensation of a current block is stored in at least one of the variable cache buffer and the external memory; And a data reading unit reading data to perform motion compensation by accessing at least one of the variable cache buffer and the external memory according to a result of the cache checking unit. The method of claim 1, The bandwidth determining unit may determine the maximum bus bandwidth of remaining bandwidths excluding the second bandwidth and the third bandwidth of the first bandwidth, The first bandwidth is a total bandwidth of a bus through which data is exchanged between a decoder and the external memory, The second bandwidth is a bandwidth required for data exchange through the bus between the external memory and other components except the motion compensation apparatus among the components of the decoder, The third bandwidth is a bandwidth-aware movement, characterized in that when there is no data stored in the variable cache buffer, data necessary for motion compensation is required to be exchanged through the bus between the external memory and the data readout. Compensation device. The method of claim 1, The variable cache buffer uses at least one cache buffer, When the first cache buffer has a width larger than the number of bytes in one line of the block, and the second cache buffer has a width smaller than the number of bytes in one line of the block, At least one of the first cache buffer and the plurality of second cache buffers, wherein the total width of one line of the variable cache buffer is greater than the number of bytes of one line of the block . The method of claim 1, And the cache controller classifies and transmits mass data and update data among the reference data to the variable cache buffer, and determines an area for searching the mass data on the variable cache buffer. Determining a maximum bus bandwidth allowed for motion compensation; Determining the quantity and range of reference data to be stored in the variable cache buffer based on the determined bus bandwidth, and specifying the reference data to transfer the reference data from an external memory to the variable cache buffer to update the variable cache buffer. step; Checking whether data to perform the motion compensation is stored in at least one of the variable cache buffer and the external memory; And reading data to perform motion compensation by accessing at least one of the variable cache buffer and the external memory according to the verification result.

Images