KR101004825B1 - Pipielined deblocking filter using two filters simultaneously - Google Patents

Abstract

PURPOSE: A de-blocking filter apparatus of a pipeline structure using a vertical filter and a horizontal filter reusing the data is provided to reduce the entire buffer occupancy by removing the use of pre-buffer. CONSTITUTION: A buffer memory(210) stores the row direction data of a macro block. A B buffer memory(220) stores the column direction data of the macro block. A horizontal filter(260) performs horizontal filtering operation about a vertical edge by one or more pipe stages by the row direction data. A vertical filter(270) performs vertical filtering about the horizontal edge.

Description

Pipelined Deblocking Filter Using Two Filters Simultaneously}

The present invention relates to a pipeline deblocking filter device using a horizontal filter and a vertical filter. In particular, the present invention relates to a deblocking filter device having a horizontal filter and a vertical filter. It relates to a low power consumption deblocking filter by eliminating ancillary prematrix operations.

Standardized by MPEG-2 and MPEG-4, video compression codecs standardized by the Motion Picture Experts Group (MPEG) under the World Organization for Standardization (ISO), and by the Video Coding Experts Group (VCEG) under the International Telecommunication Union (ITU-T). A video compression codec such as H.263 and H.264 performs encoding and decoding processes based on a block having a predetermined size. In the block-based video compression codec, a blocking artifact phenomenon occurs that degrades the subjective and objective image quality of the image, and a deblocking filter is used in an in-loop or post manner to remove the blocking artifact.

In the H.264 / AVC codec, a block having a size of 16 × 16 is used as a basic unit for image encoding and decoding, which is called a macroblock. A video codec encoding process in units of macroblocks will be described.

FIG. 1 is a block diagram of a video codec encoder in a macroblock unit.

The video codec encoder 100 may include a discrete cosine transform (DCT) unit 110, a quantization unit 120, a motion predictor 130, a motion compensator 140, an entropy encoding and decoding unit 150, and an inverse quantization unit. 160, an inverse DCT unit 170, and a deblocking filter 180.

First, the video codec encoder 100 searches for a portion of the reference video frame having the smallest difference from the macroblock through intra or inter prediction on the macroblock of the original video frame to be encoded. After that, the residual with the most similar part is obtained. The DCT unit 110 performs a discrete cosine transform by inputting the difference value, and the quantization unit 120 performs a discrete cosine transform on the result. Perform the quantization process.

The entropy encoding and decoding unit 150 performs entropy encoding on the result of the quantization process and finally generates a compressed H.264 / AVC bitstream.

On the other hand, the video codec encoder 100 performs the reconstruction process in the same procedure as the decoding process to generate a reference image frame that will be needed in the next compression process.

That is, the entropy encoding and decoding unit 150 performs an entropy decoding process on the compressed H.264 / AVC bitstream, and the dequantization unit 160 dequantizes it. On the result, the inverse DCT unit 170 performs inverse discrete cosine transform to restore residual information. The difference value information reconstructed as described above is decoded by being combined with a macro block subjected to intra or inter motion compensation.

The decoded image frame expresses a lattice boundary error that does not exist in the original image at the macroblock or subdivided macroblock boundary. The error is a phenomenon in which abnormal cosine transforms are relatively inaccurate for block boundary pixels due to discrete cosine transforms in blocks, lossy compression is performed through quantization, and restored to a value different from the original image in inverse quantization. This is caused by a phenomenon that occurs due to block prediction.

Such blocking artifacts recommend the use of the deblocking filter 180 in modern video compression codecs in order to improve reconstruction quality due to subjective picture quality degradation. In particular, in H.264 / AVC, the deblocking filter is included as a standard so that the deblocking filter is performed in a video encoding and decoding loop.

The deblocking filter 180 of H.264 / AVC distinguishes the edge existing in the original image from the edge generated by the blocking artifact, and does not deblock the edge existing in the original image. have.

The H.264 / AVC video compression codec introduces several technologies into the standard to improve the subjective and objective image quality while showing higher compression ratio than the existing codecs such as MPEG-2 and H.263. One of them is the in-loop deblocking filter, which determines the boundary strength according to the coding conditions generated during video compression encoding and decoding, and applies the normal intensity filter and the strong intensity filter according to the boundary strength. .

In addition, the H.264 / AVC deblocking filter performs adaptive deblocking filtering by dividing the actual edges of the image appearing in the original image frame and the blocking artifacts generated by the encoding and decoding processes in units of blocks. Is implemented.

In this case, by analyzing the pixel data distribution characteristics of adjacent blocks on the basis of the block edge, various types of filter equations are performed in the filter of the normal intensity and the filter of the strong intensity. As such, the H.264 / AVC deblocking filter is highly adaptable and needs to consider complexity and a large amount of computation in actual implementation. Therefore, a high performance deblocking method and apparatus are required. Meanwhile, if the same performance is required, the performance of the deblocking filter needs to be improved to use less memory resources and operate at a lower power.

Accordingly, the present invention is to solve the problems according to the prior art described above, to output the results calculated by the pipeline stages in the horizontal filter to remove the pre-buffer, and horizontal characterized in that using the pipeline register of the vertical filter It is an object of the present invention to provide a deblocking filter device of a pipeline structure including a filter and a vertical filter.

A deblocking filter according to an aspect of the present invention includes an A buffer memory for storing column direction data of a macro block; A B buffer memory for storing row direction data of a macro block; A horizontal filter performing horizontal filtering on vertical edges in at least one pipe step by using column direction data stored in the A buffer memory, and outputting the result data in a corresponding step; And a vertical filter which receives the row direction data stored in the B buffer memory and the result data output by the horizontal filter and performs vertical filtering on the horizontal edge.

The A buffer memory may store pixel data corresponding to the rightmost column of the left macroblock of the macroblock to be processed at the start of filtering of the macroblock to be processed.

The B buffer memory may store pixel data corresponding to the lowest row among upper macroblocks of the macroblock to be processed at the start of filtering the macroblock to be processed.

First forwarding means for re-inputting the horizontal filter result data into the horizontal filter; Second forwarding means for inputting the horizontal filtering result data into the vertical filter; And third forwarding means for re-inputting the vertical filter result data into the vertical filter.

The horizontal filter performs a filtering operation using result data re-input from at least one of the A buffer memory, the first forwarding means, or a memory located outside the deblocking filter in a pipeline composed of at least one step. It can be characterized by.

The vertical filter is configured to receive one pixel pair data from the horizontal filter or the B buffer memory in at least one of the pipeline stages, and to process the inputted pixel pair data or the previous pipeline stage to perform the third filter. The filtering operation may be performed using the result data re-input through the forwarding means.

The deblocking filter according to the present invention may further include a buffer for transposing the vertically filtered result data into data of a row direction line. The buffer may be implemented as a pipeline register of the vertical filter.

As described above, according to the pipeline deblocking filter device using the horizontal filter and the vertical filter according to the present invention, there is no need to use a separate pre-buffer, and thus the pre-operation is performed in the existing structure with a decrease in the total buffer usage. There is an advantage that can reduce the waste of power.

In addition, the forwarding structure is applied to the vertical filter like the horizontal filter, and the forwarding structure of the vertical filter allows the vertical filter to continuously filter the horizontal edges filtered, thereby reusing data. This reduces the number of times the buffer memory is accessed and reduces the amount of power required.

Hereinafter, a deblocking filter device having a pipeline structure using a horizontal filter and a vertical filter according to the present invention will be described in detail with reference to the accompanying drawings.

2 is a block diagram illustrating a deblocking filter according to an embodiment of the present invention.

As shown in FIG. 2, the deblocking filter 200 includes an A buffer memory 210, a B buffer memory 220, a buffer and an external memory controller 230, an interface strength determiner and a threshold generator 240, and a pipe. And a line controller 250, a horizontal filter 260, a vertical filter 270, a plurality of forwarding units 281, 282, 282, and the like.

The A buffer memory 210 stores luma 4 × 16 or chroma 2 × 4 × 8 pixel data located to the left of the macroblock currently being processed. The B-buffer memory 220 stores luma 16 × 4 or chroma 2 × 8 × 4 pixel data located above the macroblock currently being processed.

In addition, the deblocking filter of FIG. 2 largely includes three forwarding means 281, 282, 283. The first forwarding means 281 is a component for re-inputting the horizontal filter 260 the horizontally filtered result data into the horizontal filter. If present, the first forwarding means 281 can be used to apply other types of filtering edge order.

In addition, the second forwarding means 282 is a component for forwarding the resultant data filtered by the horizontal filter 260 to the vertical filter 270. The horizontal filter 260 does not output the result data performed in the pipeline stage to the final stage, but outputs the vertical data to the vertical filter 270 directly at the stage through the second forwarding means 282. The four squares on the right side of the horizontal filter 260 of FIG. 2 mean that the result calculated at each stage of the pipeline is immediately output at the corresponding stage. The horizontal filter 260 may be configured in the form of a pipeline having a plurality of stages.

The third forwarding means 283 corresponds to a means for re-input of the result data vertically filtered by the vertical filter 270 into the vertical filter 270.

The horizontal filter 260 performs horizontal filtering on at least one pipe step by using the input row direction data, and outputs the result data in the corresponding step.

In this case, the horizontal filter 260 reads row direction data, that is, eight pixel data at once, in one pipeline stage. On the other hand, the vertical filter 270 is different from the horizontal filter 260 in that the vertical filter 270 receives the result data output by pipe processing by the horizontal filter 260 and performs vertical filtering using the result data.

Meanwhile, referring to FIG. 2, it can be seen that the result of the horizontal filter 260 is input to the A buffer memory 210. The Q block of the horizontal filter 260 is outputted with four pixel data at once and these pixel data are stored in the A buffer memory 210 for filtering on the next vertical line.

Like the horizontal filter 260, the vertical filter 270 may be configured in a pipeline form having a plurality of stages. In this embodiment, the vertical filter 270 having an eight-stage pipeline structure will be considered. The vertical filter 270 may perform a pipeline operation as follows.

Step 0: Calculations relating to P0 and Q0. (ex. p0 + q0 or ((q0-p0) << 2))

1st stage: P1 and Q1 alone or a combination of calculation results from 0th stage. (ex. p1 q1 or p1 + q1 + 2)

2nd step: P2 and Q2 only, or calculations related to the mixed results of the 0th and 1st steps. (ex. p2 + p1 + 2 or ((p0 + q0) + (p1 + q1 + 2)))

Step 3: P3 and Q3 only, or a combination of calculation results generated in steps 0, 1, and 2 related to the calculation.

Steps after Step 4: Operations related to a mixture of calculation results generated in Step 0, Step 1, Step 2 and Step 3.

For example, referring to the second stage, the vertical filter 270 is already calculated in the previous pipeline stage (first stage) and the P2 and Q2 information received from the horizontal filter 260 or the B buffer memory 220. The operation is performed on the re-entered data. Similarly, in the third stage, the vertical filter 270 is calculated by the horizontal filter 260 or the B buffer memory 220, and the information is calculated and re-input in the previous pipeline stages (first and second stages). The operation is performed using the data.

That is, the vertical filter 270 receives one pixel pair data from the horizontal filter 260 or the B buffer memory 220 in at least one of the pipeline stages, or the vertical filter and vertical filter with the input pixel pair data. 270 processes in the previous pipeline stage and receives one pixel pair data from the result data re-input through the third forwarding means 283 to perform a filtering operation.

It can be seen that the preposition logic is included in the upper left side of the vertical filter 270 of FIG. 2. This transposition logic is a component for transposing the vertically filtered result, that is, the final filtered result data, into the data of the row direction line. This pre-logic is preferably implemented with a pipeline register provided on the vertical filter 270.

2, the P block output of the vertical filter 270 is filtered pixel data. Filtered data is generally output to an external bus, but pixel data may be temporarily stored in the B buffer memory 220.

This situation may occur in the following situations. When processing one macroblock, the lowest row blocks among the upper macroblocks of the current macroblock are read from the bus and filtered. In this case, the pixel data of cells belonging to the lowermost row of the upper macroblock of the macroblock currently being processed may change in value. Of course, the existing value stored in the external memory for the cells in which the value is changed will also need to be updated through the bus. However, if the bandwidth of the external bus is limited and the resulting pixel data generation is larger than the bus bandwidth, the pixel data is temporarily stored in the B buffer memory 220. Since the bus output and the B buffer memory 220 are connected, data stored in the B buffer memory 220 is transferred to the external memory later.

In order for the deblocking filter 200 to process one macroblock, luma 16 × 16 or chroma 2 × 8 × 8 pixel data corresponding to the macroblock currently processed by the deblocking filter 200 is required.

In addition, the pixel information corresponding to the rightmost part 4x16 or chroma 2x4x8 of the immediately previous macroblock located to the left of the current macroblock and the lowermost part 16x of the macroblock located above the current macroblock Pixel data corresponding to 4 or chroma 2x8x4 is required.

The left and upper macroblocks of the macroblock to be processed currently access information stored in the external memory through a READ / WRITE process or a storage space such as on-chip SRAM or flip-flop inside the deblocking filter 200. There is a method of preparing and storing a portion of the current macroblock as information for a macroblock to be processed next.

Among the two methods, the method of configuring the temporary storage space inside the deblocking filter 200 has an advantage of excellent latency and consumes less power. In the hardware structure using the communication method of the shared bus structure, the deblocking filter ( There is an advantage in reducing the bus bandwidth required for 200). This is because external memory access has a relatively high latency and high power consumption. However, the use of additional buffers, such as on-chip SRAM or flip-flops, leads to increased production costs, and these buffers are also power consuming components, so it is advantageous to use as little as possible.

The deblocking process of the H.264 / AVC standard filters the vertical edges first (1) for luma 16 × 16 with 16 4 × 4 blocks, two chroma with 8 4 × 4 blocks, and 8 × 8 macroblocks (horizontal). Filtering), again horizontally filtered pixels (2) vertically with respect to the horizontal edges (vertical filtering) to obtain the final result.

In order to satisfy the pixel data dependency by the filtering order in the H.264 / AVC standard, each 4x4 block must be filtered by the vertical edge first and then by the horizontal edge later. Therefore, there must be a difference in starting time between the horizontal filter and the vertical filter.

In other words, each 4x4 block should be filtered in order of left vertical edge, right vertical edge, upper horizontal edge, and lower horizontal edge. The method of filtering 64 edges present in a macroblock is filtered to edges. There may be several methods depending on the order of application.

3 is a view showing an edge filtering processing sequence according to the present invention.

In the present invention, since two different filters, that is, the horizontal filter 260 and the vertical filter 270 are executed at the same time, a different edge filtering processing sequence is used than when only one filter is used. This will be described with reference to FIG. 3.

In FIG. 3, L0 to L3, T0 to T3, and 0 to 15 are described in the cell. 0 to 15 are described for the 4x4 block belonging to the macroblock to be processed at present. L0 to L3 are blocks located at the rightmost side of the left macroblock of the macroblock currently to be processed. T0 to T3 mean lowermost position blocks of the upper macroblock of the macroblock to be processed.

Circles marked at the edges mean edge filtering of the horizontal filter, and triangles mean edge filtering of the vertical filter. Numbers in circles or triangles indicate edge filtering order. For example, filtering is performed for the first time since 0 is written at the left edge of the cell 0. On the other hand, the upper edge of cell 0 and the left edge of cell 9 both have the number 6. These are filtered simultaneously in the horizontal filter 260 and the vertical filter 270 in the sixth order.

Meanwhile, in the existing deblocking filter, there is a difference in the required size, but all of them need a space for temporarily storing the result value generated in the middle of the deblocking filtering process. For example, a buffer with a predetermined size (4x16) is needed to temporarily store intermediate filtered results generated by the horizontal filter.

In the present invention, the 4 × 16 buffer for temporarily storing the intermediate filtered result is replaced with the buffer for storing the data of the right macroblock, that is, the A buffer memory 210 of FIG. 2.

For example, when filtering a 16 × 16 luma block, the rightmost 4 × 16 pixel data of the left macroblock of the current macroblock at the time of deblocking the edges 0, 1, 2, and 3 marked by the circle of FIG. Are stored in the A buffer memory 210 and are used when performing filtering of the next column.

After vertically filtering edges 0, 1, 2, and 3 marked with circles, the results are overwritten with the data stored in the A buffer memory 210. That is, the A buffer memory 210 continues to be reused. This overwritten data is also used to filter the next column.

The intermediate filtered results of the horizontal filter 260 that occur while performing edges 4, 5, 6 and 7 marked with circles are likewise stored in the A buffer memory 210. The results for the edges 4, 5, 6, and 7 marked with a circle are overwritten in the buffer space where the 0, 1, 2, and 3 edge filtering results are stored.

When the filtering for the macroblock represented in FIG. 3A is completed, the A buffer memory 210 has 4 × 16 data of the right part of the current macroblock temporarily stored for the next macroblock processing, that is, an edge marked with a circle. The result of processing 12, 13, 14, and 15 is stored.

Meanwhile, FIG. 3B illustrates a method of reusing the A buffer memory 210 for two chroma 8x8 blocks and chroma components. Referring to FIG. 3B, the result values for the 16 and 17 edges marked with the left circle of the upper chroma are overwritten with the result values for the 20 and 21 edges marked with the circle. That is, the A buffer memory 210 containing L4 and L5 contains 16 and 18 blocks after performing 16 and 17 edges, and contains 17 and 19 blocks after performing 20 and 21 edges.

4 is a diagram illustrating a moving direction of data stored in the A buffer memory according to the present invention.

Referring to FIG. 4A, a result value of four blocks of the rightmost column of the previous macroblock is stored in the A buffer memory 210 at the start of filtering of the current macroblock. After the filtering of the first column of the current macroblock is performed, the result value of the first column filtering is stored in the A buffer memory 210.

As the remaining columns of the current macroblock are filtered, the A buffer memory 210 temporarily stores four block pixel values in the second column, four block pixel values in the third column, and four block pixel values in the fourth column.

When filtering the first column of the next macro block after filtering of the current macro block is completed, the temporarily stored fourth column four block pixel values are referenced, and this operation is repeated during the filtering process of all the macro blocks.

Meanwhile, the method of reusing the A buffer memory 210 for the two chroma 8x8 blocks and the chroma component is also substantially the same as the reuse of the A buffer memory 210 in the luma block described above, and thus a detailed description thereof will be omitted.

The deblocking filter 200 using two horizontal and vertical filters 260 and 270 using parallel processing has a larger amount of computation per unit time than the conventional deblocking filter using one filter. Therefore, the deblocking filter capable of processing two horizontal and vertical filters 260 and 270 in parallel can perform real-time processing of a high resolution image even at a lower operating frequency than the conventional method.

However, in a deblocking filter having two horizontal and vertical filters 260 and 270, the horizontal and vertical filters 260 and 270 are implemented as a filter operation unit having the same structure. In order to do this, a transpose process is essential. This transposition process is independent of the actual filtering operation, but involves additional temporary storage, such as a transpose buffer, and a significant amount of power consumption.

However, in the present invention, using the pixel data input method having a different structure, the pre-matrix calculation process is eliminated by using a horizontal filter and a vertical filter operation unit having different pipelined structures.

In order to perform deblocking filtering, the deblocking filter 200 of FIG. 2 reads data to be filtered from the first external memory. The data bus aligns row- or column-oriented data and loads them all at once. Therefore, in the case of the horizontal filter 260, the pixel data of one line which is loaded is directly received as an input in the first stage of the pipeline.

In the conventional general configuration, the input pixel data is output through the pipeline stage, and the filtered pixel data is output at the same time in the final stage of the pipeline. The horizontal filter 260 of the present invention forwards the pixels that can be generated first without passing through the final stage of the pipeline in the pipeline stage to the vertical filter in each stage as the filtered result is generated.

That is, four blocks are described on the right side of the P block of the horizontal filter 260 of FIG. 2, which means that the four blocks output the value directly to the MUX at the stage where the result value is generated rather than the final stage.

5 is a diagram illustrating a pixel input and processing method of a vertical filter according to the present invention.

Unlike the existing filter, the vertical filter 270 has a structure of receiving and processing a pair of pixels at each stage of the pipeline. The left block is defined as a P block and the right block is defined as a Q block based on the vertical edges between the blocks. In addition, the upper block is referred to as the P block and the lower block as the Q block based on the horizontal edge.

A pair of pixels constitutes a pair of pixels having a distance from the same edge in the order of the distance from the near to the edge around the edge located between the P block and the Q block. That is, the pixel pair is composed of P3 and Q3, P2 and Q2, P1 and Q1, and P0 and Q0.

The operation is performed by inputting the result of the operation performed in the pixel pair received at each stage of the pipeline of the vertical filter 270 or the previous stage. Specifically, in the present invention, the pipelines of stages 0 to 3 receive and process each pixel pair as inputs, while the next stage receives and processes input values and input pixel pair data calculated in the previous stage. From the pipeline of the fourth stage, it is configured to process the operation with all pixel values and obtained intermediate operation result values.

In the conventional method, when filtering based on the vertical edge, the filter unit horizontally filters 1 × 4 horizontal rows of pixels belonging to the P block and writes the result to the 4 × 4 pre-buffer block. After performing all four 1 × 4 lines of the P block in order to obtain all 4 × 4 filtering results for the P block, they are output as 4 × 1 vertical one-pixel pixel data, respectively. A temporary buffer is used for temporary storage in the column direction for future vertical filtering.

This prematrix operation is a secondary process that is not defined in the deblocking operation presented in the H.264 / AVC standard. In the vertical filter 270 of the present invention, the input to be transposed is used to output the output of the horizontal filter 260 in this manner. The pre-operation required to solve the simple MUX input by pixel pairs as the input of the vertical filter and convert the output of the final filtered result into the sorting method used on the bus is different from the conventional method. Allows you to perform prefix operations in line step registers.

Although the present invention has been described in detail through the representative embodiments, those skilled in the art to which the present invention pertains can make various modifications without departing from the scope of the present invention. Will understand. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined by the claims below and equivalents thereof.

1 is a block diagram of a video codec encoder in units of typical macroblocks.

2 is a block diagram illustrating a deblocking filter according to an embodiment of the present invention.

3 illustrates an edge filtering processing sequence according to the present invention.

4 is a diagram illustrating a moving direction of data stored in an A-buffer memory according to the present invention;

5 illustrates a pixel input and processing method of a vertical filter in accordance with the present invention.

<Description of the symbols for the main parts of the drawings>

200: deblocking filter 210: A buffer memory

220: B buffer memory 230: external buffer memory

240: boundary strength determination unit and threshold generation unit

250: pipeline controller 260: horizontal filter

270 vertical filter 281 first forwarding means

282: second forwarding means 283: third forwarding means

Claims (8)

In the deblocking filter, An A buffer memory for storing column direction data of a macro block; A B buffer memory for storing row direction data of a macro block; A horizontal filter performing horizontal filtering on vertical edges in at least one pipe step by using column direction data stored in the A buffer memory, and outputting the result data in a corresponding step; And And a vertical filter which receives the row direction data stored in the B buffer memory and the result data output by the horizontal filter and performs vertical filtering on horizontal edges. The method of claim 1, The A buffer memory, And a pixel data corresponding to the rightmost column of the left macroblock of the macroblock to be processed at the start of filtering the macroblock to be processed at present. The method of claim 1, The B buffer memory, And at the start of filtering the macroblock to be processed, storing pixel data corresponding to the lowest row among the upper macroblocks of the macroblock to be processed. The method of claim 1, First forwarding means for re-inputting the horizontal filter result data into the horizontal filter; Second forwarding means for inputting the horizontal filtering result data into the vertical filter; And And a third forwarding means for re-inputting the vertical filter result data into the vertical filter. The method of claim 4, wherein The horizontal filter, In the pipeline consisting of at least one or more stages, the filtering operation is performed using the result data re-input from at least one of the A buffer memory, the first forwarding means or a memory located outside the deblocking filter. Deblocking Filter. The method of claim 4, wherein The vertical filter, In at least one of the pipeline stages, one pixel pair data is received from the horizontal filter or the B buffer memory and processed in the input pixel pair data or previous pipeline stages and re-processed through the third forwarding means. A deblocking filter, wherein the filtering operation is performed using the input result data. The method of claim 1, The result data of performing vertical filtering on the row direction data stored in the B buffer memory and the result data output by the horizontal filter are input, and the result data of performing vertical filtering on the horizontal edge as the data of the row direction line. And a buffer for transposing. The method of claim 7, wherein And said buffer is implemented as a pipeline register of said vertical filter.

Images