KR20050121627A - Filtering method of audio-visual codec and filtering apparatus thereof - Google Patents

Abstract

The present invention relates to a video codec, and more particularly, to a method for filtering in a video codec and a filtering device thereof. A filtering method of a video codec according to the present invention may include: determining a filtering region in units of macro blocks; Further including a predetermined area of the boundary portion between the macroblock determined as the filtering area and the previous macroblock adjacent to the filtering area; And performing causal de-blocking filtering on the filtering region in a chronological order.

Accordingly, by modifying the non-causal loop filter employed in the VC9 and further including a predetermined buffer, blocking artifacts can be efficiently prevented without increasing memory bandwidth or frame delay. You can filter.

Description

Filtering method of audio-visual CODEC and filtering apparatus

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video codec and hereinafter referred to as a codec, and more particularly, to a method for filtering in a video codec and a filtering device thereof.

Microsoft recently submitted a video compression standard, VC9, to the Society of Motion Picture and Television Engineers (SMPTE), one of the international standards bodies. Currently, the VC9 standard is being actively reviewed, and it is expected to be adopted as an international standard after some modification.

In addition to MPEG-2, MPEG-4, and H.264, which are already adopted and widely adopted as video compression standards, VC9 is expected to become a representative video compression standard. While the VC9 is close to 80% of the compression efficiency compared to the most efficient H.264, the implementation is estimated to have a high complexity-to-performance ratio of 60% of H.264. Of course, it is known to have an advantage in terms of image quality compared to MPEG-2 or MPEG-4.

According to the above-described video compression standards, digital video is encoded and decoded on a block basis. In this case, in the digital image reproduced through the decoding process, a boundary line of a block may be visually distinguished. This phenomenon is called blocking artifact, and a loop filter is used to remove the blocking phenomenon. That is, the loop filter refers to a filter inserted into a coding loop of a coder and a decoder to deblock the blocking phenomenon.

Hereinafter, the decoding process of the VC9 will be described in more detail.

1 is a block diagram of a VC9 decoder.

Referring to FIG. 1, a VC9 decoder includes a bitstream parsing unit, an inverse variable length coding unit, an inverse quantization unit, an inverse transforming unit, and a motion. A motion compensation unit, an overlap smoothing and loop filtering unit, an intensity compensation and range scaling unit, and a memory. In particular, the present invention relates to overlap smoothing and loop filters of the above-described decoder components.

When OVERLAP and LOOPFILTER are set to 1 in the VC9 decoder, the overlap smoothing and loop filtering process is performed for the reconstructed picture that has undergone the inverse VLC, inverse quantization, and inverse transform parts as shown in the figure. Can be performed according to. This filtering process should be applied before the reconstructed picture is used as a reference frame for motion compensation. Since the reconstructed image is used as a reference frame for motion compensation through the filtering process, the decoder must strictly perform the filtering process defined in the standard. In general, since a picture is composed of several slices, overlap smoothing and loop filtering for each slice are performed independently.

Hereinafter, the loop filtering will be described in detail.

The main purpose of loop filtering is to eliminate or reduce the blocking that occurs at block boundaries due to quantization. At this time, since the I picture is converted into 8x8 block units, filtering is performed in 8 pixel units vertically and horizontally. That is, loop filtering is performed every 8 pixels, 16 pixels, and 24 pixels.

Meanwhile, in the case of a P picture, a block boundary may occur vertically or horizontally every 4 pixel units according to any one of 8x8, 8x4, and 4x8. Therefore, loop filtering is performed in units of four pixels vertically and horizontally. That is, filtering is performed every 4 pixels, 8 pixels, and 12 pixels.

The loop filtering process is described in more detail as follows.

1) In an I picture Loop Filter

In the I picture, deblocking filtering to remove the blocking phenomenon is performed at all 8x8 block boundaries. 2 is a diagram illustrating pixels of a horizontal / vertical block boundary loop filtered in an I picture.

Referring to Fig. 2, the upper left portion of the picture composed of each element of YCbCr is shown. In the figure, the cross mark (+) represents a pixel and the concentric circles (o) represent a filtered pixel. As shown in the figure, the top horizontal line and the first vertical line on the left are not filtered. Also, although not displayed, the bottom one horizontal line and the last one vertical line are not filtered. That is, for one frame, assuming that N is the number of 8x8 blocks horizontally and M is the number of 8x8 blocks vertically, (7,8), (15,16) .. ((N -1) * 8-1, (N-1) * 8) Only horizontal lines are loop filtered. Also, only the (7, 8), (15, 16) ... ((M-1) * 8-1, (M-1) * 8) th vertical lines are loop filtered in the vertical direction.

Furthermore, the order of loop filtering is also very important. That is, for all of one frame, after all horizontal boundary lines are filtered, all vertical boundary lines are filtered out. Therefore, in order to filter one frame, it is necessary to filter on all boundary lines in the horizontal direction, store the result in memory, and then filter all boundary lines in the vertical direction. This non-causal filtering method requires a lot of memory bandwidth to filter a single frame and waits for both horizontal and vertical filtering to finish. May occur.

2) in a P picture Loop Filter

The P picture may be blocks in Intra mode or in Inter mode. Intra mode blocks always use only 8x8 transforms, so only 8x8 block boundaries are loop filtered. On the other hand, blocks in inter mode use 8x8, 8x4, 4x8 or 4x4 inverse transforms to decode residual errors. Thus, depending on the situation of neighboring blocks, the boundary between the current block and the neighboring block may or may not be loop filtered. Which block boundary 8x8 or subblock boundary 8x4, 4x8, or 4x4 is loop filtered is determined according to the following criteria.

(1) The boundaries between encoded subblocks (8x4, 4x8 or 4x4) in an 8x8 block are unconditionally loop filtered.

(2) If a block (or subblock) and a neighboring block (or subblock) have the same motion vector and both do not have residual errors, the block boundary is not filtered. Except in this case, it is loop filtered.

3 is a diagram illustrating examples of block boundaries that are loop filtered in a P picture.

Referring to FIG. 3, it is shown when loop filtering is performed at the boundary between adjacent blocks. Assuming that the motion vectors of the blocks are the same (unconditionally perform loop filtering if different), the blocks (or subblocks) shown in gray are coded blocks (or subblocks) with transform coefficients present. Blocks (or sub-blocks) shown in white are blocks that do not have any transform coefficients. The portions marked with thick lines represent filtered boundaries, and the portions marked with thin lines represent unfiltered boundaries. According to the above two criteria, the boundaries of the block to be filtered or the boundaries of the sub block are determined. The examples shown in FIG. 3 show only cases where two blocks are horizontally neighboring, but the same method applies even when two blocks are vertically neighboring.

4 is a diagram illustrating pixels of a block horizontal / vertical boundary that are loop filtered in a P picture.

Referring to FIG. 4A, horizontal boundaries capable of loop filtering are illustrated. Since 8x8 and 8x4 horizontal boundaries are filterable, every fourth and fifth, eighth and ninth, twelfth and thirteenth lines may be loop filterable boundary pixels.

Meanwhile, referring to FIG. 4 (b), loop-filterable vertical boundaries are illustrated.

Since 8x8 and 4x8 vertical boundaries are loop-filterable, every fourth and fifth, eighth and ninth, 12th and 13th lines may be loop-filterable boundary pixels. Like the I picture, the first and last vertical / horizontal lines are not loop filtered.

Furthermore, in the case of P pictures, the order of loop filtering is very important. First, horizontal boundaries of all 8x8 blocks for the entire frame are loop filtered in order from the top line. The filtering result of the horizontal boundaries of the 8x8 block is stored in memory. Second, all 8x4 horizontal boundary lines for the entire frame are also loop filtered in order from the top line. The filtering result is also stored in memory. Third, in the vertical direction, all 8x8 vertical boundaries for the entire frame are loop filtered starting from the left and the result is stored in memory. Finally, all 4x8 block boundaries for the whole frame are also loop filtered starting from the left and the result is stored in memory, i.e. for one frame, after all horizontal boundary lines are filtered in two steps, Loop filtering in two steps for all vertical boundary lines. Due to such a non-causal filtering method, a frame delay may occur because four loop filtering processes are required to filter one frame. Also, a large amount of memory bandwidth is required to loop filter one frame.

3) B In the picture Loop Filter

The loop filtering for the B picture is the same as that for the I picture described above.

In more detail, a process of performing loop filtering at a boundary between each block (or subblock) is described.

5 is a diagram illustrating an example of pixels that are loop filtered at a boundary between adjacent blocks.

Since the minimum number of consecutive pixels to be loop filtered in the vertical direction or the horizontal direction is 4, and the total number of pixels in the vertical or horizontal line is a multiple of 4, the loop filtering process may be performed in units of 4 pixels. Referring to FIG. 5A, for example, if the boundary between two blocks is composed of eight pixel pairs, the eight pixels are divided into two four pixel segments. In each four-pixel segment, the third pixel pair, denoted by X, is loop filtered before the other pixels, as shown. In addition, the filtering of the remaining three pixel pairs is determined according to the loop filtering result. When the comparison of the displayed third pixel pairs does not cause blocking and loop filtering is not necessary, the loop filtering may be omitted for the remaining three pixel pairs.

Referring to FIG. 5B, a loop filtering process performed on a pair of third pixels is illustrated. The pixels P4 and P5 are interchangeable pixels in the filtering process. Based on the pixels P4 and P5, a predetermined compensation value is obtained for a weighted average between the pixels P1 to P4 on the left side and the pixels P5 to P8 on the right side, and added and subtracted to each pixel P4 and P5, respectively. It is possible to eliminate or reduce the occurrence of a sudden change in the pixel value at the boundary of the block. In this case, as shown in FIG. 5 (a), if the third pixel pair needs to be filtered, the remaining three pixel pairs are also filtered. Otherwise, the loop filtering on the remaining three pixel pairs is omitted and the next Skip to the 4-pixel segment.

Meanwhile, the overlap smoothing will be described below in detail.

6 is a diagram illustrating an example of overlap smoothing.

If one of the sequence parameters, OVERLAP, is set to 1, a filtering operation called overlap smoothing is conditionally performed on the edge between two adjacent Intra blocks for Y as well as CbCr. Overlap smoothing (also referred to as an overlap transform) is performed immediately before the loop filtering described above in FIGS. 2 to 5. Overlap smoothing is a type of block-based transformation that exchanges information at block boundaries. Performing well designed overlap smoothing can minimize blocking noise.

Referring to FIG. 6, a portion of a P frame including I blocks is shown. Each pixel indicated by the rectangle may be Y or CbCr. I blocks are marked with hatched portions, and P blocks are marked with dots. The overlap smoothing part is filled with dots, and the overlap smoothing is performed on two pixels in each direction of the block boundary. The 2x2 pixels inside the circled portion are pixels that are filtered in both directions.

First, overlap smoothing is performed on four consecutive pixels in the horizontal direction at the vertical edge boundary. In the figure, it is indicated by (a0, a1, a2, a3). a0 and a1 represent two pixels on the left side based on the block boundary, and a2 and a3 represent two pixels on the right side. Next, overlap smoothing is also performed on the four pixels p0, p1, p2, and p3 continuous in the vertical direction at the horizontal boundary edge. That is, the horizontal pixels a0, a1, a2, a3 for the vertical edge are first smoothed overlap, and the vertical pixels p0, p1, p2, p3 for the horizontal edge are overlap smoothed. The intermediate result of the filtering is temporarily stored in memory.

On the other hand, it is determined whether or not overlap smoothing is performed on I, P, or B frames according to a predetermined condition. Specific conditions under which overlap smoothing is performed are described in detail in the VC9 related materials, and thus descriptions thereof will be omitted. If the condition for overlap smoothing is not met, overlap smoothing is omitted. For example, overlap smoothing may be omitted when there is only an inter block. However, for convenience of description, hereinafter, the case where both overlap smoothing and loop filtering occur will be described. However, overlap smoothing is not always performed if performed under a predetermined condition.

The overlap smoothing and loop filtering process to remove the blocking phenomenon of VC9 described above is performed by performing the loop smoothing on the horizontal boundary and the vertical boundary after performing the overlap smoothing on the vertical boundary and the horizontal boundary in the frame unit or the slice unit. As a result, the result of the previous horizontal filtering must be stored in memory. Therefore, a large memory bandwidth is required, which causes unnecessary frame delay and increases frame memory. Due to such non-causality of VC9 overlap smoothing and loop filtering, hardware implementation of the codec is difficult.

Accordingly, an object of the present invention is to provide a method and an filtering device for efficiently filtering without increasing memory or bandwidth by improving non-causal overlap smoothing and loop filtering in a video codec.

According to an aspect of the present invention, there is provided a filtering method of a video codec, the method comprising: determining a filtering region in units of macro blocks; Further including a predetermined area of the boundary portion between the macroblock determined as the filtering area and the previous macroblock adjacent to the filtering area; And performing causal de-blocking filtering on the filtering area further comprising a predetermined area of a boundary portion between blocks in a temporal order. do.

The deblocking filtering may be performed by deblocking filtering in a horizontal direction with respect to one filtering area, then deblocking filtering in a vertical direction, and deblocking filtering in a causal manner that is passed to the next filtering area.

In addition, the predetermined region of the inter-block boundary portion may include a plurality of pixels of at least four pixel units.

The deblocking filtering includes at least one of overlap smoothing and loop filtering, and after performing overlap smoothing according to a predetermined condition in macroblock units, it is preferable to perform loop filtering.

In addition, in the overlap smoothing, the overlap smoothing is preferably performed by using a value of at least two pixels in both directions in a symmetrical manner between the boundary parts between the blocks, and after overlapping the 8 × 8 block boundary parts in the horizontal direction, the vertical direction is performed. It is preferable to overlap smooth the 8 × 8 block boundary.

In the loop filtering, it is preferable to perform loop filtering by using a value of at least 4 pixels in both directions in a symmetrical manner between the boundary portions between blocks.

In the loop filtering, it is preferable to loop filter the 8x8 block boundary part in a horizontal direction, loop filter the 8x4 block boundary part, and loop filter the 8x8 block boundary part in a vertical direction, and then loop filter the 4x8 block boundary part. Do.

In particular, the video codec preferably follows the VC9 standard of the Society of Motion Picture and Television Engineers (SMPTE).

On the other hand, according to another field of the present invention, the above technical problem, the filtering device of the video codec, the buffer unit for storing the filtering area in units of macro blocks; A row buffer and a column buffer for storing a predetermined area of a boundary portion between the macro block stored in the buffer unit and the previous macro block adjacent to the macro block; And a filtering unit performing causal de-blocking filtering on the filtering area stored in the buffer unit, the row buffer, and the column buffer in chronological order. do.

Hereinafter, with reference to the accompanying drawings looks at a preferred embodiment according to the present invention.

7 is a block diagram of a filtering device according to a preferred embodiment of the present invention.

Referring to FIG. 7, the filter according to the present invention performs filtering on a macro block basis, as compared to the conventional filter of VC9, which filters on a frame basis. To this end, a separate row buffer 630 and a column buffer 640 for temporarily storing pixel information of the boundary of the previous macro block are further included.

That is, the filtering device according to the present invention includes a filtering unit 610 for removing a blocking phenomenon, a buffer unit (L_BUF) 620 for storing the data to be filtered and the filtering result, and pixel information of the boundary of the previous macro block. A row buffer ROW_BUF 630 and a column buffer COL_BUF 640 are stored.

The VC9 codec uses a 16x16 macroblock as a basic processing unit. Therefore, the overlap smoothing and loop filtering of the present invention are also configured to be processed in units of 16 × 16 macroblocks. That is, after overlap smoothing is performed for each macro block, loop filtering is performed for each macro block. The buffer unit 620 is a 16x16 sized buffer, and stores a filtering intermediate result and a final result of the 16x16 macroblock. The row buffer 630 is a buffer having a size of at least 20 × 8, and stores the horizontal pixels of the boundary portion of the block located at the top of the macro block currently being filtered, and stores the intermediate and final results of the filtered pixels. The column buffer 640 has a minimum size of 8 × 16 and stores vertical pixels of the boundary portion of the block located to the left of the macro block currently being filtered, and stores intermediate and final results of the filtered pixels.

In order to explain the operation of the filtering apparatus according to the present invention, one frame composed of four macro blocks in a horizontal direction and three MBs in a vertical direction is assumed. 8 is a diagram illustrating a frame in which overlap smoothing and loop filtering are performed according to the present invention.

Referring to Fig. 8, each macro block is numbered. In order to process overlap smoothing and loop filtering on a macroblock basis according to the present invention, a filtering operation of four cases of macroblocks will be described. In particular, the filtering operation in the CASE 1 region as the filtering region of the macro block MB1 and the CASE 2 region as the filtering region of the macro block MB2 will be described in the horizontal direction. In addition, the filtering operation in the CASE 3 region as the filtering region of the macro block MB5 and the CASE4 region as the filtering region of the macro block MB6 will be described in the vertical direction.

First, FIG. 9 is an embodiment illustrating an order of filtering in block units according to the present invention in the CASE 1 region illustrated in FIG. 8.

When motion compensation for one macro block (see FIG. 1) is completed, overlap smoothing is performed on the macro block, and loop filtering is then performed on the macro block. For CASE 1, we show the overlap smoothing and loop filtering process for macro block MB1.

9 and 7, the motion compensated macro block MB1 is stored in the buffer unit L_BUF 620. First, it is determined whether or not to perform overlap smoothing on the macroblock according to the above-described predetermined condition. If the overlap smoothing does not correspond to a condition, the overlap smoothing process may be omitted, and the loop filtering may be immediately performed. However, a case where overlap smoothing is required for convenience of explanation is described as an example.

FIG. 9A illustrates the filtering subject region in the case of CASE 1. FIG. Since the macro block MB1 is the first macro block of the first slice, the row buffer 630 and the column buffer 640 are empty. The boundary portion of MB1 and MB2 is filtered at the time of processing of MB2.

First, as shown in FIGS. 9B and 9C, overlap smoothing is performed. Referring to FIG. 9B, pixels displayed in black in a portion indicated by a thick dotted line are used for overlap smoothing. Sixteen rows consisting of four pixels represented by (p1, p0, q0, q1) are sequentially inputted row by row to the latches p1, p0 and q0, q1 of FIG. That is, the two pixels on the left are input to p1 and p0, and the two pixels on the right are input to q0 and q1. The pixels of the latch are input to the filtering unit 610 in one cycle. In this module, overlap smoothing is performed. Output values of the filtering unit 610 that performs overlap smoothing are input to P1, P0, Q0, and Q1, respectively. The P1, P0, Q0, and Q1 values update the corresponding values of the buffer unit L_BUF 620 again. The actual update is made only at the boundary where the condition is met as shown in FIG. 9 (b).

Meanwhile, FIG. 9C illustrates a process in which overlap smoothing is performed at a block boundary in a horizontal direction. The basic operation is the same as in the case of FIG. The processing of the pixels in the right four columns in the figure is processed together in the processing of the macro block MB2. However, it is also possible to process the processing of at least the right two columns of pixels together in the processing of MB2.

When the overlap smoothing for the MB1 macroblock is completed, loop filtering is performed immediately. Referring to FIG. 9 (d), a loop filtering process of an 8 × 8 horizontal block boundary is illustrated. Pixels at the horizontal 8x8 block boundary in the macro block MB1, indicated by black concentric circles, are loop filtered. The boundary portion between the macro block MB1 and the macro block MB5 at the bottom thereof is loop filtered during the filtering process of the macro block MB5. Further, the right four pixel pairs of the macro block MB1 are loop filtered in the filtering process of the adjacent macro block MB2. Therefore, the pixels in the portion indicated by the thick dotted line are the pixels used for the loop filtering of the black pixels.

The portions indicated by the thick dotted lines are inputted into the column vectors, which are all 12 vertical pixel pairs, in turn into p0, p1, p2, p3, q0, q1, q2, and q3 latches shown in FIG. Each column consists of eight pixels. That is, as shown in the figure, the upper 4 pixels are input to p3, p2, p1, p0, and the lower 4 pixels are input to q0, q1, q2, q3.

Also, as indicated at the top of FIG. 9 (d), the order in which 12 column vectors are input is shown. That is, as described above with reference to FIG. 5, the third pixel pair of the four pixel segments is first subjected to loop filtering, so that the 1, 5, and 9 column vectors are first loop filtered at each of the 4 pixel segments. According to the filtering result, it is determined whether loop filtering of the remaining pixels of the four pixel segments is performed. Each pixel input to the p3, p2, p1, p0 and q0, q1, q2, and q3 latches is input to the filtering unit 610 shown in FIG. 6 in a cycle and subjected to loop filtering. The actual loop filtering is performed as described above with reference to FIG. 5. The loop filtering process may be applied to any filtering method including the filtering method of VC9. Of the filtering results, only pixels corresponding to the boundary of the block indicated by the black concentric circles are input to P0 and Q0 of FIG. 7. When it is determined that filtering is necessary, that is, when the boundary of the block is determined to be a part in which a blocking phenomenon occurs, not an outline of the image, the corresponding value is updated in the buffer unit 630 (L_BUF) with values of P0 and Q0.

In the same manner, the filtering processing of the fifth, sixth, seventh and eighth column vectors is performed, and the filtering processing of the next 9th, 10th, 11th and 12th column vectors is performed. As described above, the loop filtering process is performed on each column vector included in the target region, but the update of the actual memory is performed only for the pixels of the boundary portion of the block. For example, an update is performed only at an 8x8 boundary for an I picture or at a boundary between encoded blocks for a P picture.

Meanwhile, referring to FIG. 9E, a filtering process of an 8 × 4 horizontal block boundary is illustrated. Pixels at the horizontal 8x4 block boundary in the macro block MB1, indicated by black concentric circles, are loop filtered. The pixels in the part indicated by the thick dotted line are the pixels used for the loop filtering of the black pixels. The actual loop filtering process is the same as described above with reference to FIG. 9 (d). In the case of VC9, when the current macro block is intra, loop filtering on pixels of an 8 × 4 block horizontal boundary may be omitted.

After the loop filtering on the pixels of the horizontal block boundary for the macro block MB1 is finished, the loop filtering is performed on the pixels of the vertical block boundary for the macro block MB1. First, as shown in FIG. 9 (f), first, pixels of a vertical 8 × 8 block boundary in the macro block MB1 are loop filtered. At this time, the filtering process for the lower 8 pixel rows of the block boundary is performed at the processing of the macro block MB5. In addition, the four right pixel pairs of the macro block MB1 are filtered during the filtering process of the adjacent macro block MB2. Therefore, the pixels in the portion indicated by the thick dotted line are the pixels used for the loop filtering of the black pixels.

Next, as shown in Fig. 9G, pixels of the vertical 4x8 block boundary in the macro block MB1 are loop filtered. Although the vertical and horizontal directions are changed, the actual loop filtering process is the same as the description of FIGS. 9 (d) and 9 (e) with respect to the horizontal direction. In the case of VC9, when the current macro block is intra, loop filtering on pixels of a 4 × 8 block vertical boundary may be omitted.

The loop filtered result is temporarily stored in the buffer unit 620 and then written to the external frame memory. In the case of the macro block MB1, there is only a change in the pixels in the region shown in Fig. 9A, so that the data of the region shown in Fig. 9A is written to the external frame memory. 9 (h) shows contents to be recorded in the frame memory outside the filtering result.

Then, the right 8x16 pixels of the macro block MB1 corresponding to the boundary of both blocks are loaded into the column buffer 640 for the overlap smoothing and loop filtering processing of the next macro block MB2. Further, 16x16 pixel values of the macro block MB2 to be overlapped smoothed and loop filtered are loaded into the buffer unit 620 (see FIG. 7).

As described above, all overlap smoothing and loop filtering processes are performed by segments of 4 pixels. In addition, after horizontal overlap smoothing is completed for one macro block, vertical overlap smoothing is performed. When overlap smoothing is completed, loop filtering is performed in the horizontal and vertical directions.

10A and 10B illustrate an exemplary embodiment of overlap smoothing and loop filtering in block units according to the present invention in the CASE 2 region illustrated in FIG. 8.

10A and 10B, an overlap smoothing and loop filtering process of the macro block MB2 is illustrated. (a) indicates an overlap smoothing and loop filtering target area. In addition to the macro block MB2, 8 × 16 pixels temporarily stored in the column buffer during overlap smoothing and loop filtering of the macro block MB1 are included in the target area.

First, as shown in (b) to (d), overlap smoothing is performed in the horizontal direction and the vertical direction with respect to the 8x8 block boundary. The detailed process is as described above with reference to FIGS. 9B and 9C. In particular, the four pixel segments on the left side of (d) have already been processed in the overlap smoothing process of the MB1 macroblock in CASE 1, and the four pixel segments on the right side of (d) are processed in the overlap smoothing process of the next MB3 macroblock. It is not included in the overlap smoothing process range.

When overlap smoothing is completed for the area indicated in (a), loop filtering is performed next. The pixels to be filtered at the horizontal 8x8 block boundary are indicated by black concentric circles in (e). The lower 8x8 pixels and the four pixels on the right side are processed during the filtering of the next slice or the next macro block, and only the pixels of the portion indicated by the thick dotted line are used for filtering the pixels indicated by the black concentric circles. That is, all 16 column vectors consisting of eight pixel columns are sequentially input to the p and q latches shown in FIG. 7 and subjected to loop filtering. The detailed loop filtering process is the same as described above with reference to FIGS. 9D to 9H. Next, as shown in (f) of FIG. 10B, a loop filtering process for the 8 × 4 horizontal boundary is performed. A description of the loop filtering process is as described above.

Meanwhile, the 8x8 vertical boundary filtering and the 4x8 vertical boundary filtering in the vertical direction are illustrated in FIGS. 10B (g) and (k). Referring to (g), eight row vectors in the area indicated by the dotted line are input and filtered in the order shown on the left. That is, eight pixels of each vector are input to the latches p3, p2, p1, p0, q0, q1, q2, q3, respectively, and filtered. The loop filtering process is the same as described with reference to FIG. 5 for each of the four pixel segments.

Next, 4x8 vertical boundary pixels are filtered as shown in FIG. 10B (h). Similarly, the processing for the remaining 8x8 vertical borders and 4x8 vertical borders is shown in Figures (i) and (j). (k) indicates the content to be recorded in the frame memory outside the filtering result.

Then, the right 8x16 pixels of the macroblock MB2 corresponding to the boundary of both blocks are loaded into the column buffer 640 for the filtering process of the next macroblock MB3. Further, 16x16 pixel values of the macro block MB3 to be filtered are loaded into the buffer unit 620 (see FIG. 7). In this manner, the filtering process is performed on the first slice from the macro blocks MB1 to MB4 in the horizontal direction.

11A and 11B illustrate an exemplary embodiment of overlap smoothing and loop filtering in block units according to the present invention in the CASE 3 region illustrated in FIG. 8.

11A and 11B, an overlap smoothing and loop filtering process for MB5, which is the first macro block of the second slice, is illustrated. After the filtering process of the macro block MB4, the lower 16 × 8 pixels of the macro block MB1 are loaded into the row buffer 630. In addition, 16x16 data of the macroblock MB5 to be filtered is loaded into the buffer unit 620.

The overlap smoothing process for the pixels at the horizontal and vertical 8x8 boundaries of the macro block MB5 is shown in Figs. 11A to 11D. Only the target pixels are changed, and the overlap smoothing operation principle is the same as in the above-described case of FIG. 9 or FIGS. 10A and 10B.

In addition, the loop filtering process for the pixels of the horizontal direction 8x8 boundary and the 8x4 boundary of the macroblock MB5 is illustrated in FIGS. 11A and 11B and FIGS. Again, the filtering operation principle is the same as in the above-described case of FIGS. 9, 10A, and 10B except that only the target pixels are changed. In addition, the filtering process for pixels of the vertical 8x8 boundary and the 4x8 boundary is shown in FIGS. 11B (i) and (j).

The overlap smoothing and loop filtering results are shown in FIG. 11B (k), and the filtered result values are written to the external memory.

The buffer unit 620, the row buffer 630, and the column buffer 640 are updated to filter the next macro block. Note that the pixels updated at this time are not the original pixel values before filtering, but the pixels of the filtered result.

12A to 12C illustrate an exemplary embodiment of overlap smoothing and loop filtering in block units according to the present invention in the CASE 4 region illustrated in FIG. 8.

12A to 12C, (b) to (e) illustrate overlap smoothing processes in horizontal and vertical directions. Also, (f) to (i) show a loop filtering process for pixels of a block boundary in a horizontal direction, and (j) to (m) show a loop filtering process for pixels of a block boundary in a vertical direction. do. (n) indicates the overlap smoothing and loop filtering results, and pixels in the thick dotted line are written to the external frame memory. The filtering process of the remaining macroblocks is all performed on the same principle.

As described above, the filtering device according to the present invention is characterized in that the overlap smoothing and loop filtering is performed in macro block units as compared to the conventional filter of VC9, which performs overlap smoothing and loop filtering in units of frames. To this end, a separate row buffer 630 and a column buffer 640 for temporarily storing pixel information of the boundary of the previous macro block are further included.

That is, after performing overlap smoothing on horizontal 8x8 and vertical 8x8 boundary parts according to a predetermined condition in macroblock units, loop filtering of horizontal 8x8 and 8x4 block boundary parts is performed, and 8x8 and 4x8 block boundary parts in a vertical direction. Perform loop filtering of. The pixels of the boundary portion to be used for overlap smoothing and loop filtering of the next macro block among the performed results are stored in the row buffer or column buffer and used for overlap smoothing and loop filtering of the next macro block.

Accordingly, causal filtering, which filters in macroblock order according to time order, becomes possible. Therefore, the memory bandwidth required in comparison with the filtering processing in units of frames can be reduced, and frame delay can be prevented.

The above-described embodiment is a macroblock-based filtering processing method (method 1), which is suitable for reducing the memory bandwidth, but the control is complicated. As another embodiment according to the present invention, there is a method (method 2) in which the memory bandwidth is slightly increased compared to the method 1, but the control is simplified.

That is, the horizontal filtering process and the vertical filtering process are separated. First, an overlap smoothing and loop horizontal filtering process is completed on a horizontal boundary portion of each macroblock included in one slice. In addition, overlap smoothing and loop filtering are completed for the horizontal boundary for each macroblock included in the next slice. In this way, after the overlap smoothing and loop filtering are completed on the horizontal boundaries of all slices, the overlap smoothing and loop filtering is performed on the vertical boundaries from the first slice. In this way, when the overlap smoothing and loop filtering processing is completed in the horizontal direction and the vertical direction, the result is written to the external frame memory.

For convenience of description, in the above embodiments, the sizes of the buffer unit, the row buffer, and the column buffer are constantly limited, but in actual implementation, the size of the buffer may be flexibly determined. For example, the horizontal size of the buffer unit and the row buffer may be increased up to one slice size. In this case, the bandwidth for loading the data of the row buffer into the memory is eliminated. In other words, instead of increasing the size of the buffer, the memory bandwidth is reduced so that the two can be properly adjusted.

In the above, the filtering method and apparatus in the codec of VC9 have been described, but it can be variously applied to other standards including a loop filter. That is, the above description is only one embodiment of the present invention, and those skilled in the art may implement the present invention in a modified form without departing from the essential characteristics of the present invention. Therefore, it should be interpreted to include various embodiments that are not limited to the examples according to the present invention but within the scope equivalent to those described in the claims.

As described above, according to the present invention, there is provided a filtering method for causal processing according to a temporal order in macroblock units in a video codec, and a filtering device thereof.

In other words, by modifying the non-causal loop filter adopted by the VC9 and further including a predetermined buffer, blocking artifacts can be efficiently filtered without increasing memory bandwidth or frame delay. can do.

1 is a block diagram of a VC9 decoder;

2 illustrates pixels of a horizontal / vertical block boundary loop filtered in an I picture;

3 is a diagram illustrating examples of block boundaries that are loop filtered in a P picture;

4 is a diagram illustrating pixels of a block horizontal / vertical boundary loop filtered in a P picture;

5 is a diagram illustrating an example of pixels that are loop filtered at a boundary between adjacent blocks;

6 is a diagram illustrating an example of overlap smoothing;

7 is a block diagram of a filtering device according to a preferred embodiment of the present invention;

8 is a view showing a frame to be filtered according to the present invention,

FIG. 9 is a diagram illustrating an order of filtering in block units according to the present invention in the CASE 1 region illustrated in FIG. 8;

10A and 10B illustrate an example of an order of filtering in block units according to the present invention in the CASE 2 region shown in FIG. 8;

11A and 11B illustrate an example of an order of filtering in block units according to the present invention in the CASE 3 region shown in FIG. 8;

12A to 12C illustrate an example of filtering in block units according to the present invention in the CASE 4 region illustrated in FIG. 8.

Claims (20)

In the filtering method of the video codec, Determining a filtering area in units of macro blocks; Further including in the filtering region a predetermined region of a boundary portion between the macroblock determined as the filtering region and a previous macroblock adjacent to the macroblock; And And performing causal de-blocking filtering on the filtering area further including a predetermined area of the boundary between the blocks. The method of claim 1, The deblocking filtering may include a deblocking filtering in a horizontal direction with respect to one filtering area, a deblocking filtering in a vertical direction, and a deblocking filtering in a causal manner that is transferred to the next filtering area. . The method of claim 1, The predetermined area of the boundary portion between blocks includes a plurality of pixels of at least four pixel units. The method of claim 1, The deblocking filtering method includes at least one of overlap smoothing and loop filtering. The method of claim 4, wherein The loop filtering is performed after performing the overlap smoothing according to a predetermined condition in macro block units. The method of claim 4, wherein The overlap smoothing method is characterized in that the overlap smoothing using a value of at least two pixels in both directions symmetrically the boundary between the blocks. The method of claim 4, wherein The overlap smoothing may include overlap smoothing the 8 × 8 block boundary portion in the horizontal direction and then overlap smoothing the 8 × 8 block boundary portion in the vertical direction. The method of claim 4, wherein The loop filtering method includes performing loop filtering using at least four pixel values in both directions in a symmetrical manner between the boundary portions of the blocks. The method of claim 4, wherein The loop filtering may include loop filtering the 8x8 block boundary in a horizontal direction, loop filtering the 8x4 block boundary in a horizontal direction, loop filtering the 8x8 block boundary in a vertical direction, and then loop filtering the 4x8 block boundary in a vertical direction. Filtering method. The method of claim 1, The video codec is based on the VC9 standard of the Society of Motion Picture and Television Engineers (SMPTE). In the filtering device of the video codec, A buffer unit which stores the filtering area in units of macro blocks; A row buffer and a column buffer for storing a predetermined area of a boundary portion between the macro block stored in the buffer unit and a previous macro block adjacent to the macro block; And And a filtering unit performing causal de-blocking filtering of the filtering area stored in the buffer unit, the row buffer, and the column buffer in a chronological order. The method of claim 11, The filtering unit may perform deblocking filtering in a horizontal direction with respect to one filtering area, deblocking filtering in a vertical direction, and deblocking filtering in a causal manner to be transferred to the next filtering area. The method of claim 11, The predetermined region of the inter-block boundary portion includes a plurality of pixels of at least four pixel units. The method of claim 11, The deblocking filtering comprises at least one of overlap smoothing and loop filtering. The method of claim 14, The loop filtering is performed after performing the overlap smoothing according to a predetermined condition in units of macro blocks. The method of claim 14, The overlap smoothing is filtering apparatus, characterized in that for performing the overlap smoothing using a value of at least two pixels in both directions symmetrically the boundary between the blocks. The method of claim 14, The overlap smoothing may include overlap smoothing the 8 × 8 block boundary part in the horizontal direction and then overlap smoothing the 8 × 8 block boundary part in the vertical direction. The method of claim 14, In the loop filtering, the loop filtering is performed by using a value of at least four pixels in both directions in a symmetrical manner between the boundary portions of the blocks. The method of claim 14, The loop filtering may include loop filtering the 8x8 block boundary in a horizontal direction, loop filtering the 8x4 block boundary in a horizontal direction, loop filtering the 8x8 block boundary in a vertical direction, and then loop filtering the 4x8 block boundary in a vertical direction. Filtering device. The method of claim 11, The video codec is a filtering device, characterized in that the VC9 standard of the Society of Motion Picture and Television Engineers (SMPTE).