KR101007384B1 - Method for adaptive filtering reflecting features of neighbor blocks - Google Patents

Abstract

PURPOSE: An adaptive filtering method considering the property of an adjacent block which effectively reduces an unnecessary high frequency component in a block boundary is provided to conserve important edge information of the original image by providing an adjacent block by removing the high frequency component of the block boundary. CONSTITUTION: If error generates in one block of an original image, the similarity of the adjacent block is compared with the block(S200). If similarity is greater than a threshold value, the presence of the edge is detected(S204). If the edge does not exist, the high frequency component of the error-generated block is removed(S206). If the edge exists, a directivity filter is applied(S208).

Description

Adaptive filtering method considering characteristics of adjacent blocks {METHOD FOR ADAPTIVE FILTERING REFLECTING FEATURES OF NEIGHBOR BLOCKS}

The present invention relates to an adaptive filtering method. In particular, when an error occurs in one block in an image, the present invention relates to an adaptive filtering method considering characteristics of an adjacent block for filtering an error block in consideration of characteristics of an adjacent block. It is about.

In general, video compression techniques such as JPEG, H.26x, and MPEG have a disadvantage in that a block phenomenon occurs using block-based encoding. The disadvantage of this block phenomenon is that the boundary is generated at the boundary of blocks because the original image is divided into blocks and each block is independently encoded and processed without considering correlation between blocks. Many studies have been conducted to remove such block phenomenon, and there are a scheme used in single layer encoding and a layer encoding method including one base layer and a plurality of enhancement layers.

The most representative of the techniques used in single layer coding is a technique based on the POCS (Projection Onto Convex Sets) technique. POCS technique is to define convex sets and projectors to the sets from the image data and various constraints, and then remove the block phenomenon without damaging the original image through repeated projection to each set. To restore the image. The algorithm using the POCS technique consists of defining two sets of Smoothness Constraint Set (SCS) and Quantization Constraint Set (QCS) and performing repetitive projection on the two sets. SCS is mainly used to remove high frequency components of an image as a limit set for blurring. In this case, it is an important task of designing an SCS algorithm to preserve high frequencies included in an original image other than the high frequency due to a block phenomenon. QCS is a set to prevent excessive blur caused by projection to SCS. It defines conjox set of data section which original image data can have and defines the projector so that data is maintained in this section.

In addition, unlike the POCS scheme, the information related to the initial value setting can be obtained from other layers by using the correlation between layers, and there is a layer encoding method including one base layer and a plurality of enhancement layers. As described above, the hierarchical coding method can increase the efficiency of error concealment by limiting POCS initial values to more similar blocks, and is an error concealment method using correlation between layers.

As such, the method of obtaining an initial value in the existing POCS scheme used in single layer encoding is limited to neighboring blocks of one layer. However, in a layer encoding method including one base layer and a plurality of enhancement layers, such as scalable video coding (SVC), information related to initial value setting can be obtained in another layer. The SVC configures one bit stream to have one image content having various spatial resolution, quality, and various frame rates, so as to fit its own capability in various terminals. A video compression technique that makes it possible to receive and recover a bit stream. In this way, by obtaining information related to initial value setting in other layers, the POCS initial value can be limited to more similar blocks, thereby improving the efficiency of error concealment.

However, in the hierarchical coding method, unnecessary high frequency components are generated at the block boundary in the process of initializing a damaged block as the most similar neighboring block, which is not efficiently removed even after image reconstruction. In addition, when the edge is present in the original image, there is a problem that the high-frequency components are damaged resulting in deterioration of image quality.

SUMMARY OF THE INVENTION The present invention has been conceived to solve the above-described conventional problem, and is an adaptive filter suitable for a POCS-based error concealment technique for removing high frequency components of a block boundary generated when initial values are set during a POCS processing and preserving original image energy as much as possible. Provides a way of designing.

The present invention for achieving the above-described object in the adaptive filtering method in consideration of the characteristics of the adjacent block, comparing the similarity between the block in which the error is generated in the image consisting of a plurality of blocks and the blocks adjacent to the block in which the error occurs If the compared similarity value is not greater than a predetermined threshold, detecting whether an edge exists; if the edge exists, applying a directional filter; if the edge does not exist, bordering the adjacent blocks Filtering the image by removing a high frequency component according to a frequency characteristic of the filter.

As described above, the present invention provides an adaptive filtering method that removes high frequency components of a block boundary in an error concealment method using inter-layer correlation and considers characteristics of adjacent blocks for maintaining the edge of the original image to the maximum. It is effective to preserve important edge information of the original image and effectively reduce unnecessary high frequency components generated at the block boundary.

Hereinafter, with reference to the accompanying drawings will be described in detail the operating principle of the preferred embodiment of the present invention. In the following description of the present invention, detailed descriptions of well-known functions or configurations will be omitted if it is determined that the detailed description of the present invention may unnecessarily obscure the subject matter of the present invention. Terms to be described later are terms defined in consideration of functions in the present invention, and may be changed according to a user, a user's intention or custom. Therefore, the definition should be based on the contents throughout this specification.

The present invention provides an apparatus and method for designing an adaptive filter suitable for a POCS-based error concealment technique in order to remove high frequency components of a block boundary generated during initial setting of a POCS process and to preserve edges in an original image. Therefore, an embodiment of the present invention to be described later will be described in detail with reference to the drawings the adaptive filter device and method in consideration of the characteristics of the neighboring block.

1 is an exemplary diagram showing adaptive filtering suitable for error concealment based on POCS according to an embodiment of the present invention.

As shown, if an error occurs in one block (a) in one image 101 divided into nine blocks, adaptive filtering suitable for error concealment based on POCS according to an embodiment of the present invention is error The generated block is discrete cosine transformed by the DCT transform unit 103 (Discrete Cosine Transform, DCT). Discrete cosine transform is a preprocessing process that converts the sampled image data into frequency components to perform compression. When a discrete frequency component is input through the DCT converter 103, the adaptive filter 105 may determine whether an edge of a block positioned on the left, right, up, or diagonal sides of the block in which an error occurs is present. For example, the directional filter is applied to remove the high frequency component of the block in which the error occurs, or the high frequency removing process is selectively applied according to the frequency characteristic of the neighboring block. For example, if an error occurs in one block divided into nine blocks, the adaptive filter unit 105 may determine the directionality according to whether edges of eight blocks adjacent to the block in which the error occurs are present. A filter is applied to remove high frequency components of an error-prone block, or a high frequency elimination process is selectively applied according to frequency characteristics of neighboring blocks.

The high frequency elimination process according to an embodiment of the present invention includes a method using a neighboring block and a method using a block high frequency elimination assumption. This high frequency removal process according to the present invention will be described later. The IDCT transformer 107 performs an inverse discrete cosine transform process that is an inverse process of the discrete cosine transform performed by the discrete cosine transform unit. do.

2 is a flowchart illustrating an adaptive filtering method suitable for error concealment based on POCS according to an embodiment of the present invention.

First, the adaptive fill filtering method suitable for error concealment based on POCS according to the present invention has the following assumptions and constraints.

1) High-Frequency Smoothing: The high-frequency component of the block boundary that occurs in the process of initializing the error generating block to the neighboring block does not exist in the original image and should be removed.

2) Edge Protection: If there is an edge in the original image, it has continuity. When restoring, edge components should be kept to the maximum.

3) Consistency: The neighboring blocks of the error generating block are used for repetitive POCS and should not be changed during restoration.

4) In the hierarchical encoding environment, there is a correlation between similarity between neighboring blocks between base layer and enhancement layer.

Through the above-described assumptions and constraints, the adaptive fill filtering method suitable for error concealment based on POCS according to the present invention will be described in detail as follows.

As shown, the adaptive filtering method suitable for error concealment based on POCS according to the present invention checks which one block occurs in the original image composed of a plurality of blocks (S200). As a result of the check, if an error occurs, the similarity between the block in which the error occurs and the block adjacent to the block in which the error occurs is compared (S202). That is, the degree of similarity between the blocks in which the error occurs and the blocks located on the left, right, top, or diagonal lines of the block in which the error occurs is compared. The similarity is an indication of how similar an error-producing block is to a block in which no error occurs, and compares the similarity with a predetermined threshold. That is, if the similarity is greater than the threshold, this means that the error of the block in which the error has occurred is not large. On the contrary, if the similarity is not greater than the threshold, the error of the block in which the error has occurred is large. That is, the characteristics of the neighboring blocks are identified to compare how similar the blocks in which the error occurs are to the neighboring blocks. In this way, coefficient distribution coordinates are generated by identifying characteristics of neighboring blocks.

As such, if the similarity is greater than the threshold, that is, if the block in which the error has occurred is not large in error compared to the adjacent block, the adaptive filter according to the present invention is terminated. On the contrary, if the similarity is not greater than the threshold value, that is, if the block in which the error occurs is large in comparison with the adjacent block, it is detected whether an edge exists to perform the adaptive filtering process according to the present invention (S204).

That is, in step S204, it is detected whether an edge exists in a block adjacent to the block in which an error occurs. If the edge is not found as a result of the detection, the high frequency component of the block in which the error occurs is removed (S206). If there is an edge as a result of the detection in step S204, a directional filter is performed (S208).

The edge refers to whether the blocks located in the horizontal, vertical and diagonal directions of the block in which the error occurs contain high frequency components. That is, the presence of the edge indicates that the high frequency component is included in any one of the horizontal, vertical and diagonal directions of the block in which the error occurs. In contrast, the absence of an edge indicates that the high frequency component is not included in any one of the horizontal, vertical and diagonal directions of the block in which the error occurs.

The directional filter is filtering performed on all blocks decoded based on a plurality of blocks, that is, at the boundaries of all blocks except the edge of the block boundary in the image. Diagonal and downward diagonal boundaries are separated to detect directionality. In other words, if an edge is present, the directional filtering is performed through blocks located at the vertical, horizontal, upward diagonal, or downward diagonal of the block in which the error occurs. In other words, the filtering is performed so as to have continuity with the blocks located on the vertical, horizontal, upward diagonal, or downward diagonal of the block in which the error occurs.

In the above-described process, it is described that a step (S206) of detecting an edge of a block in which an error has occurred is performed, and then a step of removing high frequency (S206) is performed. The process may be performed. In addition, the process of detecting the presence of the edge (S204) and the process of removing the high frequency component (S206) may be performed simultaneously in parallel in terms of time flow.

Hereinafter, a process of detecting an edge and two methods of removing a high frequency component according to an exemplary embodiment of the present invention will be described in detail.

1. Edge detection process of an error block according to an embodiment of the present invention

In the present invention, in detecting the edge of the block in which an error occurs, not only one block among a plurality of blocks located in the horizontal, vertical, and diagonal directions of the block in which the error occurs, but also an image, an image quality, and the like can be used. There is also. If there are horizontal edges in the left and right blocks of the block in which the error occurs, it is determined that the horizontal edge exists. The same applies to the vertical edges, where the edge conditions are satisfied and the directional filter is applied when the energy of the yesterday portion is 70% or more of the total energy in both of the blocks where the error is detected. The energy includes a DCT coefficient.

2. Two methods of removing high frequency components of an error block according to an embodiment of the present invention

Two methods for removing high frequency components of an error-prone block according to an embodiment of the present invention include a method using a neighboring block and a method using a block high frequency elimination assumption.

First, both of the following should be considered in the process of removing the high frequency component of the block boundary.

1) If cut-off frequency is set too high, high frequency cannot be removed properly.

2) If the cut-off frequency is made too small, blurring phenomenon occurs because the high frequency components existed in the original image.

As described above, the cut-off frequency should be considered as an important factor in removing block phenomena, and it is composed of a method using a neighboring block according to the characteristics of the neighboring block and a method using a block smoothness assumption that there is no high frequency component at the block boundary. do.

First, the method of using the neighboring block is as follows.

Assuming that there are no high frequency components at the boundary of two neighboring blocks, the overall frequency characteristics of neighboring blocks are very similar to the respective block frequency characteristics. If a block phenomenon occurs between two blocks, the overall frequency characteristics of the two blocks and the frequency characteristics of each block are very different.

Therefore, the block phenomenon can be found by comparing the overall frequency characteristics with the respective frequency characteristics. By applying this to the error concealment method using the inter-layer correlation, comparing the frequency characteristics of the entire blocks and the frequency characteristics of each neighboring block, it is possible to remove the high frequency components of the block boundary generated in the process of initializing the neighboring blocks. have. Characteristic analysis of the neighboring block to find the block phenomenon uses the distribution of DCT coefficients and consists of obtaining the maximum coordinates in the horizontal and vertical directions.

Hereinafter, the process of removing the high frequency component is made of the following process.

1) DCT transform is performed on the eight neighboring blocks of the error generating block.

2) For the result of performing the DCT transform, X _max and Y _max , which are horizontal and vertical frequency distribution coordinates, are obtained for each block, and A _max and B _max , respectively, are obtained.

3) After multiplying the maximum values A _max and B _max by 3, respectively, the coordinates a _max and b _max of the horizontal and vertical maximum frequency components are obtained.

4) Perform DCT conversion of all blocks (9 blocks) including the block in which the error occurred.

5) The coordinates of the maximum frequency component in the horizontal and vertical directions are obtained by using the characteristic analysis function f _xy of the block below to the result value obtained in the process of 4).

6) Set the frequency component corresponding to the difference between 3) and 5) to 0.

3 is an exemplary diagram illustrating a method of removing a high frequency component of an error block according to an exemplary embodiment of the present invention.

As shown, one block is an image of 24 × 24 size and consists of 9 blocks. In this image, it is assumed that the middle block 305 has an error. In addition, it is assumed that the 302 block located above the block in which the error occurs is the most similar block found in the enhancement layer. Then, DCT transformation of blocks adjacent to the block in which an error occurs, that is, blocks 301, 302, 303, 304, 306, 307, 308, and 309 can be obtained as shown in Table 1 below. .

TABLE 1

Item 301 302 303 304 306 307 308 309 24 X 24 MAX (x, y) 1, 3 2, 7 4, 5 1, 7 2, 7 6, 3 3, 7 3, 5 22, 24

Among the horizontal and vertical frequency distribution coordinates obtained in Table 1, the horizontal and vertical coordinates obtained by multiplying the maximum value by 3 are 18 (6x3) and 21 (7x3). In this case, the neighboring block DCT distribution coordinates 18 and 21 include only high frequency components in the original image, but 22x24, which is the DCT distribution coordinates of the entire block (24x24), lacks similarity in the process of initializing into the most similar neighboring blocks in addition to the high frequency components of the original image. The high frequency component of the block boundary generated accordingly is included. This is an unnecessary error that occurs during the initialization of the block and should be removed.

Second, the method using the block high frequency elimination assumption is described as follows.

Assuming that the results of performing the DCT transformation on the eight blocks adjacent to the block where the error occurred are shown in Table 2 below, the horizontal and vertical coordinates a _max and b _max are 24 (8x3), 24 ( 8x3).

TABLE 2

Item 301 302 303 304 306 307 308 309 24 X 24 MAX (x, y) 1, 3 2, 8 4, 5 1, 7 1, 1 6, 3 8, 7 3, 5 22, 24

In Table 2, 24 and 24, DCT distribution coordinates, are out of cut-off frequency to remove unnecessary high frequency components. Therefore, when there are many high frequency components in the neighboring block, the influence of the neighboring block should be minimized as much as possible, which can be implemented by minimizing the window size for performing POCS.

If there are many high frequency components in the neighboring block, the cut-off frequency will be out of the coordinate values of unnecessary high frequency components.

Therefore, in this case, the influence of neighboring blocks should be minimized as much as possible. This can be solved by minimizing the window size for POCS execution. Analyzing the distribution characteristics of the DCT coefficients of damaged blocks (e.g. 8x8 pixels) and blocks containing one pixel (e.g. 10x10 pixels) using the fact that there are no high-frequency components in the block boundary according to the smoothly changing image. The high frequency component can be removed at the block boundary.

Using the fact that there are no high frequency components in the block boundary based on the assumption that the image changes smoothly, the block is analyzed by analyzing the distribution characteristics of the DCT coefficients of the damaged block (8x8 pixels) and the block containing one pixel (for example, 10x10 pixels). High frequency components can be removed at the boundary. The procedure for analyzing the distribution characteristics of the DCT coefficients is the same as using the above neighboring block.

In addition, whether the neighboring block has many high-frequency components can be determined by comparing the horizontal and vertical maximum coordinates of the block A _max , B _max obtained above, and the block including one pixel.

4 to 6 are diagrams showing the results of experiments between the conventional method and the method according to the practice of the present invention.

As shown, Figures 4a, 5a and 6a shows the original image, Figures 4b, 5b and 6b shows the image when the 4-average method is applied, Figures 4c, 5c, and 6c is a conventional Jung 4D, 5D, and 6D show simulation results according to an exemplary embodiment of the present invention.

In addition, when the adaptive filter considering the characteristics of the neighboring block is applied, various conventional error concealment methods and the method according to the present invention are compared. In addition, in this experiment, gray image with 352x288 size resolution and 256 levels of brightness was used to measure the performance of the restoration result, and the peak signal-to-noise ratio as shown in Table 3 below. Ratio, PSNR).

TABLE 3

3 4 5 4-average method 25.501 26.39 27.228 Jung's way 26.930 26.578 28.453 Method according to the invention 28.286 28.014 29.675

As a result of the experiment, the original image, 4-average, Jung method and the method according to the present invention are applied to the three images, and it can be seen that the method according to the present invention is superior to the conventional methods.

In addition, by providing an adaptive filtering method according to an embodiment of the present invention in order to remove high frequency components of a block boundary and maintain the edge of the original image in an error concealment method using inter-layer correlation, important edge information of the original image is provided. In addition to preserving the noise, effectively eliminating unnecessary high-frequency components at the block boundary, it also improves the objective image quality. In addition, compared with the conventional Jung method, the performance of 1.44 dB is improved in terms of PSNR.

1 is a diagram illustrating adaptive filtering suitable for error concealment based on POCS according to an embodiment of the present invention.

2 is a flowchart illustrating an adaptive filtering method suitable for error concealment based on POCS according to an embodiment of the present invention.

3 is an exemplary view showing a method of removing a high frequency component of an error block according to an embodiment of the present invention.

4 to 6 is a view showing a comparison result of experiments of the conventional method and the method according to an embodiment of the present invention.

Claims (6)

In the adaptive filtering method considering the characteristics of the adjacent block, Comparing the similarity between the blocks in which an error is generated and the blocks adjacent to the blocks in which the error occurs in an image composed of a plurality of blocks; If the value of the compared similarity is not greater than a predetermined threshold, detecting whether an edge exists; And applying a directional filter if the edge exists, and filtering the image by removing a high frequency component according to a frequency characteristic of the boundary of adjacent blocks if the edge does not exist. The method of claim 1, When there is a high frequency component at the boundary between the adjacent blocks, the first frequency to remove the high frequency component generated in the process of initializing the adjacent block by comparing the frequency characteristics of all the blocks and the frequency characteristics of the neighboring blocks of each block in the image; Process,  And if there is no high frequency component at a boundary between the adjacent blocks, removing the high frequency component by applying the directional filter to the block in which the error is generated by using the detected edge. The method of claim 2, wherein the first process is Performing a discrete cosine transform on each of the adjacent blocks; Obtaining a horizontal and vertical frequency distribution coordinate value for each block by using the discrete cosine transform value; Obtaining a first coordinate of a maximum frequency component by multiplying a frequency distribution coordinate value having a maximum horizontal and vertical coordinate by a predetermined value from the obtained horizontal and vertical frequency distribution coordinate values for each block; Performing a second cosine transform on the block in which the error occurs and one block adjacent to the block in which the error occurs, thereby performing a second maximum frequency component of the block located in a vertical, horizontal, or diagonal direction of the block in which the error occurs; Obtaining the coordinates, Adaptive filtering method further comprising the step of removing the high frequency component by setting the frequency component corresponding to the difference between the first coordinate and the second coordinate to 0 The method of claim 2, wherein the second process Performing a discrete cosine transform on each of the adjacent blocks; Obtaining a horizontal and vertical frequency distribution coordinate value for each block by using the discrete cosine transform value; Obtaining a first coordinate of a maximum frequency component by multiplying a frequency distribution coordinate value having a maximum horizontal and vertical coordinate by a predetermined value from the obtained horizontal and vertical frequency distribution coordinate values for each block; Performing a discrete cosine transform on all blocks of the image to obtain second coordinates of maximum frequency components of blocks located in a vertical, horizontal, or diagonal direction of the block in which the error occurs; And removing a high frequency component by setting a frequency component corresponding to the difference between the first coordinate and the second coordinate to zero. The method of claim 1, wherein the edge is Adaptive filtering method characterized in that the adjacent block located in any one of the horizontal, vertical, and diagonal direction of the block in which the error occurs. The method of claim 1, wherein the directional filter Adaptive filtering method characterized in that the filtering to have a continuity with the blocks located on the vertical, horizontal, upward diagonal or downward diagonal of the block in which the error occurs.

Images