JP5698629B2 - Image processing apparatus, filter processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus having a deblocking filter function, a filter processing method, and a program.

  In recent years, with the advance of multimedia, there are many opportunities to handle moving images. Image data has a large amount of information, and a moving image, which is a set of images, has an information amount exceeding 1 Gbit / sec in high-definition (1920 × 1080) broadcasting, which is the mainstream of television broadcasting.

  Therefore, the moving image data is H.264. Information is compressed and transmitted and stored by an encoding standard technique represented by H.264 / AVC (Advanced Video Coding) and HEVC (High Efficiency Video Coding).

  H., an encoding standard technology. H.264 / AVC uses a tool such as orthogonal transform, inter-frame prediction, intra-frame prediction, arithmetic coding, and deblocking filter to achieve compression of about 1/100 of a moving image. A deblocking filter, which is one of the tools, has a function of suppressing block distortion due to encoding in units of macroblocks, and has an effect of suppressing image quality deterioration particularly when the compression rate is increased.

  In the deblocking filter process, first, Bs values (Boundary Strength) are set for two adjacent blocks. In setting the Bs value, the definition shown in FIG. 1 is used. FIG. 1 is a diagram showing the definition of the Bs value. Deblocking filter processing is performed using the Bs value set based on FIG. 1 (Non-Patent Document 1).

  In addition, regarding this deblocking filter processing, there is a technique for determining whether or not the difference value of pixels at the boundary between adjacent blocks is coding distortion based on the Qp value. The Qp value is an encoding quality setting value at the time of encoding (Patent Document 1).

JP 2007-336468 A

Supervised by Satoshi Okubo, “Revised Third Edition H.264 / AVC Textbook”, Impress R & D, p. 144-148, 2009

  In the prior art, based on whether one of two adjacent blocks p and q belongs to an intra macroblock, is located at a macroblock boundary, has an orthogonal transform coefficient, or has a different reference picture or motion vector value A Bs value is set. As shown in FIG. 1, the Bs value is set to a low value when either of the blocks p and q does not belong to the intra macroblock.

  Therefore, when both the blocks p and q are inter macroblocks, the Bs value becomes low despite the presence of coding distortion related to motion vector prediction. The coding distortion related to motion vector prediction becomes larger as a high-resolution image including large motion blur and many moving objects.

  Therefore, the conventional technique has a problem that when both the blocks p and q are inter macroblocks, the Bs value is set low, and the coding distortion related to motion vector prediction cannot be sufficiently removed.

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to provide an image processing apparatus, a filter processing method, and a program that can appropriately remove coding distortion related to motion vector prediction existing at a block boundary. .

An image processing apparatus according to an aspect of the present invention is an image processing apparatus that performs filter processing to remove distortion at a block boundary of an image, and is based on the direction and magnitude of motion vectors of adjacent first blocks and second blocks. A setting unit that sets parameters used for deblocking filter processing, and a filter unit that performs deblocking filter processing at a boundary between the first block and the second block using the parameters set by the setting unit; The setting unit includes: a difference value calculation unit that calculates a horizontal and vertical difference value between the motion vector of the first block and the motion vector of the second block; the motion vector of the first block; An average value calculating unit for calculating an average value in the horizontal and vertical directions with the motion vector of the second block; the first block and the second block; A difference absolute value calculation unit that calculates a difference absolute value of each pixel adjacent to the boundary between the threshold, a threshold setting unit that sets a threshold based on the difference value and the average value, and the difference absolute value and the threshold based on the comparison result, Ru and a parameter setting unit configured to set the parameters.

The threshold setting unit may set the threshold based on the difference value, the average value, and the size of the first block or the second block.

A filter processing method according to another aspect of the present invention is a filter processing method executed by an image processing apparatus that performs filter processing for removing distortion at a block boundary of an image, and includes a first block and a second block adjacent to each other. Based on the direction and magnitude of the motion vector, a setting step for setting parameters used for the deblocking filter process, and a parameter set by the setting step is used to delimit the boundary between the first block and the second block. a filter step of performing a blocking filter, was closed, the setting step, the horizontal motion vector of the second block and the motion vector of the first block, the difference value calculation step of calculating a difference value in the vertical direction , Horizontal and vertical of the motion vector of the first block and the motion vector of the second block An average value calculating step of calculating an average value of the direction, a difference absolute value calculating step of calculating a difference absolute value of each pixel adjacent to a boundary between the first block and the second block, the difference value, and the average a threshold setting step of setting a threshold value based on a value based on the comparison result of the difference absolute value and the threshold to have a parameter setting step of setting the parameters.

According to another aspect of the present invention, there is provided a program for performing a filtering process for removing distortion at a block boundary of an image, wherein the direction of motion vectors of the first block and the second block adjacent to the computer and A deblocking filter process is performed at the boundary between the first block and the second block using a setting step for setting parameters used in the deblocking filter process based on the size and the parameters set in the setting step A filter step is executed, and the setting step includes a difference value calculating step for calculating a horizontal and vertical difference value between the motion vector of the first block and the motion vector of the second block, and the motion of the first block Horizontal and vertical average values of the vector and the motion vector of the second block An average value calculating step for calculating, a difference absolute value calculating step for calculating a difference absolute value of each pixel adjacent to a boundary between the first block and the second block, a threshold value based on the difference value and the average value a threshold setting step of setting, based on the comparison result of the difference absolute value and the threshold value, that having a parameter setting step of setting the parameters.

  According to the present invention, it is possible to appropriately remove coding distortion related to motion vector prediction existing at a block boundary.

The figure which shows the definition of Bs value. 1 is a block diagram illustrating an example of a schematic configuration of an image encoding device according to Embodiment 1. FIG. FIG. 3 is a block diagram illustrating an example of a configuration of a deblocking filter unit according to the first embodiment. FIG. 3 is a block diagram illustrating an example of a configuration of a setting unit according to the first embodiment. FIG. 3 is a block diagram illustrating an example of a configuration of a parameter setting unit according to the first embodiment. The figure which shows an example of the adjacent relationship of the block p and the block q. The figure which shows the case where the block adjacent to a horizontal direction has a big motion vector in a horizontal direction. The figure which shows an example of the relationship table of a motion vector average value and a threshold value. The figure which shows an example of the relationship table of a motion vector average value, block size, and a threshold value. The figure which shows an example of the adjacent pixel used for threshold value determination. 6 is a flowchart illustrating an example of filter processing (part 1) in the first embodiment. 9 is a flowchart illustrating an example of filter processing (part 2) in the first embodiment. FIG. 9 is a block diagram illustrating an example of a configuration of a parameter setting unit according to a second embodiment. 10 is a flowchart illustrating an example of filter processing according to the second embodiment. FIG. 10 is a block diagram illustrating an example of a configuration of a parameter setting unit according to a third embodiment. 10 is a flowchart illustrating an example of filter processing according to the third embodiment. FIG. 10 is a block diagram illustrating an example of a configuration of an image encoding device according to a fourth embodiment.

  Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

[Example 1]
<Configuration>
FIG. 2 is a block diagram illustrating an example of a schematic configuration of the image encoding device 10 according to the first embodiment. 2 includes a preprocessing unit 101, a prediction error signal generation unit 102, an orthogonal transformation unit 103, a quantization unit 104, an entropy coding unit 105, an inverse quantization unit 106, and an inverse orthogonal transformation unit 107. A decoded image generation unit 108, a deblocking filter unit 109, a frame memory 110, a motion prediction unit 111, and a motion compensation unit 112.

  The preprocessing unit 101 rearranges the pictures according to the picture type, and sequentially outputs the picture type and the frame image for each frame. Further, the preprocessing unit 101 may perform a process of generating a reduced image and investigating the direction of movement and the like.

  The prediction error signal generation unit 102 obtains macroblock data (hereinafter also referred to as MB data) obtained by dividing an encoding target image of input moving image data into blocks (MacroBlock) of 16 × 16 pixels (pixels). .

  The prediction error signal generation unit 102 generates a prediction error signal from the MB data and the MB data of the prediction image output from the motion compensation unit 112. The prediction error signal generation unit 102 outputs the generated prediction error signal to the orthogonal transformation unit 103.

  The orthogonal transform unit 103 performs DCT (Discrete Cosine Transform) transform on the input prediction error signal in units of 8 × 8 or 4 × 4. In addition to the DCT transform, orthogonal transform such as Hadamard transform may be performed.

  The orthogonal transform unit 103 acquires orthogonal transform coefficients that are separated into frequency components in the horizontal and vertical directions by orthogonal transform processing. In this orthogonal transformation process, data is collected in the low frequency component and the information amount can be compressed by converting the frequency component using the spatial correlation of the image.

  The quantization unit 104 reduces the code amount of the data by quantizing the orthogonally transformed data (also referred to as orthogonal transformation coefficient), and the quantized value is converted into the entropy coding unit 105 and the inverse quantization unit. It outputs to 106.

  The entropy encoding unit 105 entropy encodes the data output from the quantization unit 104 and the motion vector information output from the motion prediction unit 111, and outputs a bitstream. Entropy coding is a method of assigning variable-length codes according to the appearance frequency of symbols.

  For example, the entropy encoding unit 105 basically assigns a shorter code to a combination of coefficients having a high appearance frequency, and assigns a longer code to a combination of coefficients having a low appearance frequency. This attempts to shorten the code length as a whole. For example, H.M. In H.264, entropy coding of a method called CAVLC (Context-Adaptive Variable Length Coding) or CABAC (Context-Adaptive Binary Arithmetic Coding) can be selected.

  The inverse quantization unit 106 inversely quantizes the data output from the quantization unit 104. The inverse quantization unit 106 outputs the inversely quantized data to the inverse orthogonal transform unit 107.

  The inverse orthogonal transform unit 107 performs inverse orthogonal transform on the inversely quantized data to convert the frequency component into a pixel component. The inverse orthogonal transform unit 107 outputs the transformed data to the decoded image generation unit 108. By performing the inverse orthogonal transform process, a signal comparable to the prediction error signal before encoding is obtained.

  The decoded image generation unit 108 adds the MB data of the image subjected to motion compensation by the motion compensation unit 112 and the prediction error signal decoded by the inverse orthogonal transform unit 107. Thereby, the processing image equivalent to the decoding side can also be generated on the encoding side.

  An image generated on the encoding side is referred to as a locally decoded image, and by generating the same processed image on the encoding side as that on the decoding side, it is possible to perform differential encoding on and after the next picture. The decoded image generation unit 108 outputs the MB data of the locally decoded image generated by the addition to the deblocking filter unit 109.

The deblocking filter unit 109 performs deblocking filter processing when the deblocking filter condition is satisfied. The deblocking filter condition is H.264. The H.264 / AVC encoding technique is as follows.
Bs> 0 Condition 1
| P0−q0 | <α ... Condition 2
| P1-q0 | <β (Condition 3)
| Q1-q0 | <β (Condition 4)
A deblocking filter process is performed when all of these conditions 1 to 4 are satisfied.

  The Bs value is H.264. This is the block boundary strength defined by H.264 / AVC and has a value of 0 or more and 4 or less. A case of 0 means that deblocking filter processing is not performed. When the block p and the block q are adjacent to each other in the horizontal direction, p0 is a pixel of the block p at the boundary with the block q, and p1 is a pixel in the block p that is adjacent to p0 in the horizontal direction. q0 is a pixel of the block q at the boundary with the block p, and q1 is a pixel in the block q adjacent to q0 in the horizontal direction. When the blocks p and q are adjacent in the vertical direction, the horizontal direction described above may be read as the vertical direction.

  Here, in the direction in which the motion vector is large, a large motion blur occurs and the edge component decreases. Therefore, when the motion vectors of two blocks adjacent in the moving direction indicate the same direction and the size is large, the difference between the pixel values at the boundary of the adjacent blocks should be small.

  Therefore, when the block p and the block q have a motion vector, the deblocking filter unit 109 sets parameters used for the deblocking filter based on the direction and magnitude of the motion vector, and performs a filtering process. Details of the deblocking filter unit 109 will be described later.

  The frame memory 110 stores MB data of a locally decoded image that has been subjected to deblocking filtering. The locally decoded image stored in the frame memory 110 can be a reference image.

  The motion prediction unit 111 performs a motion search using the MB data in the encoding target image and the MB data of the reference image acquired from the frame memory 110, and obtains an appropriate motion vector.

  The motion vector is a value indicating a spatial deviation in units of blocks obtained by using a block matching technique for searching for a position most similar to the MB of the encoding target image from the reference image in units of blocks.

  In the motion vector encoding, not the motion vector component itself but the difference vector with the motion vector of the surrounding MB is encoded. The motion prediction unit 111 outputs the searched motion vector to the motion compensation unit 112, and outputs motion vector information including information on the motion vector and the reference image to the entropy encoding unit 105.

  The motion compensation unit 112 performs motion compensation on the reference image data with the motion vector provided from the motion prediction unit 111. As a result, MB data as a motion-compensated reference image is generated. The motion-compensated MB data is output to the prediction error signal generation unit 102 and the decoded image generation unit 108 as a reference image.

(Deblocking filter part)
Next, the configuration of the deblocking filter unit 109 will be described. FIG. 3 is a block diagram illustrating an example of the configuration of the deblocking filter unit 109 according to the first embodiment. As illustrated in FIG. 3, the deblocking filter unit 109 includes a setting unit 201 and a filter unit 202.

  The setting unit 201 sets parameters used for the deblocking filter based on the direction and magnitude of the motion vectors of the adjacent blocks p and q. An example of this parameter is a Bs value. Adjacent blocks p and q are collectively referred to as adjacent blocks. Details of the setting unit 201 will be described with reference to FIG. Note that the setting unit 201 acquires data such as motion vectors of previously encoded blocks from the frame memory 110.

  The filter unit 202 performs deblocking filter processing using the parameters set by the setting unit 201. Refer to Non-Patent Document 1 for deblocking filter processing.

  FIG. 4 is a block diagram illustrating an example of the configuration of the setting unit 201 according to the first embodiment. The setting unit 201 illustrated in FIG. 4 includes a motion vector difference value calculation unit 301, a motion vector average value calculation unit 302, an adjacent pixel difference absolute value calculation unit 303, and a parameter setting unit 304.

  The motion vector difference value calculation unit 301 acquires the motion vector of the block p and the motion vector of the block q. The motion vector difference value calculation unit 301 calculates difference values in the horizontal and vertical directions between the motion vector of the block p and the motion vector of the block q. The motion vector difference value calculation unit 301 outputs the calculated motion vector difference value to the parameter setting unit 304. The difference value may be an absolute value.

  The motion vector average value calculation unit 302 acquires the motion vector of the block p and the motion vector of the block q. The motion vector average value calculation unit 302 calculates average values in the horizontal and vertical directions of the motion vector of the block p and the motion vector of the block q. The motion vector average value calculation unit 302 outputs the calculated average value of the motion vectors to the parameter setting unit 304.

  Note that the motion vector average value calculation unit 302 may calculate only the average value of motion vector components in the same direction as the adjacent direction of the block p and the block q. For example, when the block p and the block q are adjacent to each other in the horizontal direction, the motion vector average value calculation unit 302 calculates only the average value of the horizontal component of the motion vector of the block p and the horizontal component of the motion vector of the block q. May be.

  The adjacent pixel difference absolute value calculation unit 303 acquires the pixel value of the block p and the pixel value of the block q. The adjacent pixel difference absolute value calculation unit 303 calculates the difference absolute value between p0 and q0, for example.

  The adjacent pixel difference absolute value calculation unit 303 is a pixel adjacent to the block p and the block q such as the difference absolute value between p1 and q1 or the difference absolute value between the average of p0 and p1 and the average of q0 and q1. What is necessary is just to obtain | require the difference absolute value between. The adjacent pixel difference absolute value calculation unit 303 outputs the calculated difference absolute value to the parameter setting unit 304. Pixels of block p and block q adjacent to the block boundary are referred to as adjacent pixels.

  The parameter setting unit 304 sets parameters used for the deblocking filter based on the obtained difference value, average value of motion vectors, and the absolute difference value between adjacent pixels. The parameter setting unit 304 will be described with reference to FIG.

  FIG. 5 is a block diagram illustrating an example of the configuration of the parameter setting unit 304 according to the first embodiment. The parameter setting unit 304 illustrated in FIG. 5 includes a threshold setting unit 401 and a block boundary strength setting unit 402.

  The threshold value setting unit 401 sets a threshold value used for the determination of the absolute difference value between the horizontal and vertical pixel values based on the difference value and average value of the motion vectors. For example, when a block adjacent in the horizontal direction has a large motion vector in the horizontal direction, the threshold setting unit 401 sets a small threshold in the horizontal direction. This is because in the horizontal direction, it can be determined that the horizontal edge component due to motion blur is reduced, and thus the threshold value for the absolute difference between adjacent pixel values is reduced.

  FIG. 6 is a diagram illustrating an example of the adjacent relationship between the block p and the block q. FIG. 6A shows a case where the block p and the block q are adjacent to each other in the horizontal direction. As illustrated in FIG. 6A, the threshold setting unit 401 sets a horizontal threshold when the blocks are adjacent in the horizontal direction.

  FIG. 6B shows a case where the block p and the block q are adjacent in the vertical direction. As shown in FIG. 6B, the threshold setting unit 401 sets a vertical threshold when the blocks are adjacent in the vertical direction.

  FIG. 7 is a diagram illustrating a case where blocks adjacent in the horizontal direction have a large motion vector in the horizontal direction. As illustrated in FIG. 7, the threshold setting unit 401 sets a small horizontal threshold when blocks adjacent in the horizontal direction have a large motion vector in the horizontal direction.

  Regarding threshold setting, the threshold setting unit 401 first determines whether or not the motion vector of an adjacent block points in the same direction as the adjacent direction of the block. For example, the threshold setting unit 401 determines 10% of the larger one of the horizontal and vertical components of the two motion vectors as the horizontal threshold and the vertical threshold. When the difference value of the horizontal component of the motion vector is smaller than the horizontal threshold value and the difference value of the vertical component of the motion vector is smaller than the vertical threshold value, the threshold setting unit 401 indicates the two motion vectors in the same direction. Is determined.

  Further, the threshold setting unit 401 determines whether two motion vectors indicate the horizontal direction when adjacent blocks are adjacent in the horizontal direction. The threshold value setting unit 401 determines whether two motion vectors indicate the vertical direction when adjacent blocks are adjacent in the vertical direction. The threshold value setting unit 401 may determine whether the two motion vectors indicate horizontal or vertical and which of the horizontal component and the vertical component is larger. If the two motion vectors do not point in the same direction as the adjacent direction between the blocks, the threshold setting unit 401 notifies the block boundary strength setting unit 402 to that effect.

  When the two motion vectors point in the same direction as the adjacent direction between the blocks, the threshold setting unit 401 sets a threshold used for comparison with the absolute difference value of the pixel values according to the magnitudes of the two motion vectors. .

  FIG. 8 is a diagram illustrating an example of a relationship table between motion vector average values and threshold values. In the example illustrated in FIG. 8, the threshold setting unit 401 sets the threshold to 32, for example, when the motion vector average value is 4 or less, and sets the threshold to 16, for example, when the motion vector is greater than 4 and 8 or less.

  This threshold is an example when the image resolution is 1920 × 1080 and the number of gradations is 256, for example. Further, when the image resolution is increased, the threshold value shown in FIG. 8 can be expressed by a linear mathematical expression.

  The threshold value setting unit 401 sets a threshold value for the absolute difference value of pixel values in the horizontal direction when adjacent blocks are in the horizontal direction. Further, the threshold setting unit 401 sets a threshold for the absolute difference value of the pixel values in the vertical direction when adjacent blocks are in the vertical direction. The threshold setting unit 401 outputs the set threshold to the block boundary strength setting unit 402.

  The threshold setting unit 401 may set the threshold in consideration of the block size of the block p and / or the block q when setting the threshold. In general, when the motion vector increases, the motion detection accuracy decreases when motion detection is performed with a small-sized block.

  This is because as the motion vector increases, the motion blur increases, and the feature of the image cannot be correctly captured with a small-sized block.

  When a large motion vector is detected in a small size block, the motion detection result is likely to be erroneous. Therefore, when the difference between the pixel values existing at the block boundary is large, the possibility of block distortion is increased instead of the edge of the image. Therefore, a stronger deblocking filter is applied as a large motion vector is detected in a small-sized block.

  Based on the above idea, the threshold setting unit 401 may set a threshold using the relation table shown in FIG. FIG. 9 is a diagram illustrating an example of a relationship table of motion vector average values, block sizes, and threshold values.

  In the example illustrated in FIG. 9, the threshold setting unit 401 adaptively sets a threshold according to the average value of motion vectors and the block size. A stronger deblocking filter is applied by setting the threshold value smaller as the block size is smaller.

  Returning to FIG. The Bs value is set in accordance with the H.264 / AVC encoding technique. When the block boundary strength setting unit 402 is notified from the threshold setting unit 401 that the two motion vectors do not point in the same direction as the adjacent direction, the H.D. The Bs value set according to H.264 / AVC is output.

  When the block boundary strength setting unit 402 acquires the threshold value for the horizontal direction or the vertical direction from the threshold value setting unit 401 and acquires the absolute difference value of the adjacent pixel from the adjacent pixel difference absolute value calculation unit 303, the following determination is performed. Reset the Bs value.

  For example, when the absolute difference value in the horizontal direction is larger than the threshold value in the horizontal direction, the block boundary strength setting unit 402 adds 1 to the Bs value and resets the Bs value. Thereby, a stronger deblocking filter is applied.

  When the difference absolute value exceeds the threshold value, it is considered that the difference between the pixel values of the block boundary is increased due to the encoding distortion, although the edge component due to motion blur is reduced. Therefore, it is preferable to apply a stronger deblocking filter to the pixels at the block boundary.

FIG. 10 is a diagram illustrating an example of adjacent pixels used for threshold determination. In the example illustrated in FIG. 10, the block boundary strength setting unit 402 compares the absolute difference value between the pixel values of _pn 0 and q _n 0 (n = 1 to 8) and the horizontal threshold value. In the example illustrated in FIG. 10, each of eight adjacent pixels is compared with a threshold value. One threshold is set for each block, and the block boundary strength (Bs) is set for each adjacent pixel.

Note that one block boundary strength may be set for each block. At this time, the block boundary strength setting unit 402 _is reset and the differential value of the average value of the average value and the _q 1 _{0~q 8} 0 of p 1 0 to p ₈ 0, the block boundary strength is compared with a threshold value You may do it.

The block boundary strength setting unit 402 may set the Bs value instead of resetting the Bs value. In this case, when the block p and the block q belong to the inter macroblock, the block boundary strength setting unit 402 may set the Bs value based on the following definition 1, for example.
If difference absolute value ≦ threshold value + 4, Bs = 0
If threshold + 4 <absolute difference value ≦ threshold + 8, Bs = 1
If threshold + 8 <difference absolute value ≦ threshold + 16, Bs = 2
If threshold + 16 <difference absolute value ≦ threshold + 32, Bs = 3
If threshold + 32 <difference absolute value, Bs = 4
Thereby, even when the block p and the block q belong to the inter macroblock, a strong deblocking filter can be applied.

  It should be noted that the block boundary strength setting unit 402 determines that the H.264 or H.H. The Bs value may be set according to the H.264 / AVC definition.

<Operation>
Next, the operation of the image encoding device 10 in the first embodiment will be described. FIG. 11 is a flowchart illustrating an example of filter processing (part 1) in the first embodiment. The flow shown in FIG. 11 is an example in which the filter process is performed by resetting the Bs value, which is a parameter used for the filter.

  In step S101 shown in FIG. 11, the setting unit 201 determines whether or not the adjacent block belongs to the inter macroblock. When the adjacent block belongs to the inter macroblock (step S101-YES), the process proceeds to step S102, and when the adjacent block does not belong to the inter macroblock (step S101-NO), the process proceeds to step S107.

  In step S102, the motion vector difference value calculation unit 301 calculates a difference value between motion vectors of adjacent blocks.

  In step S103, the motion vector average value calculation unit 302 calculates the average value of motion vectors of adjacent blocks.

  In step S <b> 104, the adjacent pixel difference absolute value calculation unit 303 calculates a difference absolute value of the pixel values for the pixels located at the boundaries of the adjacent blocks. The processes of steps S102 to S104 are out of order and may be processed in parallel.

  In step S105, the threshold setting unit 401 sets a threshold based on the direction and magnitude of two motion vectors of each adjacent block. The threshold is set based on, for example, the tables shown in FIGS.

  In step S106, the block boundary strength setting unit 402 resets the Bs value when the absolute difference value of the pixel value is larger than the threshold value.

  In step S <b> 107, the filter unit 202 performs deblocking filter processing using the Bs value set or reset by the block boundary strength setting unit 402.

  As a result, a deblocking filter having a large Bs value can be applied to the boundary between adjacent blocks for the adjacent block whose edge component is reduced due to motion blur, and coding distortion can be removed.

  FIG. 12 is a flowchart illustrating an example of filter processing (part 2) in the first embodiment. The flow shown in FIG. 12 is an example in which a filter process is performed by setting a Bs value that is a parameter used for a filter.

  The processes in steps S201 to S205 shown in FIG. 12 are the same as the processes in steps S101 to S105 shown in FIG.

  In step S206, the block boundary strength setting unit 402 sets the Bs value based on the absolute difference value of the pixel value and the threshold value. For example, the block boundary strength setting unit 402 sets the Bs value based on the definition 1 described above.

  In step S207, the block boundary strength setting unit 402 determines whether the H.264 The Bs value is set based on the definition of H.264 / AVC or HEVC.

  In step S <b> 208, the filter unit 202 performs deblocking filter processing using the Bs value set in the block boundary strength setting unit 402.

  As described above, according to the first embodiment, when the block p and the block q belong to the inter macroblock, it is determined whether or not the pixel value difference at the boundary between adjacent blocks is coding distortion based on the motion vector. Judgment can be made and the Bs value can be set. By setting the Bs value, a strong deblocking filter can be applied to a boundary where a pixel value difference is large at a boundary where an edge component due to motion blur is reduced. Therefore, it is possible to appropriately remove coding distortion related to motion vector prediction existing at the block boundary.

[Example 2]
Next, an image coding apparatus according to the second embodiment will be described. In the second embodiment, a filter offset value (FilterOffset) is set as a parameter used for the deblocking filter. According to the second embodiment, by appropriately setting the filter offset value based on the direction and magnitude of the motion vector of the adjacent block, it is possible to appropriately remove the coding distortion related to motion vector prediction existing at the block boundary. Can do.

<Configuration>
The configuration of the image coding apparatus in the second embodiment is the same as the configuration shown in FIG. 2, the configuration of the deblocking filter unit is the same as the configuration shown in FIG. 3, and the configuration of the setting unit is shown in FIG. Since it is the same as that of the configuration, its description is omitted.

(Parameter setting part)
FIG. 13 is a block diagram illustrating an example of the configuration of the parameter setting unit 500 according to the second embodiment. A parameter setting unit 500 illustrated in FIG. 13 includes a threshold value setting unit 501 and an offset value setting unit 502.

  The threshold setting unit 501 is the same as the threshold setting unit 401 shown in FIG.

  The offset value setting unit 502 acquires the difference absolute value of the adjacent pixel from the adjacent pixel difference absolute value calculation unit 303 and acquires the threshold value from the threshold value setting unit 501, and when the difference absolute value is larger than the threshold value, the offset value is larger. The filter offset value is set according to the degree.

  For example, when the absolute difference value is larger than the threshold value, the magnitude is x. The offset value setting unit 502 sets int (x / 4) to the filter offset value (FilterOffset) for the size x. This filter offset value is added to the Qp value.

  For example, when x = 15, int (15/4) = 3 is the filter offset value. When Qp = 28, Qp = 28 + 3 = 31, and a deblocking filter having an intensity equivalent to Qp = 31 is performed. The offset value setting unit 502 sets a filter offset value for each adjacent pixel.

  The offset value setting unit 502 outputs the set filter offset value to the filter unit 202. Accordingly, the filter unit 202 can add the filter offset value to the Qp value and perform a deblocking filter process corresponding to the Qp value after the addition.

<Operation>
Next, the operation of the image coding apparatus according to the second embodiment will be described. FIG. 14 is a flowchart illustrating an example of filter processing according to the second embodiment. The processing in steps S301 to S305 shown in FIG. 14 is the same as the processing in steps S101 to S105 shown in FIG.

  In step S306, the offset value setting unit 502 sets the filter offset value by comparing the threshold set based on the direction and magnitude of the motion vector of the adjacent block with the absolute difference value of the adjacent pixel.

  In step S307, the filter unit 202 adds the set filter offset value to the Qp value, and performs deblocking filter processing corresponding to the Qp value after the addition.

  As described above, according to the second embodiment, by appropriately setting the filter offset value based on the direction and magnitude of the motion vector of the adjacent block, it is possible to appropriately remove the coding distortion related to motion vector prediction existing at the block boundary. can do.

[Example 3]
Next, an image coding apparatus according to the third embodiment will be described. Example 3 is an example in which Example 1 and Example 2 are combined.

<Configuration>
The configuration of the image encoding device in the third embodiment is the same as the configuration shown in FIG. 2, the configuration of the deblocking filter unit is the same as the configuration shown in FIG. 3, and the configuration of the setting unit is shown in FIG. Since it is the same as that of the configuration, its description is omitted.

(Parameter setting part)
FIG. 15 is a block diagram illustrating an example of the configuration of the parameter setting unit 600 according to the third embodiment. A parameter setting unit 600 illustrated in FIG. 15 includes a threshold setting unit 601, a block boundary strength setting unit 602, and an offset value setting unit 603.

  The threshold setting unit 601 is the same as the threshold setting unit 401 shown in FIG.

  The block boundary strength setting unit 602 is the same as the block boundary strength setting unit 402 shown in FIG. 5, and the offset value setting unit 603 is the same as the offset value setting unit 502 shown in FIG. Therefore, the parameter setting unit 600 sets a Bs value and a filter offset value, which are parameters used for the deblocking filter, based on the direction and magnitude of the motion vector of the adjacent block.

<Operation>
Next, the operation of the image coding apparatus according to the third embodiment will be described. FIG. 16 is a flowchart illustrating an example of the filter processing according to the third embodiment. The processes in steps S401 to S406 shown in FIG. 16 are the same as the processes in steps S101 to S106 shown in FIG.

  The process in step S407 is the same as the process in step S306 shown in FIG.

  In step S408, the filter unit 202 performs deblocking filter processing using the reset Bs value and the set filter offset value.

  As described above, according to the third embodiment, it is possible to appropriately remove coding distortion related to motion vector prediction existing at a block boundary. In the third embodiment, the configuration may be such that both the Bs value and the filter offset value can be set, and the user selects one of them and sets the selected parameter.

[Example 4]
FIG. 17 is a block diagram illustrating an example of a configuration of an image encoding device 700 according to the fourth embodiment. The image encoding device 700 is an example of a device in which the image encoding processing described in each of the above-described embodiments is implemented by software.

  As illustrated in FIG. 17, the image encoding device 700 includes a control unit 701, a main storage unit 702, an auxiliary storage unit 703, a drive device 704, a network I / F unit 706, an input unit 707, and a display unit 708. These components are connected to each other via a bus so as to be able to transmit and receive data.

  The control unit 701 is a CPU that controls each device, calculates data, and processes in a computer. The control unit 701 is an arithmetic device that executes a program for image encoding processing including filter processing stored in the main storage unit 702 or the auxiliary storage unit 703. The control unit 701 receives data from the input unit 707 and the storage device, calculates and processes the data, and outputs the data to the display unit 708 and the storage device.

  The control unit 701 can implement the filter processing described in each embodiment by executing a program for image encoding processing including filter processing.

  The main storage unit 702 is a ROM (Read Only Memory), a RAM (Random Access Memory), or the like, and a storage device that stores or temporarily stores programs and data such as an OS and application software that are basic software executed by the control unit 701. It is.

  The auxiliary storage unit 703 is an HDD (Hard Disk Drive) or the like, and is a storage device that stores data related to application software or the like.

  The drive device 704 reads the program from the recording medium 705, for example, a flexible disk, and installs it in the storage device.

  A predetermined program is stored in the recording medium 705, and the program stored in the recording medium 705 is installed in the image encoding apparatus 700 via the drive device 704. The installed predetermined program can be executed by the image encoding apparatus 700.

  The network I / F unit 706 has a communication function connected via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network) constructed by a data transmission path such as a wired and / or wireless line. This is an interface between the device and the image encoding device 700.

  The input unit 707 includes a keyboard having cursor keys, numeric input, various function keys, and the like, and a mouse and a slice pad for performing key selection on the display screen of the display unit 708. The input unit 707 is a user interface for a user to give an operation instruction to the control unit 701 and input data.

  The display unit 708 is configured by a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), or the like, and performs display according to display data input from the control unit 701.

  The frame memory 110 illustrated in FIG. 2 is realized by, for example, the main storage unit 702 or the auxiliary storage unit 703, and the configuration other than the frame memory 110 illustrated in FIG. 2 includes, for example, the control unit 701 and the main storage unit 702 as a work memory. Can be realized.

  The program executed by the image encoding device 700 has a module configuration including each unit described in the first to third embodiments. As actual hardware, when the control unit 701 reads a program from the auxiliary storage unit 703 and executes it, one or more of the above-described units are loaded onto the main storage unit 702, and one or more of the units are main. It is generated on the storage unit 702.

  As described above, the filtering process described in each embodiment described above may be realized as a program for causing a computer to execute the filtering process. The filter processing described above can be realized by installing this program from a server or the like and causing the computer to execute it.

  It is also possible to record the program on the recording medium 705 and cause the computer or portable terminal to read the recording medium 705 on which the program is recorded to realize the above-described filtering process. The recording medium 705 is a recording medium that records information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, a magneto-optical disk, etc., or information electrically, such as a ROM, flash memory, or the like. Various types of recording media such as a semiconductor memory for recording can be used. Further, the filtering process described in each of the above embodiments may be implemented in one or a plurality of integrated circuits.

  In the first to fourth embodiments, the image coding apparatus has been described as an example. However, the present invention can be similarly applied to an image decoding apparatus having a deblocking filter function. The image encoding device and the image decoding device are collectively referred to as an image processing device. In the above embodiments, H. Although H.264 / AVC has been described as an example, any coding technique that performs deblocking filter processing can be applied.

  Each embodiment has been described in detail above. However, the present invention is not limited to the specific embodiment, and various modifications and changes other than the above-described modification are possible within the scope described in the claims. .

DESCRIPTION OF SYMBOLS 10 Image coding apparatus 101 Preprocessing part 102 Prediction error signal generation part 103 Orthogonal transformation part 104 Quantization part 105 Entropy coding part 106 Inverse quantization part 107 Inverse orthogonal transformation part 108 Decoded image generation part 109 Frame memory 110 Motion prediction part 111 motion compensation unit 201 setting unit 202 filter unit 301 motion vector difference value calculation unit 302 motion vector average value calculation unit 303 adjacent pixel difference absolute value calculation unit 304 parameter setting unit 401, 501, 601 threshold setting unit 402, 602 block boundary strength Setting unit 502, 603 Offset value setting unit 701 Control unit 702 Main storage unit 703 Auxiliary storage unit 704 Drive device 706 Network I / F unit 707 Input unit 708 Display unit

Claims (4)

An image processing apparatus that performs filter processing to remove distortion at a block boundary of an image,
A setting unit that sets parameters used for deblocking filter processing based on the direction and magnitude of the motion vectors of the adjacent first block and second block;
A filter unit that performs deblocking filter processing at a boundary between the first block and the second block using the parameters set by the setting unit;
Equipped with a,
The setting unit
A difference value calculation unit for calculating a difference value in the horizontal and vertical directions between the motion vector of the first block and the motion vector of the second block;
An average value calculating unit for calculating an average value in the horizontal and vertical directions of the motion vector of the first block and the motion vector of the second block;
A difference absolute value calculation unit that calculates a difference absolute value of each pixel adjacent to a boundary between the first block and the second block;
A threshold setting unit that sets a threshold based on the difference value and the average value;
The difference absolute value based on the comparison result between the threshold, the image processing apparatus Ru and a parameter setting unit configured to set the parameters. The threshold setting unit includes:
The difference value, the average value, and the image processing apparatus according to claim 1, wherein setting the threshold value based on the size of the first block or the second block. A filter processing method executed by an image processing apparatus that performs filter processing for removing distortion at a block boundary of an image,
A setting step for setting parameters used for deblocking filter processing based on the direction and magnitude of the motion vectors of the adjacent first block and second block;
A filter step for performing a deblocking filter process at a boundary between the first block and the second block using the parameters set in the setting step;
I have a,
The setting step includes
A difference value calculating step of calculating a horizontal and vertical difference value between the motion vector of the first block and the motion vector of the second block;
An average value calculating step for calculating horizontal and vertical average values of the motion vector of the first block and the motion vector of the second block;
A difference absolute value calculating step for calculating a difference absolute value of each pixel adjacent to a boundary between the first block and the second block;
A threshold setting step for setting a threshold based on the difference value and the average value;
Based on the comparison result of the difference absolute value and the threshold value, the filter processing method for chromatic and parameter setting step of setting the parameters. A program for performing filter processing to remove distortion at the block boundary of an image,
A setting step for setting parameters used in the deblocking filter processing based on the direction and magnitude of the motion vectors of the first block and the second block adjacent to the computer;
Using the parameters set by the setting step, is performed and a filter step of performing deblocking filtering at the boundary between the said first block the second block,
The setting step includes
A difference value calculating step of calculating a horizontal and vertical difference value between the motion vector of the first block and the motion vector of the second block;
An average value calculating step for calculating horizontal and vertical average values of the motion vector of the first block and the motion vector of the second block;
A difference absolute value calculating step for calculating a difference absolute value of each pixel adjacent to a boundary between the first block and the second block;
A threshold setting step for setting a threshold based on the difference value and the average value;
A parameter setting step for setting the parameter based on a comparison result between the absolute difference value and the threshold value .

Images