JP2007116580A - Image processing method and image processing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing method and an image processing device capable of performing an optimal deblock process considering characteristics of display means. <P>SOLUTION: In deblock processing means 502 of the image processing method, luminance value within an image is calculated, then an effect of noise reduction in a dark portion in the image is increased by setting a noise reduction parameter based on a noise reduction parameter function which makes the calculated luminance value to be an argument, thereby subjective image quality of the image can be improved. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a method for performing selective image filtering, and more particularly to an image processing method for removing coding distortion caused by compression coding processing on an image.

  For example, H is an algorithm for compressing and encoding an image signal. 261, H. 263, H. 263+, MPEG-1, MPEG-2, MPEG-4, and MPEG-4AVC (H.264) are widely known. In these, the luminance signal of an image is converted into DCT coefficients by executing DCT (Discrete Cosine Transform) in units of blocks. When the image signal is compressed, the DCT coefficient may be roughly quantized to cause visual image distortion. One type of image distortion is block distortion that occurs along block boundaries in the image.

  In the case of a moving image, there are an I picture that is compressed using only information in a picture (field or frame), and a P picture and a B picture that are compressed using difference information from other pictures based on motion detection information. is there. When these are mixed, the appearance of block distortion differs between pictures.

  Various techniques for reducing the block distortion as described above have been proposed. For example, a deblocking filter is introduced in the line of the F-filter for appendix of the MPEG-4 recommendation as a technique to be performed at the time of decoding. (Refer nonpatent literature 1).

  FIG. 5 is a diagram for explaining a conventional deblocking process using a low-pass filter. FIG. 5A shows the luminance value of the decoded image before the filter process, and FIG. 5B shows the filter process. The subsequent luminance value is shown. In these figures, 10 points on a line orthogonal to the block boundary shown in FIG. 4 are shown as representatives.

  As shown in FIG. 5 (a), the difference dif1 occurs in the luminance level in the line orthogonal to the block boundary indicated by the dotted line, and the block distortion in which the block boundary becomes discontinuous occurs.

Therefore, conventionally, when the difference dif1 in the luminance level of the signal at the block boundary is smaller than a certain threshold value α, as shown in FIG. The block distortion is reduced by eliminating the straight edges.
MPEG-4 Part2 (ISO / IEC 14496-2), pp. 448-450, F. 3 1 Deblocking filter

  The level of white and black that can be expressed, the contrast, the color, and the like differ depending on the display device, and thus the appearance of the image decoded by the display device varies greatly. For example, in a plasma display (PDP), even if the video signal is a completely black screen, it always shines, so that “black floating” occurs in which the black luminance increases. As a result, the brightness of the dark portion of the image is increased in the PDP, and the block distortion of the dark portion is conspicuous as compared with the CRT. However, the conventional deblocking filter performs processing uniformly without considering the characteristics of the final display means, so that there is a problem that sufficient image quality cannot be obtained depending on the display means.

  In view of the above problems, an object of the present invention is to provide an image processing method and an image processing apparatus capable of performing an optimal deblocking process in consideration of characteristics of display means.

  In order to solve the above problems, a first technical invention includes a calculation step of calculating a luminance value in an input image, a parameter setting step of setting a noise removal parameter according to the calculated luminance value, and the noise removal And a noise removal step of performing noise removal on the input image based on the parameter.

  According to a second technical invention, in the first technical method, the parameter setting step sets a noise removal parameter based on a noise removal parameter function having the calculated luminance value as an argument.

  According to a third technical invention, in the second technical method, the lower the calculated luminance value, the higher the noise removal effect.

  According to a fourth technical invention, in the third technical method, the noise removal parameter is a threshold value for determining whether or not to perform noise removal.

  According to a fifth technical invention, in the third technical method, the noise removal parameter is a parameter for changing a cutoff frequency of a noise removal filter.

  According to a sixth technical invention, in the second technical invention, a plurality of the noise removal parameter functions are retained, and an optimum noise removal parameter is selected from the plurality of retained noise removal parameter functions according to the connected display means. A function setting step for setting a function is included.

  According to a seventh technical invention, in the sixth technical invention, the seventh technical invention has a detection step of detecting a type or characteristic of the connected display means, and the first-stage function setting step includes a plurality of noise removal parameter functions held in the plurality. From the above, an optimum noise removal parameter function is set according to the detected type or characteristic of the connected display means.

  According to an eighth technical invention, in the seventh technical invention, the characteristic detected by the detecting step is information on a degree of black floating of the connected display means.

  According to a ninth technical invention, in the seventh technical invention, the detecting step automatically determines the type of the display means to be connected.

  According to a tenth technical invention, in the seventh technical invention, the detection step is characterized in that an observer designates a type of display means to be connected.

  The eleventh technical invention includes a specifying step in which, in the sixth technical invention, an observer of the connected display means designates an arbitrary noise removal parameter function from among the plurality of retained noise removal parameter functions. It is characterized by.

  According to a twelfth technical invention, a calculation unit that calculates a luminance value in an input image, a setting unit that sets a noise removal parameter according to the calculated luminance value, and a noise in the input image based on the noise removal parameter An image processing apparatus including noise removal means for performing removal.

  According to the image processing method and the image processing apparatus of the present invention, the noise removal parameter is adaptively set according to the luminance value of the image, thereby increasing the noise removal effect in the dark part of the image and visually displaying the image by the black floating of the display means. The encoding distortion in the dark part of the image that is conspicuous is reduced, and the subjective image quality observed on the display means is improved.

  Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing an embodiment of an image processing method and an image processing apparatus according to the present invention.

  In FIG. 1, 500 is an input unit, 501 is a decoding unit, 502 is a deblocking processing unit, 503 is an output unit, 504 is a control unit, 505 is a parameter setting unit, 506 is a parameter storage unit, and 507 is a display unit.

  Compressed image data is input from the input unit 500. The decoding unit 501 performs decoding processing on the compressed image data input via the input unit 500, outputs the restored image to the deblock processing unit 502, and compresses the compressed image data indicating the format of the restored image and the content of the compression processing. Image information is read and output to the control means 504.

  On the other hand, the parameter storage unit 506 stores in advance functions of deblock parameters α and β that improve the apparent image quality of the display image, assuming that the display unit 507 is black. These deblocking parameters are used when determining whether or not to perform the deblocking process on the decoded image input from the decoding unit 501. FIG. 3 is a graph showing some examples of functions of the deblocking parameters α and β stored in the parameter information storage unit 506. The value of the deblocking parameter decreases as the value of the Y signal representing the luminance value of the pixel decreases. Is set to a function that increases. FIG. 3A is a graph showing examples of types of deblocking parameter functions. For example, a function that decreases linearly as a Y signal increases, a function that decreases by drawing a curve, or a stepwise decrease. There are functions. FIG. 3B is a graph showing an example in which the degree of decrease in the deblock parameter function is changed.

  Since the stronger the deblocking process is performed, the larger the deblocking parameter value is, the stronger the deblocking process is performed the lower the value of the Y signal, that is, the darker the image. Further, different functions may be used so that the deblocking parameters α and β have different values.

  In addition to the parameters α and β used for determining whether or not to perform the deblocking process as the deblocking parameter, the frequency characteristics of the filter used in the deblocking process, more specifically, the cutoff frequency May be used.

  The control unit 504 uses both the compressed image information input from the decoding unit 501 and the parameter function information input from the parameter storage unit 506, and uses a plurality of deblocking parameters stored in the parameter storage unit 506. Parameter setting information for selecting and setting one of them is output to the parameter setting means 505.

  In addition to the decoded image, the compressed image information input from the decoding unit 501 includes the picture type, motion vector, reference image, and the like of the decoded image used at the time of decoding. You may use it. In this case, the subjective image quality of the image after the deblocking process can be improved.

  The parameter setting unit 505 selects one of a plurality of deblocking parameters stored in the parameter storage unit 506 based on the parameter setting information input from the control unit 504 and outputs the selected one to the deblocking processing unit 502.

  The deblocking processing unit 502 performs deblocking processing on the image input from the decoding unit 501 using the deblocking parameter input from the parameter setting unit 505, and outputs it to the display unit 507 via the output unit 503.

  FIG. 2 is a diagram showing an embodiment of the deblocking processing unit 502. In FIG. 2, 11 is an input unit, 12 is a block boundary determination unit, 13a and 13b are selection units, 16 is a filter processing unit, 18 is an output unit, and 19 is an input unit.

  The decoded image input via the input unit 11 is input to the block boundary determination unit 12, the selection unit 13a, the selection unit 13b, and the filter processing unit 16.

The block boundary determination unit 12 determines whether or not to perform the deblocking process on the pixel from the decoded image input via the input unit 11 and the deblocking parameter input via the input unit 19. Here, it is assumed that the deblocking target pixel values input via the input unit 11 are p3, p2, p1, p0, q0, q1, q2, and q3 in FIG. However, each pixel value is obtained by adding a symbol to the order of arrangement of pixel positions. Also, parameters input via the input unit 19 are α and β. Here, as shown in FIG.
dif1 = | p0-q0 |,
dif2 = | p1-p0 |,
dif3 = | q1-q0 |,
And When the conditions dif1 <α, dif2 <β, and dif3 <β are satisfied, a flag DebOn indicating that the deblocking process is performed is output to the selection unit 13a and the selection 13b.

  In addition to viewing the luminance difference between pixels as block boundary determination, when compressed image data can be used, encoded information such as a picture type of a decoded image, a motion vector, and a reference image may be used. . In this case, it is possible to switch the number of pixels used in the filter and the cutoff frequency of the filter according to the encoding information of the block including the target pixel.

  The filter processing unit 16 applies a low-pass filter to the pixels p3, p2, p1, p0, q0, q1, q2, q3 of the decoded image input via the input unit 11, and determines the pixel after the filter application. It outputs to the selection part 13b.

  Based on the flag DebOn input from the block boundary determination unit 12, the selection unit 13 a and the selection unit 13 b select either the pixel before deblocking via the input 11 or the pixel after deblocking that is the output of the filter processing unit 16. Select and output to the output unit 18.

  FIG. 6 is a flowchart showing a deblocking processing method. In step S09, deblock parameters α and β are set using the average luminance value of the small area around the target pixel as an argument of the deblock parameter function. Here, the size of the small region may be set to 1 × 1, and the luminance value of the target pixel may be used as an argument of the deblock parameter function as it is.

  In step S10, the horizontal line of the target pixel is input as a deblocking target pixel. In step S11, it is determined whether or not the pixel is a deblocking target pixel. If it is a deblocking target pixel, the process proceeds to step S12, and a deblocking process is performed. If it is not a target block, the process proceeds to step S13.

  In step S13, the vertical line of the target pixel is input as the deblock target pixel. In step S14, it is determined whether or not the pixel is a deblocking target pixel. If it is a deblocking target pixel, the process proceeds to step S15 and a deblocking process is performed. If it is not a target block, the process proceeds to step S16.

  In step S16, it is determined whether or not an unprocessed pixel exists in the target block. If there is an unprocessed pixel, the process proceeds to step S17. If there is no target pixel, the process proceeds to step S18. In step S17, the target pixel is changed to an unprocessed pixel, and the processes in and after step S09 are repeated.

  In step S18, it is determined whether or not there is an unprocessed block on the screen. If there is an unprocessed block, the process proceeds to step S19. If there is no unprocessed block on the screen, the deblocking process on the screen is terminated.

  In step S19, the target block is changed to an unprocessed block, and the processes in and after step S09 are repeated.

  With the operation as described above, the image processing method of FIG. 1 performs strong deblocking processing in the dark portion of the image restored from the compressed image data, against block distortion that is noticeable due to black floating of the display means 507. With such a configuration, it is possible to realize a high-quality display even in a dark part in an image that has been visually problematic in the conventional deblocking process. Here, each component of the block diagram shown in FIG. 1 may be realized by hardware or may be realized by software.

  The deblocking processing unit 502 in FIG. 1 may perform mosquito noise removal processing. In this case, increasing the effect of removing mosquito noise that occurs in the dark part of the image by changing the parameter that specifies the strength of mosquito noise removal according to the brightness value of the image is effective in improving subjective image quality. It is.

  In FIG. 1, the decoding unit 501 is not necessarily required. When the decoding unit 501 does not exist, a decoded image is input from the input unit 500. In this case, information indicating where the block boundary exists may be received from the outside.

  Note that the parameter setting unit 505 in FIG. 1 does not select a parameter from the parameter storage unit 506 but may calculate and set the parameter each time using a predetermined calculation formula.

(Embodiment 2)
FIG. 7 is a block diagram showing an embodiment of the image processing method and the image processing apparatus of the present invention having the function of setting the optimum deblocking parameter according to the display means according to the present embodiment.

  In FIG. 7, reference numeral 600 denotes control means, and 601 denotes parameter storage means. In the figure, blocks that perform the same operations as those in the block diagram of the encoding method shown in FIG.

  The parameter storage unit 601 stores in advance a plurality of deblocking parameter functions that improve the apparent image quality of the display image in accordance with the degree of black floating of the display unit 507. FIG. 3B shows an example of the deblock parameter function stored in the storage unit 601. It is more effective to improve the subjective image quality by setting a function having a larger inclination in the drawing as the degree of black floating of the display means 507 is stronger.

  The control unit 600 uses both the parameter function information input from the parameter storage unit 601 and the information about the degree of black float of the display unit 507 input from the display unit 507, and uses a plurality of information stored in the parameter storage unit 601. One of the deblocking parameter functions is selected or a deblocking parameter function is set by calculation.

  As a first setting method of the deblocking parameter function, information on the brightness actually observed on the display unit 507 when the pixel luminance value Y = 0 is input from the display unit 507 to the control unit 504. There is a method for storing in advance which is the most suitable deblocking function.

  As a second setting method of the deblocking parameter function, an optimal setting method of the deblocking parameter function when detailed information on the degree of black float of the display unit 507 connected in the control unit 600 is obtained will be described below. To do.

  The degree of brightness that a person actually perceives on the display with respect to the luminance value Y of the pixel is represented by Z, and the relationship between Y and Z when viewing an image on a certain reference CRT is indicated by a broken line in FIG. Is represented by a function g1 (Y). Further, it is assumed that the relationship between Y and Z when an image is viewed on a certain PDP is represented by a function g2 (Y) indicated by a solid line in FIG. In the figure, the function g2 (Y) representing the visual characteristics of the PDP is shown to have a certain level of brightness even at a low luminance value, and the degree of black float is ΔZ = g2 (Y) -g1 (Y) expressed. Also, the broken line shown in FIG. 8B is a curve representing the optimum deblocking parameter α in a certain reference CRT, and this curve is represented by α = f1 (Y) with respect to the luminance value Y of the pixel. . Also, the practice shown in FIG. 5B is a curve representing the optimum deblocking parameter α on the black floating PDP, and is represented by α = f2 (Y). The higher the degree of black floating, the higher the deblocking parameter value and the stronger deblocking processing will produce a higher quality image, so the larger the black floating level, the larger the value according to the size of ΔZ. When the function T (ΔZ) is used, the optimal deblocking parameter α on the PDP can be obtained as α = f2 (Y) = f1 (Y) + T (ΔZ). However, T (ΔZ) is a function that takes a large value in accordance with the degree of black float ΔZ. For example, T (ΔZ) = C × ΔZ, where C is a constant, is stored in the parameter storage unit 601 in advance. It shall be remembered. Similarly, f1 (Y) and g1 (Y) are also stored in the parameter storage unit 601 in advance. Also, g2 (Y) is input from the display means 507, and f2 (Y) is calculated from f1 (Y), g1 (Y), g2 (Y), and T (ΔZ).

  The function f2 (Y) is indicated by a solid line in FIG. Similarly, the deblock parameter β can also be calculated. In addition to the deblocking parameters α and β, the optimum filter cutoff frequency on the PDP can be obtained from the optimum cutoff frequency of the noise removal filter on the reference CRT by the same method. .

  As a third setting method of the deblocking parameter function, there is a method in which the observer manually sets while viewing the display means 507. In this case, it is not necessary to input black floating information from the display unit 507 to the control unit 600.

  As a fourth setting method of the deblock parameter function, there is a method in which the control unit 600 acquires the type of the display unit 507. For example, when the display means 507 is a CRT, the parameter function f1 (Y) of FIG. 8B is used, and when the display means 507 is a PDP, the parameter function f2 (Y) of FIG. It should just be used.

  As a fifth setting method of the deblocking parameter function, the observer may input the type of the display unit 507. In this case, the designation of the type of the display unit 507 by the user is input to the control unit 600. As the input means, there are a method of directly inputting with a remote controller (not shown), a method of selecting from options displayed on the display means 507, and the like.

  Note that brightness information around the display unit 507 may be input to the control unit 600 and used for setting the deblocking parameter function. This makes it possible to remove block distortion that is conspicuous due to ambient brightness.

  Parameter setting for selecting one of the deblocking parameters stored in the parameter storage unit 601 by using the compressed image information input from the decoding unit 501 after setting an optimal deblocking parameter function Information is output to the parameter setting means 505.

  By the operation as described above, the image processing method of FIG. 7 sets the optimum deblocking parameter according to the degree of black floating of the display unit 507, and performs the deblocking process on the image restored from the compressed image data. . With such a configuration, when the degree of black floating is small, the deblocking process is weakened so that it is not excessively blurred, and when the degree of black floating is large, the deblocking process is strengthened. By strongly removing the block distortion, it is possible to realize a high-quality display even when display means having different levels of black float are connected.

  It should be noted that the present invention is not limited to the above-described embodiment, and various changes and modifications can be made without departing from the scope of the present invention.

  Each functional block in the block diagrams shown in FIGS. 1, 2, 6 and 7 is typically realized as an LSI which is an integrated circuit. This LSI may be made into one chip or a plurality of chips. (For example, the functional blocks other than the memory may be integrated into one chip.) Although the LSI is used here, it may be referred to as an IC, a system LSI, a super LSI, or an ultra LSI depending on the degree of integration.

  Further, the method of circuit integration is not limited to LSI's, and implementation using dedicated circuitry or general purpose processors is also possible. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI, or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

  Further, if integrated circuit technology comes out to replace LSI's as a result of the advancement of semiconductor technology or a derivative other technology, it is naturally also possible to carry out function block integration using this technology. Biotechnology can be applied.

  INDUSTRIAL APPLICABILITY The image processing method and the image processing apparatus according to the present invention have an effect that the compression-coded image can be deblocked according to the display performance. It is.

The block diagram of the deblocking method in Embodiment 1 of this invention Block diagram of the deblocking processing unit 502 Graph showing examples of deblocking parameters The figure which shows the example of the pixel value input into the deblocking processing means 502 The figure for demonstrating the operation | movement of the conventional deblocking process The flowchart for demonstrating operation | movement of the deblocking process of Embodiment 1 of this invention. Block diagram of a deblocking method in Embodiment 2 of the present invention Graph to explain how to derive optimal deblocking parameters

Explanation of symbols

500 Input unit 501 Decoding unit 502 Deblock processing unit 503 Output unit 504, 600 Control unit 505 Parameter setting unit 506, 601 Parameter storage unit 507 Display unit 11, 19 Input unit 18 Output unit 12 Block boundary determination unit 13a, 13b Selection unit 16 Filter processing section

Claims (12)

A calculation step for calculating a luminance value in the input image, a parameter setting step for setting a noise removal parameter according to the calculated luminance value, and a noise removal step for performing noise removal on the input image based on the noise removal parameter And an image processing method. The image processing method according to claim 1, wherein the parameter setting step sets a noise removal parameter based on a noise removal parameter function having the calculated luminance value as an argument. The image processing method according to claim 2, wherein the noise removal parameter function has a value that increases a noise removal effect as the calculated luminance value is lower. The image processing method according to claim 3, wherein the noise removal parameter is a threshold value for determining whether or not to perform noise removal. The image processing method according to claim 3, wherein the noise removal parameter is a parameter for changing a cutoff frequency of a noise removal filter. A function setting step of holding a plurality of the noise removal parameter functions and setting an optimum noise removal parameter function according to a connected display means from the plurality of held noise removal parameter functions. The image processing method according to claim 2. Furthermore, it has a detection step for detecting the type or characteristic of the connected display means, and the previous function setting step includes a step of detecting the detected connected display means from among the plurality of held noise removal parameter functions. The image processing method according to claim 6, wherein an optimum noise removal parameter function is set according to the type or characteristic. 8. The image processing method according to claim 7, wherein the characteristic detected by the detection step is information on a degree of black floating of the connected display means. 8. The image processing method according to claim 7, wherein the detecting step automatically determines the type of display means to be connected. 8. The image processing method according to claim 7, wherein in the detection step, an observer designates a type of display means to be connected. 7. The image processing according to claim 6, further comprising a designation step in which an observer of the connected display means designates an arbitrary noise removal parameter function from among the plurality of held noise removal parameter functions. Method. A calculation means for calculating a luminance value in the input image; a parameter setting means for setting a noise removal parameter based on a noise removal parameter function having the calculated luminance value as an argument; and the input image based on the noise removal parameter. An image processing apparatus comprising noise removal means for performing noise removal.

Images