JP2011022656A - Image processor and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To effectively prevent the difference in level of gradation in the periphery of a highlighted section from being made to be conspicuous. <P>SOLUTION: An image processor is provided with: a first coefficient generation part for calculating a first coefficient based on a difference between the luminance value of a targeted pixel in an image and a mean luminance value in peripheral pixels arranged in the periphery of the targeted pixel; a second coefficient generation part for calculating a second coefficient different from the first coefficient with respect to the targeted pixel at random; and a dither processing part for correcting the luminance value of the targeted pixel by using the first coefficient and the second coefficient. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing program.

  When shooting with a camera, if there is an extremely bright object in the field, a part of the shot image will be overexposed. Unlike the other portions, the whiteout portion (highlight portion) has no gradation at all, so that the contour is raised due to the gradation step, giving an unnatural impression to the observer.

  Patent Document 1 discloses a signal processing technique for making the contour inconspicuous. This technique intentionally adds random noise to a signal close to the saturation level. As a result, a dither effect (roughness) can be given around the highlight portion, and the outline can be made ambiguous.

JP-A-2005-72835

  However, it has been found that even if this signal processing technique is applied to a digital camera as it is, the dither effect does not necessarily appear depending on the application method and usage situation. Further, there is a possibility that the dither effect may appear in a portion where it does not have to appear.

  Therefore, an object of the image processing apparatus and the image processing program of the present invention is to effectively make the gradation step around the highlight portion inconspicuous.

An image processing apparatus according to the present invention includes a first coefficient generation unit that obtains a first coefficient based on a difference between a luminance value of a target pixel in an image and an average luminance value of peripheral pixels arranged around the target pixel. The second coefficient generation unit that randomly obtains a second coefficient different from the first coefficient for the target pixel, and the luminance value of the target pixel is corrected using the first coefficient and the second coefficient A dither processing unit.
The dither processing unit may calculate a correction value using at least the first coefficient and the second coefficient, and then add the calculated correction value to the luminance value of the target pixel.
The first coefficient is set to 0 when the luminance value of the target pixel is equal to the average luminance value of the surrounding pixels, and when the first coefficient is 0, the dither processing is performed. The correction value calculated by the unit may be zero.
Further, 0 may be included as a value that can be taken by the second coefficient, and when the second coefficient becomes 0, the correction value calculated by the dither processing unit may be 0.
The image processing apparatus further includes a third coefficient generation unit that obtains a third coefficient based on the luminance value of the target pixel, and the dither processing unit uses the third coefficient in addition to the first coefficient and the second coefficient. The correction value may be calculated.
The third coefficient is set to be 0 when the luminance value of the pixel of interest is equal to or less than a predetermined threshold, and is calculated by the dither processing unit when the third coefficient is 0. The correction value may be 0.

  An image processing program according to the present invention includes a first coefficient generation step for obtaining a first coefficient based on a difference between a luminance value of a target pixel in an image and an average luminance value of peripheral pixels arranged around the target pixel. The second coefficient generation step for randomly obtaining the second coefficient different from the first coefficient for the target pixel, and the luminance value of the target pixel is corrected using the first coefficient and the second coefficient The dither processing step to be performed is realized by a computer.

  According to the image processing apparatus and the image processing program of the present invention, the gradation step around the highlight portion can be effectively made inconspicuous.

It is a block diagram which shows the structure of the digital camera 10 in embodiment of this invention. It is an example of LUT27 used when calculating the coefficient A. FIG. It is an example of LUT28 used when calculating the coefficient B. FIG. It is an example of LUT29 used when calculating the coefficient C. It is a flowchart which shows the flow at the time of a dither process. (A) It is a figure which shows the example of the JPEG image 30 before a dither process. (B) It is a figure which shows the correspondence of the X coordinate value and Y signal value of the pixel located on the straight line Z in the JPEG image 30 before a dither process. (C) It is a figure which shows the correspondence of the X coordinate value and Y 'signal value of the pixel located on the straight line Z in the JPEG image 30 after a dither process.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this embodiment, a digital camera is used as an example of the image processing apparatus of the present invention. FIG. 1 is a block diagram showing a configuration of a digital camera 10 according to an embodiment of the present invention.

  As shown in FIG. 1, the digital camera 10 includes an imaging lens 11, an imaging device 12, an A / D conversion unit 13, a buffer memory 14, an image processing unit 15, a control unit 16, a display unit 17, an operation unit 18, and a memory 19. A recording I / F unit 20, a recording medium 21, and a bus 22.

  The imaging lens 11 forms a subject image on the imaging surface of the imaging element 12. Note that the imaging lens 11 may be composed of a plurality of lenses such as a zoom lens and a focus lens. The imaging element 12 photoelectrically converts the subject light that has passed through the imaging lens 11 and outputs analog image signals corresponding to the R, G, and B colors. The image sensor 12 acquires an image of the subject when a release button 25 described later is fully pressed. Note that the images include moving images in addition to still images.

  An image signal output from the image sensor 12 is input to the A / D converter 13. The A / D conversion unit 13 performs A / D conversion on the analog image signal output from the image sensor 12 and changes it to a digital image signal. The digital image signals are collected into one frame and recorded in the buffer memory 14 as image data. The buffer memory 14 temporarily records image data in the pre-process and post-process of image processing by the image processing unit 15.

The image processing unit 15 performs image processing on the image data recorded in the buffer memory 14. Examples of the image processing include well-known white balance adjustment, color interpolation, gradation conversion processing, and contour enhancement processing. The image processing unit 15 also executes processing for compressing in JPEG (Joint Photographic Experts Group) format and the like, and processing for decompressing and restoring the compressed data. For example, the number of gradations before compression processing is 12 bits, and the number of gradations after compression processing is 8 bits. Hereinafter, the image after the compression processing is referred to as a JPEG image.
The image processing unit 15 has functions of a coefficient generation unit 23 and a dither processing unit 24. The coefficient generation unit 23 calculates the coefficients A, B, and C necessary for dither processing using the lookup tables LUT27, LUT28, and LUT29 for each pixel included in the JPEG image. The calculation of the coefficient will be described later.
The dither processing unit 24 performs dither processing on the JPEG image. A pixel in a JPEG image is composed of an image signal value indicating a luminance component (Y) (hereinafter referred to as Y signal value) and an image signal value indicating a color difference component (hereinafter referred to as Cr, Cb signal value). The dither processing unit 24 performs dither processing by adding the dither signal value to the Y signal value of each pixel constituting the JPEG image according to the following equation (1). 2 ^α is the amplitude of the coefficient C.

Each coefficient is obtained as follows. Note that a pixel to be subjected to dither processing is referred to as a target pixel. Also, the horizontal direction of the JPEG image is the X axis and the vertical direction is the Y axis.
First, the coefficient generation unit 23 uses the LUT 27 to calculate a coefficient A corresponding to the Y signal value of the target pixel. FIG. 2 is an example of the LUT 27. The horizontal axis represents the Y signal value, and the vertical axis represents the coefficient A. Further, when the number of gradations of the JPEG image is 8 bits, 0 ≦ Y signal value ≦ 255. The LUT 27 is set so that the coefficient A = 0 when the Y signal value is less than or equal to the threshold value (0 ≦ Y signal value ≦ 230 in FIG. 2), and the Y signal value is higher than the threshold value (230 in FIG. 2). <Y signal value ≦ 255), the higher the Y signal value, the larger the coefficient A is set.
Next, the coefficient B is obtained as follows. First, the coefficient generation unit 23 sets an area of vertical 7 pixels × horizontal 7 pixels around the target pixel. Then, the coefficient generation unit 23 calculates the average value of the Y signal values of the 48 pixels that are included in this region and exclude the pixel of interest, and sets the average Y signal value of the peripheral pixels. Further, the coefficient generation unit 23 calculates the difference value D according to the following equation.
Difference value D = | Average Y signal value of surrounding pixels−Y signal value of target pixel |
Then, the coefficient generation unit 23 uses the LUT 28 to calculate a coefficient B corresponding to the difference value D between the Y signal value of the target pixel and the average Y signal value of surrounding pixels. FIG. 3 is an example of the LUT 28. The horizontal axis is the difference value D, and the vertical axis is the coefficient B. The LUT 28 is set so that the coefficient B = 0 when the difference value D = 0, and the coefficient B is set to increase as the difference value D increases.
Next, the coefficient generation unit 23 uses the LUT 29 to calculate a coefficient C corresponding to the X coordinate value of the target pixel. FIG. 4 is an example of the LUT 29. The horizontal axis is the X coordinate value, and the vertical axis is the coefficient C that is predetermined as a random value. For example, the -2 ^alpha ≦ coefficient C ≦ + 2 ^α. In the example of FIG. 4, α = 3.

As described above, the dither signal value is a pixel in which the Y signal value is near the saturation level, and the Y signal value of the pixel having a difference between the Y signal value and the average Y signal value of the surrounding pixels is It is set to give variation.
The control unit 16 performs overall control of the digital camera 10 according to a predetermined sequence program. The display unit 17 displays various images under the control of the control unit 16. Various images displayed on the display unit 17 include images acquired by imaging, images recorded on a recording medium 21 described later, menu images, and the like. The operation unit 18 includes a release button 25, a cross key 26, and the like. The memory 19 records LUT27, LUT28, LUT29, and the like. The release button 25 is operated during imaging. The cross key 26 is operated on the above menu image or the like. The state of the release button 25 and the cross key 26 is detected by the control unit 16, and a sequence based on the detected state of the button or key is executed.

  The recording I / F unit 20 includes a connector for connecting the recording medium 21. When the recording I / F unit 20 and the recording medium 21 are connected, data writing / reading is executed on the recording medium 21. The bus 22 connects the buffer memory 14, the image processing unit 15, the control unit 16, the display unit 17, and the recording I / F unit 20, thereby executing data and signal input / output.

  FIG. 5 is a flowchart showing a flow during dither processing.

  Step S101 is processing for reading a JPEG image. The control unit 16 reads from the recording medium 21 a JPEG image that is a target of dither processing.

  Step S102 is processing for calculating a Y signal value. The coefficient generator 23 calculates the Y signal value of the target pixel.

  Step S103 is a process for calculating the coefficient A. The coefficient generation unit 23 uses the LUT 27 to calculate the coefficient A corresponding to the Y signal value calculated in step S102.

  Step S104 is processing for determining whether or not the coefficient A ≠ 0. When the coefficient A is not 0 (when the determination in step S104 is YES), the coefficient generation unit 23 proceeds to step S105. On the other hand, when the coefficient A = 0 (when the determination in step S105 is NO), the coefficient generation unit 23 proceeds to step S111 described later.

  Step S105 is processing for calculating the average Y signal value of the peripheral pixels. The coefficient generation unit 23 calculates the average Y signal value of the peripheral pixels around the target pixel.

  Step S106 is a process of calculating the difference value D. The coefficient generation unit 23 calculates a difference value D between the Y signal value of the target pixel and the average Y signal value of surrounding pixels.

  Step S107 is processing for calculating the coefficient B. The coefficient generation unit 23 uses the LUT 28 to calculate a coefficient B according to the difference value D calculated in step S106.

  Step S108 is a process for determining whether or not the coefficient B ≠ 0. When the coefficient B ≠ 0 (when the determination in step S108 is YES), the coefficient generation unit 23 proceeds to step S109. On the other hand, when the coefficient B = 0 (when the determination in step S108 is NO), the coefficient generation unit 23 proceeds to step S111 described later.

  Step S109 is processing for calculating an X coordinate value. The coefficient generation unit 23 calculates the X coordinate value of the target pixel.

  Step S110 is a process for calculating the coefficient C. The coefficient generation unit 23 uses the LUT 29 to calculate a coefficient C corresponding to the X coordinate value calculated in step S109.

  Step S111 is a process of calculating the Y ′ signal value. The dither processing unit 24 calculates the Y ′ signal value using the Y signal value, the coefficient A, the coefficient B, and the coefficient C of the target pixel. If the determination in step S104 or step S108 is NO, the dither signal value is zero. Therefore, in such a pixel, the Y signal value before the dither process is the same as the Y ′ signal value after the dither process. Further, the coefficient generation unit 23 calculates Cb and Cr signal values of the target pixel.

  Step S112 is a process of determining whether or not the Y ′ signal value has been calculated for all the pixels. When the dither processing unit 24 has calculated the Y ′ signal value for all the pixels (when the determination in step S112 is YES), the dither processing unit 24 proceeds to step S113. On the other hand, if the dither processing unit 24 has not calculated the Y ′ signal value for all the pixels (when the determination in step S112 is NO), the dither processing unit 24 returns to step S102.

Step S113 is a process of recording the JPEG image after the dither process. The dither processing unit 24 combines the Y ′ signal value calculated in step S111 with the Cb and Cr signal values for each pixel, and generates a JPEG image after the dither processing. Then, the control unit 16 records the JPEG image after the dither processing on the recording medium 21 and ends the series of processing.
For example, FIG. 6A shows an example of a JPEG image 30 before dither processing. The Y signal values of the pixels included in the region K are all the same, for example, Y signal value = 255. Similarly, the Y signal values of the pixels included in the region L are all the same, for example, Y signal value = 245. The Y signal values of the pixels included in the region M are all the same, and Y signal value = 230. Further, the Y signal values of the pixels included in the region N are all the same, and Y signal value = 220. FIG. 6B is a diagram showing a correspondence relationship between the X coordinate value and the Y signal value of the pixel located on the straight line Z in the JPEG image 30 before the dither process.
In this case, the regions K, L, and M are regions composed of pixels whose Y signal values are near the saturation level, that is, highlight portions, and the difference values of the Y signal values between the regions K and L, and the regions The difference value of the Y signal value between L and region M is large. Therefore, gradation steps are generated at the boundary between the region K and the region L and at the boundary between the region L and the region M.

FIG. 6C is a diagram illustrating a correspondence relationship between the X coordinate value and the Y ′ signal value of the pixel located on the straight line Z in the JPEG image 30 after the dither processing. The boundary between the region K and the region L and the boundary between the region L and the boundary M are made ambiguous by the dither processing. On the other hand, the Y ′ signal value remains uniform at the boundary between the region K and the region L and at portions other than the boundary between the region L and the boundary M.
As described above, the dither processing unit 24 calculates Y ′ signal value = Y signal value + (coefficient A × coefficient B × coefficient C) / ^2α . For example, as the Y signal value of the target pixel is higher, the gradation step becomes more conspicuous, so the necessity for dither processing is higher. In the above formula, when the Y signal value of the target pixel is low, the coefficient A = 0, and the higher the Y signal value of the target pixel, the larger the coefficient A. Therefore, the dither processing unit 24 can add the dither signal value only to the highlight portion by multiplying by the coefficient A as in the above equation.
Further, as the difference value D between the Y signal value of the target pixel and the average Y signal value of the surrounding pixels is larger, the gradation step is more conspicuous. Therefore, the necessity of dither processing is higher. The nature is low. In the above formula, when the difference value D = 0, the coefficient B = 0, and the larger the difference value D, the larger the coefficient B. Therefore, the dither processing unit 24 can add the dither signal value only to the boundary portion where there is a difference in the Y signal value by multiplying by the coefficient B as in the above formula.
Further, generally, in an image of the object scene, pixels that are spatially close to each other tend to have Y signal values close to each other. In the above formula, the coefficient C corresponding to the X coordinate value of the target pixel is multiplied. Thus, the dither processing unit 24 can add random dither signal values even between adjacent pixels.
That is, in the digital camera 10 of the present embodiment, the dither effect does not appear except in the vicinity of the highlight portion, and the dither effect appears only in the periphery of the highlight portion. Therefore, according to the digital camera 10 of the present embodiment, the gradation step around the highlight portion can be effectively made inconspicuous. In addition, since the dither effect does not appear in areas other than the highlight area such as the area inside the highlight area, a natural impression can be given to the observer.

  In the above embodiment, an example is shown in which the coefficient C corresponding to the X coordinate value of the target pixel is calculated using the LUT 29, but the present invention is not limited to this. For example, the dither processing unit 24 performs random number calculation for each pixel of interest, calculates a random value, and sets the random value as the coefficient C.

  In the above embodiment, an example in which α = 3 in the LUT 29 is shown, but the value of α is not limited to this. Further, the value of α may be set in consideration of ISO sensitivity, saturation, and edge degree. For example, if the ISO sensitivity is high, the S / N ratio of the image is high, and the gradation steps around the highlight portion in the image tend not to be noticeable. Therefore, there is no problem even if the value of α is set small. Further, the value of α may be increased as the intensity of saturation correction is lower, and the value of α may be increased as the intensity of edge enhancement is lower.

  In the above embodiment, an example is shown in which the average value of the Y signal values of 48 pixels excluding the target pixel is calculated when calculating the average Y signal value of the surrounding pixels. However, the present invention is not limited to this. . For example, an average value of Y signal values of 49 pixels including the target pixel may be calculated. Moreover, although the example which sets the area | region of vertical 7 pixels x horizontal 7 pixels was shown, the magnitude | size of an area | region is not restricted to this. For example, the smaller the region, the finer the Y ′ signal value increments as shown in FIG.

  In the above-described embodiment, an example in which the LUT 27 is a curve has been described, but the present invention is not limited to this. For example, a straight line of a linear function may be used. Further, when 230 <Y signal value ≦ 240, the coefficient A = 1 may be set, and when 240 <Y signal value ≦ 255, the coefficient A = 2 may be set. Moreover, although the example which becomes 0 <= coefficient A <= 2 was shown, the value of the coefficient A is not limited. The same applies to the LUT 28 and the coefficient B.

  In the above embodiment, an example is shown in which the coefficients are calculated in the order of the coefficient A, the coefficient B, and the coefficient C. However, the present invention is not limited to this. For example, the coefficient generation unit 23 may calculate the coefficients in the order of the coefficient B, the coefficient A, and the coefficient C, or may calculate three coefficients in parallel.

  In the above embodiment, an example in which the dither signal value is calculated using the coefficient A, the coefficient B, and the coefficient C has been described. However, the present invention is not limited to this. For example, the dither processing unit 24 may calculate the dither signal value using only the coefficient B and the coefficient C without using the coefficient A.

  In the above embodiment, the dither signal value is calculated by multiplying the coefficient A, the coefficient B, and the coefficient C. However, the present invention is not limited to this. For example, using addition, subtraction, division, or the like, an expression is obtained in advance such that the dither signal value becomes 0 when at least one of coefficient A, coefficient B, and coefficient C is 0. . Then, the dither processing unit 24 may calculate the dither signal value using this equation.

  In the above embodiment, the digital camera 10 has been described as an example, but the present invention is not limited to this. For example, the present invention can be similarly applied to a digital camera having a configuration other than that shown in FIG. Moreover, the image processing apparatus such as a computer capable of executing the function illustrated in FIG. 1 and the flow of the flowchart illustrated in FIG. 5 may be used. In this case, for example, it is possible to electrically connect to an electronic device such as a recording medium of a memory card or a digital camera, and an image recorded on the recording medium or an image acquired by the digital camera can be captured. Any form is acceptable. Moreover, the program which can perform the function shown in FIG. 1 and the flow of the flowchart shown in FIG. 5 with a computer may be sufficient. In the case of such a program, the program is preferably recorded on a computer-readable recording medium such as a memory card, an optical disk, or a magnetic disk.

DESCRIPTION OF SYMBOLS 10 ... Digital camera, 15 ... Image processing part, 16 ... Control part, 23 ... Coefficient generation part, 24 ... Dither processing part

Claims (7)

A first coefficient generation unit for obtaining a first coefficient based on a difference between a luminance value of a target pixel in the image and an average luminance value in peripheral pixels arranged around the target pixel;
A second coefficient generation unit that randomly calculates a second coefficient different from the first coefficient for the target pixel;
A dither processing unit that corrects a luminance value of the target pixel using the first coefficient and the second coefficient;
An image processing apparatus comprising: The image processing apparatus according to claim 1.
The dither processing unit calculates a correction value using at least the first coefficient and the second coefficient, and then adds the calculated correction value to the luminance value of the target pixel. The image processing apparatus according to claim 2,
The first coefficient is set to be 0 when the luminance value of the target pixel and the average luminance value of the surrounding pixels are equal,
When the first coefficient is zero, the correction value calculated by the dither processing unit is zero. In the image processing device according to any one of claims 2 and 3,
The possible value of the second coefficient includes 0,
The image processing apparatus according to claim 1, wherein when the second coefficient is 0, the correction value calculated by the dither processing unit is 0. The image processing apparatus according to any one of claims 2 to 4,
A third coefficient generation unit for obtaining a third coefficient based on a luminance value of the target pixel;
The dither processing unit calculates the correction value using the third coefficient in addition to the first coefficient and the second coefficient. The image processing apparatus according to claim 5.
The third coefficient is set to be 0 when the luminance value of the target pixel is equal to or less than a predetermined threshold,
The image processing apparatus according to claim 1, wherein when the third coefficient is zero, the correction value calculated by the dither processing unit is zero. A first coefficient generation step for obtaining a first coefficient based on a difference between a luminance value of a target pixel in the image and an average luminance value in peripheral pixels arranged around the target pixel;
A second coefficient generation step for randomly obtaining a second coefficient different from the first coefficient for the target pixel;
A dither processing step of correcting a luminance value of the target pixel using the first coefficient and the second coefficient;
An image processing program characterized by realizing the above by a computer.

Images