JP4793175B2 - Image processing program and image processing apparatus - Google Patents

Description

  The present invention relates to an image processing program and an image processing apparatus, and more particularly to an image processing program and an image processing apparatus that perform Retinex processing and histogram processing.

  When a subject is imaged under backlight conditions, the image of the subject portion becomes an unclear backlight image with low brightness and contrast as the detailed mode becomes difficult to distinguish. In addition to such backlit images, poor images due to overexposure and underexposure, blurring and blurring during shooting, noise, insufficient light quantity, etc. are being used to improve image quality by improving brightness and contrast through image processing. Yes. As one method of such image processing, Retinex processing (Retinex processing) is known.

  In the Retinex process, the input image data is held for the high-quality part, and the image quality is mainly improved for the low-quality part. In this Retinex processing, each pixel data of the original image is corrected to a value reflecting the pixel data of the surrounding pixels by a Gaussian filter, and the reference component data of the original image is obtained from the natural logarithm of the corrected pixel data. The illuminance component data is calculated by dividing the pixel data of the original image by the pixel component of the original image. That is, the original image is divided into two constituent components, a reference component and an illuminance component. Then, after performing processing for correcting the brightness and gradation (contrast) such as gamma correction for the illuminance component, the corrected illuminance component and the reference component are combined, and the original image is backlit. Image data in which the image quality of a low image quality portion such as an image portion is improved can be generated. In order to obtain the reference component, the reflectance R (x, y) is normalized.

  Here, normalization of the reflectance R (x, y) will be described with reference to FIG. The reflectance R (x, y) is calculated by the following equation.

ここで、ｘは、横方向の座標を、ｙは、縦方向の座標を、Ｉ（ｘ，ｙ）は、座標（ｘ，ｙ）における強度値を、Ｆ（ｘ，ｙ）は、座標（ｘ，ｙ）のフィルタ係数を、＊は、畳み込み演算（コンボリューション）をそれぞれ示す。

Here, x is a coordinate in the horizontal direction, y is a coordinate in the vertical direction, I (x, y) is an intensity value at the coordinate (x, y), and F (x, y) is a coordinate ( x, y) and * indicate convolution operations (convolution), respectively.

  FIG. 7 shows the normalization of an integer value in the range of 0 to 255, where the minimum value of the reflectance R (x, y) obtained by the above equation is 0 and the maximum value is 255 for a certain still image. It is the histogram which totaled appearance frequency.

  The median value M is obtained from this histogram, the upper limit value of the range including 45% of pixels from the median value to the larger side is U, and the lower limit value of the range including 45% of pixels from the median value to the smaller side is D. .

  The values of R (x, y) corresponding to the values of U and D are UpR and DownR, and the normalized reflectance (normalized Retinex value) refle (x, y) is the value of R (x, y). Is 0.0 when the value is equal to or less than DownR, 1.0 when the value of R (x, y) is equal to or greater than UpR, and when the value of R (x, y) is greater than DownR and smaller than UpR,

とする。

And

  When processed in this way, refle (x, y) is normalized to take a value between 0.0 and 1.0. Experiments have shown that it is better to add 0.3 to the value thus obtained so that the value of refle (x, y) takes a value from 0.3 to 1.3. Yes.

  Since refle (x, y) is obtained in this way, R (x, y) must be stored for all pixels in order to obtain the clipped range (between UpR and DownR). Furthermore, R (x, y) is a numerical value obtained by logarithmic calculation, so it must be recorded using a decimal point. When storing in floating point, for example, when these processes are performed by a personal computer (hereinafter referred to as a PC), the data processing unit of a general PC CPU and operating system is 32 bits. In the case of a double precision real number in bytes, since it is stored in 8 bytes, a huge storage capacity for recording R (x, y) obtained by logarithmic calculation processing is required.

  Japanese Patent Laid-Open No. 2001-69525 (Patent Document 1) discloses a solution to the problem that color balance is lost or color misregistration occurs when this Retinex process is performed independently for each of the RGB planes. Discloses a method of converting RGB values into a coordinate space composed of luminance components and color components such as YCbCr and YIQ, performing Retinex processing only on the luminance component Y, and returning to RGB while maintaining the color components. Has been. When this method is used, only the luminance component is adjusted, and the color component is not adjusted, so that the color balance is not lost and color shift does not occur. Further, in this method, since the Retinex process is performed only on the luminance component, the calculation amount is small and the process can be executed at a high speed compared to the case where the Retinex process is performed on each of the RGB planes.

Japanese Patent No. 3731577 (Patent Document 2) discloses a method for increasing the processing speed of the Retinex processing. In this method, a reduced image (low resolution) is formed from the original image by a method such as an average pixel method, and a peripheral average luminance image (blurred image) obtained by calculating a peripheral average luminance for each pixel of the reduced image is formed. A retinex processed image is formed from the enlarged image of the blurred image and the original image, and an output image is formed from the retinex processed image and the original image.
JP 2001-69525 A Japanese Patent No. 3731577

  However, in the processing disclosed in the conventional literature, the size of the memory for storing the reflectance is the same as that of the original image (for example, literature 1: (0090), “original FULL (2,560 × 1,920) The resolution is returned to the resolution and the processing is executed by the Retinex processing unit 407)), so that the size of the memory for storing the reflectance increases as the original image becomes larger, and when the processed image is output, As the original image becomes larger, the data amount of R (x, y) also correspondingly increases, so the logarithmic calculation amount of R (x, y) also increases, and the time until image output starts increases. There was a problem.

  The present invention has been made to solve the above-described problems, and provides an image processing program and an image processing apparatus capable of performing Retinex processing and histogram processing at high speed using a memory having a small storage capacity. The purpose is that.

In order to achieve this object, the image processing program according to claim 1 is executed by an image processing apparatus that performs image correction processing. For each pixel of a reduced image obtained by reducing the original image, a pixel value is obtained. And a pixel value of the reduced Retinex image formed by the reduced Retinex image forming step, which forms a reduced Retinex image by calculating the reflectance from the peripheral average luminance of the pixel and the pixel. A normalization parameter setting step for setting an upper limit value and a lower limit value for performing normalization by summing up the frequency of each pixel value in the entire range of values, and a reduced Retinex image formed by the reduced Retinex image forming step Is normalized based on the upper limit value and the lower limit value set by the normalization parameter setting step. A reduced image correcting step of correcting the pixel value of the normalized value is larger as the reduced image of the pixel values becomes large so that the reduced image after correction, the corrected reduced image by the reduced image correction step The histogram processing parameter acquisition step for acquiring the parameters for performing histogram processing by summing up the frequency of each pixel value in the entire range taken by the pixel value, and for each pixel of the original image, the pixel value and the peripheral average luminance of the pixel The reflectance is obtained from the above, and the normalized Retinex value obtained by normalizing the reflectance based on the upper limit value and the lower limit value set by the normalization parameter setting step is obtained, and the corrected value increases as the normalized Retinex value increases. wherein as the pixel values of the original image increases corrects the pixel values of the original image, the histogram processing Based on the obtained histogram processing parameter by parameter obtaining step, and an original image correction step of the histogram correcting corrected pixel values, to be executed by the image processing apparatus.

The image processing program according to claim 2 is the image processing program according to claim 1, wherein the reduced Retinex image forming step forms a reduced Retinex image only for the luminance signal of the reduced image, and the reduced image correction step includes The reduced Retinex image formed by the reduced Retinex image forming step is normalized based on the upper limit value and the lower limit value set by the normalization parameter setting step , and the corrected value increases as the normalized value increases. After the luminance signal of the reduced image is corrected so that the luminance signal of the reduced image is increased, the reduced image is returned to the color image. In the histogram processing parameter acquisition step, the pixel value of the color image corrected by the reduced image correction step is Aggregate the frequency of each pixel value in the entire range The parameter for processing is acquired, and the original image correction step obtains a Retinex value for only the luminance signal of the original image, and the Retinex value is based on the upper limit value and the lower limit value set by the normalization parameter setting step. Normalization, correcting the luminance signal of the original image so that the luminance signal of the original image after correction increases as the normalized Retinex value increases , and returns the color pixel value to the color pixel value. Histogram correction is performed based on the parameters acquired in the histogram processing parameter acquisition step.

  The image processing program according to claim 3 is the image processing program according to claim 1 or 2, wherein the normalization parameter setting step takes all pixels of the reduced Retinex image formed by the reduced Retinex image forming step. Among the values in the range from the maximum value to the minimum value, the upper limit value and the lower limit value of the clip range including a predetermined ratio of pixels with respect to all pixels are set as normalization parameters.

The image processing apparatus according to claim 4 performs image correction processing, and for each pixel of a reduced image obtained by reducing the original image, the reflectance is obtained from the pixel value and the peripheral average luminance of the pixel. The reduced Retinex image forming means for forming the reduced Retinex image, and the frequency of each pixel value in the entire range of values taken by the pixel values of the reduced Retinex image formed by the reduced Retinex image forming means Normalization parameter setting means for setting an upper limit value and a lower limit value when performing normalization, and an upper limit value and a lower limit set by the normalization parameter setting means for the reduced Retinex image formed by the reduced Retinex image forming means normalized based on the value, so that the pixel values of the reduced image after correction as the normalized value is increased becomes larger A reduced image correcting means for correcting the pixel value of the serial reduced image, counts the frequency of each pixel value in the full range of pixel values take the corrected reduced image by the reduced image correcting unit, a parameter for performing histogram processing For each pixel of the original image to be acquired, the histogram processing parameter acquisition means to obtain the reflectance from the pixel value and the peripheral average luminance of the pixel, and the reflectance is the upper limit value set by the normalization parameter setting means Then, a normalized Retinex value normalized based on the lower limit value is obtained, and the pixel value of the original image is corrected so that the pixel value of the original image after correction increases as the normalized Retinex value increases. , Based on the histogram processing parameter acquired by the histogram processing parameter acquisition means, And an original image correcting means for ram corrected.

The image processing device according to claim 5 is the image processing device according to claim 4, wherein the reduced Retinex image forming unit forms a reduced Retinex image only for the luminance signal of the reduced image, and the reduced image correcting unit includes The reduced Retinex image formed by the reduced Retinex image forming unit is normalized based on the upper limit value and the lower limit value set by the normalization parameter setting unit, and the corrected value increases as the normalized value increases. The luminance signal of the reduced image is corrected so that the luminance signal of the reduced image is increased and then returned to the color image, and the histogram processing parameter acquisition unit is configured to take all pixel values of the color image corrected by the reduced image correction unit. The frequency of each pixel value in the range is totaled, a parameter for performing histogram processing is acquired, and the original image Correcting means, only for the luminance signal of the original image determined Retinex value, the Retinex value normalized based on the set upper limit value and the lower limit value by the normalization parameter setting means, the normalized Retinex value is greater The correction of the luminance signal of the original image is performed so that the luminance signal of the original image after correction becomes larger and the color pixel value is restored, and the color pixel value is based on the parameter acquired by the histogram processing parameter acquisition unit. To correct the histogram.

  The image processing apparatus according to claim 6 is the image processing apparatus according to claim 4 or 5, wherein the normalization parameter setting means takes all pixels of the reduced Retinex image formed by the reduced Retinex image forming means. Among the values in the range from the maximum value to the minimum value, the upper limit value and the lower limit value of the clip range including a predetermined ratio of pixels with respect to all pixels are set as normalization parameters.

  According to the image processing program of the first aspect, the reduced Retinex image forming step obtains the reflectance from the pixel value and the peripheral average luminance of the pixel for each pixel of the reduced image obtained by reducing the original image. The reduced Retinex image is formed, and the normalization parameter setting step totals the frequency of each pixel value in the entire range of values taken by the pixel values of the reduced Retinex image, and calculates the upper limit value and the lower limit value for normalization. Set.

  The histogram processing parameter acquisition step normalizes the reduced Retinex image formed by the reduced Retinex image forming step based on the upper limit value and the lower limit value set by the normalization parameter setting step, and the normalized value and the reduced image And the frequency of each pixel value in the entire range taken by the pixel value of the corrected reduced image is totaled to obtain a parameter for performing histogram processing.

  Therefore, it is possible to reduce the capacity of the memory for storing the reduced image and the intermediate calculation result, and obtain the parameters for normalization and the parameters for histogram processing based on the reduced image obtained by reducing the original image. Can be acquired at high speed.

  In the original image correction step, for each pixel of the original image, the reflectance is obtained from the pixel value and the peripheral average luminance of the pixel, and the reflectance is based on the upper limit value and the lower limit value set in the normalization parameter setting step. Normalized normalized Retinex value, corrected based on the normalized Retinex value and the pixel value of the original image, and corrected based on the histogram processing parameter acquired by the histogram processing parameter acquisition step Histogram correction is performed on the pixel values. In this original image correction step, since each pixel of the original image can be sequentially processed, the processed pixel values can be sequentially output. Therefore, there is an effect that it is possible to shorten the time from the start of Retinex processing until the start of output.

  According to the image processing program of the second aspect, in addition to the effect of the image processing program of the first aspect, the reduced Retinex image forming step forms a reduced Retinex image only for the luminance signal of the reduced image, and The image correction step normalizes the reduced Retinex image formed by the reduced Retinex image forming step based on the upper limit value and the lower limit value set by the normalization parameter setting step, and the normalized value and the reduced image The reduced image is corrected based on the luminance signal and then returned to the color image, and the histogram processing parameter acquisition step totals the frequency of each pixel value in the entire range taken by the pixel value of the color image corrected by the reduced image correction step. The parameter for performing histogram processing is acquired, and the original image correction step is The Retinex value is obtained only for the luminance signal of the image, the Retinex value is normalized based on the upper limit value and the lower limit value set by the normalization parameter setting step, the normalized Retinex value, the luminance signal of the original image, and The color pixel value is corrected and returned to the color pixel value, and the color pixel value is histogram-corrected based on the parameter acquired in the histogram processing parameter acquisition step. Therefore, Retinex processing is performed for each of the RGB values constituting the color image. Compared to the case, there are effects that the processing is fast and color misregistration does not occur.

  According to the image processing program of the third aspect, in addition to the effect produced by the image processing program of the first or second aspect, the normalization parameter setting step includes the step of reducing the Retinex image formed by the reduction Retinex image forming step. Since the upper and lower limit values of the clip range including a predetermined percentage of all the pixels in the range from the maximum value to the minimum value of all pixels are set as normalization parameters, the upper limit determined As the value and the lower limit, it is possible to obtain substantially the same value as the upper limit and the lower limit determined using the original image. Therefore, there is an effect that the upper limit value and the lower limit value can be obtained at a high speed while maintaining the accuracy. When the maximum value and the minimum value are used as normalization parameters instead of the upper limit value and the lower limit value, the value obtained from the original image and the value obtained from the reduced image may be greatly different.

  According to the image processing apparatus of the fourth aspect, the reduced Retinex image forming unit obtains the reflectance for each pixel of the reduced image obtained by reducing the original image from the pixel value and the peripheral average luminance of the pixel. A reduced Retinex image is formed, and the normalization parameter setting means totals the frequency of each pixel value in the entire range of values taken by the pixel values of the reduced Retinex image, and calculates an upper limit value and a lower limit value for normalization. Set.

  The histogram processing parameter acquisition unit normalizes the reduced Retinex image formed by the reduced Retinex image forming unit based on the upper limit value and the lower limit value set by the normalization parameter setting unit, and the normalized value and the reduced image And the frequency of each pixel value in the entire range taken by the pixel value of the corrected reduced image is totaled to obtain a parameter for performing histogram processing.

  Therefore, it is possible to reduce the capacity of the memory for storing the reduced image and the intermediate calculation result, and obtain the parameter for normalization and the parameter for histogram processing based on the reduced image obtained by reducing the original image. So you can get at high speed.

  The original image correction means obtains a reflectance for each pixel of the original image from the pixel value and the peripheral average luminance of the pixel, and the reflectance is based on the upper limit value and the lower limit value set by the normalization parameter setting means. Normalized normalized Retinex value, corrected based on the normalized Retinex value and the pixel value of the original image, and corrected based on the histogram processing parameter acquired by the histogram processing parameter acquisition means Histogram correction is performed on the pixel values. In this original image correction means, since each pixel of the original image can be sequentially processed, the value of the processed pixel can be sequentially output. Therefore, there is an effect that it is possible to shorten the time from the start of Retinex processing until the start of output.

  According to the image processing apparatus of the fifth aspect, in addition to the effect produced by the image processing apparatus of the fourth aspect, the reduced Retinex image forming unit forms a reduced Retinex image only for the luminance signal of the reduced image, and The image correction means normalizes the reduced Retinex image formed by the reduced Retinex image forming means based on the upper limit value and the lower limit value set by the normalization parameter setting means, and the normalized value and the reduced image After correcting the reduced image based on the luminance signal, the reduced image is returned to the color image, and the histogram processing parameter acquisition means sums up the frequency of each pixel value in the entire range taken by the pixel values of the color image corrected by the reduced image correction means. The parameter for performing the histogram processing is acquired, and the original image correction means performs the retinex only for the luminance signal of the original image. Obtain the value, normalize the Retinex value based on the upper limit value and the lower limit value set by the normalization parameter setting means, perform correction based on the normalized Retinex value and the luminance signal of the original image, and color Returning to the pixel value, the color pixel value is subjected to histogram correction based on the parameter acquired by the histogram processing parameter acquisition means, so the processing is faster than when Retinex processing is performed for each of the RGB values constituting the color image. In addition, there is an effect that no color misregistration occurs.

  According to the image processing apparatus of the sixth aspect, in addition to the effect produced by the image processing apparatus of the fourth or fifth aspect, the normalization parameter setting unit is configured to reduce the reduced Retinex image formed by the reduced Retinex image forming unit. Since the upper and lower limit values of the clip range including a predetermined percentage of all the pixels in the range from the maximum value to the minimum value of all pixels are set as normalization parameters, the upper limit determined As the value and the lower limit, it is possible to obtain substantially the same value as the upper limit and the lower limit determined using the original image. Therefore, there is an effect that the upper limit value and the lower limit value can be obtained at a high speed while maintaining the accuracy. When the maximum value and the minimum value are used as normalization parameters instead of the upper limit value and the lower limit value, the value obtained from the original image and the value obtained from the reduced image may be greatly different.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a block diagram illustrating an electrical configuration of a printer 1 having a function of performing image processing according to the present embodiment. In the present embodiment, the image processing program installed in the printer 1 is image data (original image data or the like) input from the personal computer (hereinafter referred to as “PC”) 2, the digital camera 21, or the external medium 20. On the other hand, a Retinex process (Retinex process) and a histogram process are executed to correct a low image quality region such as a backlight image portion of image data.

  As shown in FIG. 1, the printer 1 includes a CPU 11, a ROM 12, a RAM 13, a print head, and the like, and includes a printing unit 15 that performs printing (output) on a print medium (for example, a paper medium), an output image size, and the like. And an operation panel 16 having a user operation unit (for example, a numeric keypad) through which the user can input the input value.

  The printer 1 includes an interface (hereinafter referred to as “I / F”) 17 that can be connected to the PC 2 via the cable 5, an I / F 18 that can be connected to the digital camera 21 via the cable 6, and an external medium 20. And an external media slot 19 that can be detachably mounted (for example, a flash memory such as an SD memory card or a memory stick). As a communication method performed by these I / Fs 17 and 18, USB (Universal Serial Bus) is used.

  Therefore, the printer 1 can input the image data stored in the PC 2 via the cable 5 and the I / F 17 and also the image data photographed by the digital camera 21 via the cable 6 and the I / F 18. It is possible to input via. Further, it is possible to input image data stored in the external medium 20 from the external medium 20 mounted in the external media slot 19.

  The CPU 11 is an arithmetic processing device that controls the entire printer 1. The ROM 12 stores various control programs executed by the CPU 11 and fixed values used when the programs are executed, and stores an image processing program for performing image processing such as Retinex processing and histogram processing. An image processing program memory 12a, a print control program memory 12b for storing a print control program for performing printing, and the like are provided.

  The RAM 13 has a working area for storing various register groups required when the control program is executed by the CPU 11, a temporary area for temporarily storing data being processed, and the like, and can be accessed at random. An original image memory 13a for storing original image data; a reduced Retinex image memory 13b for storing the reflectance required in the Retinex processing of the reduced image after reducing the original image; and each pixel in the histogram processing A histogram memory 13c for storing the frequency of values is provided.

  The original image data memory 13 a stores image data input from the PC 2, the digital camera 21, and the external media 20 via the I / F 17, I / F 18, and external media slot 19, respectively. In the present embodiment, both the original image data and the output image data are composed of RGB values, and these RGB values are values in the range of “0” to “255”.

  The RGB value is a value having as constituent components an R value representing red, which is the three primary colors of light, a G value representing green, and a B value representing blue. Since various colors are generated by mixing the three primary colors of light, the color of each pixel of the input image is one color (hue, gradation, etc.) by a combination of R value, G value, and B value (RGB value). ) Is displayed. The luminance (lightness) increases as the RGB value increases.

  The reduced Retinex image memory 13b is a memory that stores the reflectance Rs (x, y) obtained in the Retinex process for only the luminance signal of the reduced image by reducing the original image. As a method for reducing the original image, the value of the pixel closest to the position of the original image corresponding to the pixel of the image to be reduced is sampled, and the value of the sampled pixel is used as it is. There are known a bicubic method for performing an interpolation operation using pixels around a position of an original image corresponding to a pixel of an image to be performed, an average pixel method, and the like. When the processing speed is prioritized over the image quality, the nearest neighbor method is used, and when the image quality is prioritized over the processing speed, the bicubic method or the average pixel method is used.

  The original image is reduced by any one of these methods, and the reflectance is calculated for each pixel (pixel) of the reduced image using Equation 1 described above, and stored in the reduced Retinex image memory 13b.

  The histogram memory 13c is a memory for summing up the frequency (number of pixels) in which the RGB value of the image takes each integer value from 0 to 255 in the histogram processing after completing the Retinex processing, and summing up the frequencies. is there. When the histogram is created, an upper limit value U, a lower limit value D, a median value, and the like for performing histogram processing are determined based on the histogram.

  Next, image processing executed by the CPU 11 will be described with reference to FIGS. 2 and 3. 2 and 3 are flowcharts showing the image processing. The process shown in FIG. 2 is a so-called pre-process, and in this process, a clip range for normalizing the reflectance R is set. Note that no image is output while this pre-processing is being performed.

  In this preprocessing, first, original image data is read from the PC 2 or the digital camera 21 and stored in the original image memory 13a of the RAM 13 (S1). Next, the original image is reduced to form a reduced image, and each pixel of the reduced image is converted into a luminance signal Y and color signals Cb and Cr (S2).

  As a method for forming a reduced image, a nearest neighbor method or an average pixel method is used, and the luminance signal Y and the color signals Cb and Cr are calculated by the following equations.

演算された輝度信号および色信号をＲＡＭ１３などに一時記憶し、後の処理で使用するようにしてもよいが、記憶容量を削減したい場合には、これらを記憶せずに、必要な場合、元画像から画素値を読み出して上式により演算で求めてもよい。

The calculated luminance signal and color signal may be temporarily stored in the RAM 13 or the like and used in later processing. However, if it is desired to reduce the storage capacity, these are not stored, and if necessary, the original The pixel value may be read from the image and calculated by the above formula.

  Next, the reflectance Rs (x, y) is calculated for the luminance signal Y calculated by the above equation. Note that “s” is a suffix for the reduced image, and no suffix is added to the original image. The reflectance Rs is calculated by the following equation, assuming that the luminance value of each pixel of the reduced image is Is (x, y) and the filter for the reduced image is Fs (x, y).

なお、「＊」は、畳み込み演算（コンボリューション）を示し、ｌｏｇは、ｅを底とする自然対数である。

Note that “*” indicates a convolution operation (convolution), and log is a natural logarithm with e as the base.

  Next, the reflectance Rs (x, y) obtained by the above calculation is stored in the reduced Retinex image memory 13b of the RAM 13 (S4).

  Next, Rs (x, y) is compared with the maximum value and the minimum value (S5). Specifically, the reflectance Rs (x, y) obtained for the first coordinate is set as the maximum value and the minimum value, and the reflectance Rs (x, y) obtained from the next is compared with the maximum value and the minimum value, respectively. When the reflectance Rs (x, y) obtained this time is larger than the maximum value, the reflectance Rs (x, y) obtained this time is set as a new maximum value, and the reflectance Rs (x, y) obtained this time is obtained. Is smaller than the minimum value, the reflectance Rs (x, y) obtained this time is set as a new minimum value, and the reflectance Rs (x, y) obtained this time is smaller than the maximum value and larger than the minimum value. Is a process of not changing the maximum value and the minimum value.

  Next, it is determined whether or not the processing of S3 to S5 has been performed for all the coordinates of the reduced image (S6). If there are still unprocessed coordinates (S6: No), the processing returns to S3 and all of them are performed. When the process for the coordinates of (5) is completed (S6: Yes), the reflectance Rs (x, y) is normalized based on the maximum value and the minimum value obtained by the process of S5, and a histogram is formed (S7). .

  Next, a median value is obtained from the formed histogram (S8), and a clipped range (clip range) of the reflectance Rs (x, y) is determined from the median value and the histogram (S9). This clip range is, for example, a value larger than the median value, and an upper limit value that includes 45% of all samples is the upper limit value of the clip range, and a lower value that is less than the median value and includes 45% of all samples. The value is the lower limit value of the clip range.

  Through the above pre-processing, a reduced Retinex image composed of the reduced image reflectance obtained by reducing the original image is formed, and the clip range for normalizing the reflectance of the original image is obtained from the reduced Retinex image. It is done. As a result, the number of calculations for obtaining the clip range can be made much smaller than the number of calculations for the original image, and the processing speed is increased. Further, since the Retinex image of the reduced image is stored, there is an advantage that the processing can be performed with a small storage capacity compared with the case of storing the Retinex image of the original image. Further, there is no large difference between the clip range obtained using the reduced image and the clip range obtained using the original image. The following table shows the difference between the clip range (upper limit and lower limit) obtained from the reduced image and the clip range obtained from the original image.

この表が示すように、元画像により求めた最大値は、２．７１２、最小値は、−４．０６３であり、縮小画像により求めた最大値は、１．７２９、最小値は、−２．６０７である。よって、元画像により求めた最大値と縮小画像により求めた最大値との差は、０．９８３であり、元画像により求めた最小値と縮小画像により求めた最小値との差は、１．４５６であって、かなり大きな差が認められる。

As shown in this table, the maximum value obtained from the original image is 2.712, the minimum value is −4.063, the maximum value obtained from the reduced image is 1.729, and the minimum value is −2 .607. Therefore, the difference between the maximum value obtained from the original image and the maximum value obtained from the reduced image is 0.983, and the difference between the minimum value obtained from the original image and the minimum value obtained from the reduced image is 1. 456, and a considerable difference is recognized.

  On the other hand, the upper limit value of the clip range obtained from the original image is 0.825, the lower limit value is −0.822, the upper limit value obtained from the reduced image is 0.742, and the lower limit value is −0.755. It is. Therefore, the difference between the upper limit value obtained from the original image and the upper limit value obtained from the reduced image is 0.083, and the difference between the lower limit value obtained from the original image and the lower limit value obtained from the reduced image is 0. 0. 067, and it can be seen that these differences are small.

  Next, the Retinex process of the original image, which is a post process performed based on the clip range obtained by the process shown in FIG. 2, will be described. FIG. 3 is a flowchart showing the original image Retinex processing. In this post-processing, each pixel of the original image is processed, and the processed pixel value is sequentially output to the printing unit 15.

  First, the reflectance R (x, y) is calculated for the original image (S11). Next, based on the clip range obtained using the reduced image, R (x, y) is normalized to obtain a normalized reflectance (normalized Retinex value) refle (x, y) (S12).

  Next, using this refle (x, y), a pixel value Out (x, y) subjected to Retinex processing for luminance is obtained by the following equation (S13).

次に、このＯｕｔ（ｘ，ｙ）と、色信号Ｃｂ、Ｃｒとに基づいて、ＲＧＢ値に変換する（Ｓ１４）。この変換は、次式により算出される。

Next, based on this Out (x, y) and the color signals Cb and Cr, conversion into RGB values is performed (S14). This conversion is calculated by the following equation.

以上の処理により元画像の輝度信号にレティネックス処理が施され、色信号に基づいてＲＧＢ値に戻した画素値が求められる。次に、この処理された画素値を印刷部１５に出力する（Ｓ１５）。よって、前処理により、正規化を行うクリップ範囲が特定されているので、後処理では、順次各画素について処理を行い印刷部１５に出力することができる。その結果、処理を開始してから、印刷が開始されるまでの時間を短縮することができる。

Through the above processing, the Retinex processing is performed on the luminance signal of the original image, and the pixel value returned to the RGB value is obtained based on the color signal. Next, the processed pixel value is output to the printing unit 15 (S15). Therefore, since the clip range to be normalized is specified by the preprocessing, each pixel can be sequentially processed and output to the printing unit 15 in the postprocessing. As a result, the time from the start of processing to the start of printing can be shortened.

  Next, it is determined whether or not the process has been completed for all the pixels of the original image (S16). If there is a pixel that has not been processed yet (S16: No), the process returns to S11 and all the pixels are processed. When the process is finished for (S16: Yes), the process is finished.

  As described above, as described in the first embodiment, the Retinex process is performed on the luminance signal of the reduced image obtained by reducing the original image, the clip range is obtained from the reduced Retinex image, and based on the clip range. Since the Retinex processing is performed on the original image, the storage capacity of the memory for storing the Retinex image can be reduced, and the processing for obtaining the clip range can be performed at high speed. In addition, the clip range is obtained in the pre-processing, and in the post-processing, the Retinex process can be performed on the pixels of the forward dimension image based on the clip range obtained in the pre-processing, and the processing can be performed. The time from the start to the start of output can be shortened.

  Next, a second embodiment will be described. When Retinex processing is performed, the dynamic range may become narrow, or the dark part may become too bright, which may cause unnaturalness. Therefore, image quality can be improved by performing histogram processing after performing Retinex processing. In the second embodiment, histogram processing is performed after performing Retinex processing.

  The histogram processing will be described with reference to FIG. FIG. 4A is an example of an image, and FIG. 4B is a histogram formed from the image. This histogram is a total of frequencies (number of pixels) for each value in the entire range of values taken by pixel values. In the case of a color image, each value of R, G, B is 8 bits, that is, when a value from 0 to 255 is taken, for each integer value from 0 to 255, R, G of the pixel that is the value , B values are added up by adding 1. For example, when the R, G, and B values of a certain pixel are (100, 200, 200), the values are aggregated by adding 1 to the value of 100 and 2 to the value of 200.

  In the histogram created in this way, a ratio (cut ratio) excluding the bright part and the dark part is set, and the upper limit value U and the lower limit value D are obtained from the ratio. For example, when the ratio excluding the bright part and the dark part is 3%, the number of pixels included in the bright part is three times the total number of pixels (three values of R, G, and B per pixel). ), The upper limit value U is set so as to include the number corresponding to 3% from the maximum value 255 to the upper limit value U. Similarly, the number corresponding to 3% of the number of pixels included in the dark portion is 3 times the total number of pixels. The lower limit value U is set so as to include from the minimum value 0 to the lower limit value D.

  In this way, when the upper limit value U and the lower limit value D are set, the value in the range from the minimum pixel value to the lower limit value D is set as the minimum value, and the value in the range from the upper limit value U to the maximum value is set as the maximum value. The value in the range from the lower limit value D to the upper limit value U is corrected with a linear or non-linear function.

That is, when the input pixel value is input, the corrected pixel value is output, the range that the pixel value takes is 0 to 255, and γ2 is a constant,
0 if input is less than or equal to lower limit D
255 if input is greater than or equal to upper limit U
When input is larger than the lower limit value D and smaller than the upper limit value U,
a = (input-D) / (UD)

この式でγ２＝１とすると、ｏｕｔｐｕｔは、下限値と上限値との間の値を線形に拡大することになる。

When γ2 = 1 in this equation, output is a linear expansion of the value between the lower limit value and the upper limit value.

  You may set according to the value of (gamma) 1 of Numerical formula 5 in Retinex processing. If the value of γ1 is increased, the correction is performed brighter. If the value of γ is small, the correction is not performed much. Therefore, when γ1 is large, the lower limit value U in the histogram processing is increased (the cut amount or the cut ratio of the dark portion is increased), and the pixel value is changed to the darker side as a whole, thereby tightening the image.

  In addition, the portion of backlight or the like is corrected brightly by Retinex processing, but is corrected brightly as a whole, resulting in an impression different from the original image. Therefore, by applying a gamma correction opposite to the gamma correction in the Retinex process by the histogram process, it is possible to return the brightened part to a natural state.

Further, in accordance with gamma γ1 used for gamma correction in Retinex processing, gamma γ2 in histogram processing may be obtained by the following equation. That is,
γ2 = (γ1-1.0) /A+1.0
(Where A is a value from 5 to 20).

  Γ2 is set so that the median value after Retinex processing approaches the median value before Retinex processing.

The original pixel value is input, the corrected pixel value is output, the whole range that the pixel value takes is B, the maximum value of the whole range that the pixel value takes is MAX, the minimum value is MIN, the upper limit value of the clip range is U, When the lower limit is D, the median value of the original image is OM, the median value of the image processed by the Retinex processing step is RM, and the constant is α,
If input is in the lower end range (D or less), it is set to MIN.
When input is the upper end range (U or more), it is set to MAX,
If input is outside the above range,
a = (input-D) / (UD)
Let output be the value of a raised to the power of γ2 by B,
γ2 = log (target / MAX) / log (RM / MAX)
However, target = (OM × α + RM) / (1 + α)
Here, when the constant α is 1, the target is set in the middle between the median value before Retinex processing and the median value after Retinex processing. If the value of the constant α is larger than 1, the target becomes the original image. When the value approaches the median value and the constant α is set to a value smaller than 1, the median value is set so as to be separated from the median value of the original image. The constant α may be set to a predetermined value in advance or may be arbitrarily set by the user.

  FIG. 5 is a flowchart showing pre-processing in the second embodiment. Among the pre-processing, the processing of S1 to S9 is the same as the processing shown in FIG.

Subsequent to the process of S9, Rs (x, y) stored in the reduced retinex memory 13b is normalized with the clip range obtained in the process of S9, and refles (x, y) is obtained by Expression 4 (S21).
Next, Outs (x, y) is obtained by the following equation (S22).

次に、このＯｕｔｓ（ｘ，ｙ）を輝度信号として、数式６によりＲＧＢ値に変換し、ヒストグラムに登録する（Ｓ２３）。このヒストグラムの登録とは、ＲＧＢ値それぞれが取る値は、０から２５５までの整数値であり、ここで得られた、Ｒ値、Ｇ値、Ｂ値の各値に対応する値の頻度に１を加算することである。加算された値は、ヒストグラムメモリ１３ｃに記憶される。

Next, this Outs (x, y) is converted into an RGB value by Equation 6 using the luminance signal as a luminance signal and registered in the histogram (S23). In this histogram registration, each RGB value takes an integer value from 0 to 255, and the frequency of the value corresponding to each of the R value, G value, and B value obtained here is one. Is to add. The added value is stored in the histogram memory 13c.

  It is determined whether or not the processing from S21 to S23 has been performed for all the coordinates of the reduced image (S24). If there are coordinates that have not yet been processed (S24: No), the processing of S21 is performed. Returning to FIG. 5, if all the coordinates have been processed (S24: Yes), parameters in the histogram processing are acquired from the histogram formed in the histogram memory 13c (S25). As described above, the parameters in this histogram processing are the lower limit value D, the upper limit value U, or the gamma value γ2.

  Using the reduced image obtained by reducing the original image by the above processing, a normalization parameter in the Retinex process and a parameter in the histogram process that is executed after the Retinex process are acquired.

  Next, post-processing after these parameters are set in advance will be described with reference to FIG. FIG. 6 is a flowchart showing post-processing.

  In the post-processing, first, a luminance signal is obtained for a pixel of the original image, a reflectance R (x, y) is calculated for the luminance signal by Equation 1 (S31), and R (x, y) is calculated based on the clip range. Normalized refle (x, y) is obtained (S32).

  Next, Out (x, y) is calculated by Equation 5 (S33), and the calculated Out (x, y) is returned to the RGB value by Equation 6 as a luminance signal (S34), and the RGB value is obtained in S25. Histogram correction is performed based on the parameters (S35) and output to the printing unit 15 (S36).

  Next, it is determined whether or not the processing for all the pixels of the original image has been completed (S37). If there is a pixel that has not yet been processed (S37: No), the processing returns to S31 and all pixels are processed. When the above process is completed (S37: Yes), the subsequent process is terminated.

  As described above, according to the second embodiment, Retinex processing is performed on the luminance of a color image, and after returning to a color image, histogram processing is performed. As a result, the dynamic range of the image can be expanded and a higher quality image can be obtained. In the case of performing the histogram processing, the image quality can be further improved by setting the parameters in the histogram processing according to the parameters in the Retinex processing.

  Note that the reduced Retinex image forming step and the reduced Retinex image forming means described in the claims correspond to the processing of S3 in the flowchart shown in FIG. 5, and the normalization parameter setting step and the normalization parameter setting means are the same as those in FIG. 5 corresponds to the processing of S7 to S9 in the flowchart shown in FIG. 5, the reduced image correction step and the reduced image correction means correspond to the processing of S21 to S23 in the flowchart shown in FIG. 5, and the original image correction step and the original image correction means include The process of the flowchart shown in FIG.

  As described above, the present invention has been described based on the embodiments. However, the present invention is not limited to the above embodiments, and various improvements and modifications can be made without departing from the spirit of the present invention. It can be easily guessed.

  For example, in the above embodiment, the color expression system is the RGB format, but the present invention may be applied to other color expression systems other than the RGB format, for example, the CMY format.

  In the above embodiment, the image processing program of the present invention is executed by the CPU 11 incorporated in the printer 1, but is supplied as an application to the personal computer and executed by the CPU incorporated in the personal computer. You may make it do.

  The Retinex process may be SSR (single scale method) or MSR (multiscale method).

  In the image processing of the above embodiment, the Retinex processing or the like is performed by the CPU 11, but may be performed by a DSP (Digital Signal Processor). If a DSP is used, processing such as a product-sum operation can be executed at a higher speed.

1 is a block diagram showing an electrical configuration of a printer equipped with an image processing program according to an embodiment of the present invention. It is a flowchart which shows the pre-processing performed by an image processing program. It is a flowchart which shows the post-process performed following a pre-process. It is a figure for demonstrating a histogram process, (a) is an image figure which shows an example of an image, (b) is a histogram of the pixel value of the image. It is a flowchart which shows the pre-processing in 2nd Embodiment. It is a flowchart which shows the post-process in 2nd Embodiment. It is a histogram for demonstrating the clip range for performing normalization.

Explanation of symbols

1 Printer 2 Personal Computer 11 CPU
12 ROM
12a Image processing program memory 13 RAM
13a Original image memory 13b Retinex image memory

Claims (6)

In an image processing program executed by an image processing apparatus that performs image correction processing,
For each pixel of the reduced image obtained by reducing the original image, a reduced Retinex image forming step for forming a reduced Retinex image by obtaining a reflectance from the pixel value and the peripheral average luminance of the pixel;
Normalization that counts the frequency of each pixel value in the entire range of values taken by the pixel values of the reduced Retinex image formed by the reduced Retinex image forming step and sets the upper and lower limits for normalization A parameter setting step;
The reduced Retinex image formed by the reduced Retinex image forming step is normalized based on the upper limit value and the lower limit value set by the normalization parameter setting step, and the reduced image after correction becomes larger as the normalized value becomes larger A reduced image correction step for correcting the pixel value of the reduced image so that the pixel value of the image is increased ;
A histogram processing parameter acquisition step for totalizing the frequency of each pixel value in the entire range taken by the pixel values of the reduced image corrected by the reduced image correction step, and acquiring a parameter for performing the histogram processing;
For each pixel of the original image, a reflectance is obtained from the pixel value and the peripheral average luminance of the pixel, and the reflectance is normalized based on the upper limit value and the lower limit value set by the normalization parameter setting step. Obtaining a normalized Retinex value, correcting the pixel value of the original image so that the pixel value of the original image after correction increases as the normalized Retinex value increases , and is acquired by the histogram processing parameter acquisition step An image processing program for causing the image processing apparatus to execute an original image correction step of performing histogram correction on a corrected pixel value based on the histogram processing parameter that has been corrected. The reduced Retinex image forming step forms a reduced Retinex image for only the luminance signal of the reduced image,
The reduced image correction step normalizes the reduced Retinex image formed by the reduced Retinex image forming step based on the upper limit value and the lower limit value set by the normalization parameter setting step, and the normalized value is The luminance signal of the reduced image is corrected so that the luminance signal of the reduced image after correction becomes larger as it becomes larger , and then returned to the color image.
The histogram processing parameter acquisition step totals the frequency of each pixel value in the entire range taken by the pixel values of the color image corrected by the reduced image correction step, and acquires a parameter for performing histogram processing,
In the original image correction step, a Retinex value is obtained only for the luminance signal of the original image, the Retinex value is normalized based on the upper limit value and the lower limit value set in the normalization parameter setting step, and the normalized latency The luminance signal of the original image is corrected and returned to the color pixel value so that the luminance signal of the original image after correction increases as the Nex value increases , and the color pixel value is acquired by the histogram processing parameter acquisition step. The image processing program according to claim 1, wherein histogram correction is performed based on the parameters.   In the normalization parameter setting step, pixels in a predetermined ratio with respect to all the pixels out of values in a range from a maximum value to a minimum value of all the pixels of the reduced Retinex image formed by the reduced Retinex image forming step. The image processing program according to claim 1, wherein an upper limit value and a lower limit value of a clip range including a normalization parameter are set. In an image processing apparatus that performs image correction processing,
For each pixel of the reduced image obtained by reducing the original image, a reduced Retinex image forming unit that forms a reduced Retinex image by obtaining a reflectance from the pixel value and the peripheral average luminance of the pixel;
Normalization that aggregates the frequency of each pixel value in the entire range of values taken by the pixel values of the reduced Retinex image formed by the reduced Retinex image forming means and sets the upper and lower limits for normalization Parameter setting means;
The reduced Retinex image formed by the reduced Retinex image forming unit is normalized based on the upper limit value and the lower limit value set by the normalization parameter setting unit, and the reduced image after correction is increased as the normalized value increases. Reduced image correction means for correcting the pixel value of the reduced image so as to increase the pixel value of the image;
Histogram processing parameter acquisition means for totalizing the frequency of each pixel value in the entire range taken by the pixel values of the reduced image corrected by the reduced image correction means, and acquiring parameters for performing histogram processing;
For each pixel of the original image, the reflectance is obtained from the pixel value and the peripheral average luminance of the pixel, and the reflectance is normalized based on the upper limit value and the lower limit value set by the normalization parameter setting means. A normalized Retinex value is obtained, the pixel value of the original image is corrected so that the pixel value of the original image after correction increases as the normalized Retinex value increases , and is acquired by the histogram processing parameter acquisition means An image processing apparatus comprising: an original image correcting unit that performs histogram correction on a corrected pixel value based on the histogram processing parameter that has been corrected. The reduced Retinex image forming means forms a reduced Retinex image for only the luminance signal of the reduced image,
The reduced image correction means normalizes the reduced Retinex image formed by the reduced Retinex image forming means based on the upper limit value and the lower limit value set by the normalization parameter setting means, and the normalized value is The luminance signal of the reduced image is corrected so that the luminance signal of the reduced image after correction becomes larger as it becomes larger , and then returned to the color image.
The histogram processing parameter acquisition means aggregates the frequency of each pixel value in the entire range taken by the pixel values of the color image corrected by the reduced image correction means, and acquires a parameter for performing histogram processing,
The original image correcting means obtains a Retinex value for only the luminance signal of the original image, normalizes the Retinex value based on the upper limit value and the lower limit value set by the normalization parameter setting means, and the normalized Retinex The luminance signal of the original image is corrected so that the luminance signal of the original image after correction becomes larger as the value becomes larger, and is returned to the color pixel value, and the color pixel value is acquired by the histogram processing parameter acquisition unit The image processing apparatus according to claim 4, wherein histogram correction is performed based on the parameter.   The normalization parameter setting means includes a predetermined proportion of pixels in the range from the maximum value to the minimum value of all the pixels of the reduced Retinex image formed by the reduced Retinex image forming means. The image processing apparatus according to claim 4, wherein an upper limit value and a lower limit value of a clip range including are set as normalization parameters.

Images