JP2007272878A - Image processing program and image processing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing program and an image processing device capable of carrying out Retinex processing at high speed. <P>SOLUTION: A peripheral average luminance image obtaining peripheral average luminance on each pixel of a reduced image is obtained. This processing is computed by carrying out convolution of a coefficient of a filer prepared using a Gaussian function, and the values of peripheral pixels. The filter coefficient is expressed by F(x, y), and the convolution is expressed by F(x, y)*I(x, y). A reduced Retinex image is then formed of the peripheral average luminance image formed by the convolution, and the reduced image obtained by reducing an original image. The thus formed reduced Retinex image is enlarged and normalized to form an output image using the value of the original image. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing program and an image processing apparatus, and more particularly to an image processing program and an image processing apparatus that can speed up Retinex processing.

  When a subject is imaged under backlight conditions, the image of the subject portion becomes an unclear backlight image with low brightness and contrast as the detailed mode becomes difficult to distinguish. In addition to such backlit images, poor images due to overexposure and underexposure, blurring and blurring during shooting, noise, insufficient light quantity, etc. are being used to improve image quality by improving brightness and contrast through image processing. Yes. As one method of such image processing, Retinex processing (Retinex processing) is known.

  In the Retinex process, the input image data is held for the high-quality part, and the image quality is mainly improved for the low-quality part. In this Retinex processing, each pixel data of the original image is corrected to a value reflecting the pixel data of the surrounding pixels by a Gaussian filter, and the reference component data of the original image is obtained from the natural logarithm of the corrected pixel data. The illuminance component data is calculated by dividing the pixel data of the original image by the pixel component of the original image. That is, the original image is divided into two constituent components, a reference component and an illuminance component. Then, after performing processing for correcting the brightness and gradation (contrast) such as gamma correction for the illuminance component, the corrected illuminance component and the reference component are combined, and the original image is backlit. Image data in which the image quality of a low image quality portion such as an image portion is improved can be generated.

  Japanese Patent No. 3731577 (Patent Document 1) discloses a method for increasing the processing speed of the Retinex processing. As shown in FIG. 7, this method forms a reduced image (low resolution) from the original image by a method such as an average pixel method, and calculates a peripheral average luminance image (peripheral average luminance) for each pixel of the reduced image ( A blurred image), an enlarged blurred image obtained by enlarging the blurred image, and an original image to form a Retinex processed image, and further, an output image is formed from the Retinex processed image and the original image.

In FIG. 7, the horizontal coordinate is x, the vertical coordinate is y, and the pixel values (density values) of the original image and the reduced image are represented by I (x, y). When a blurred image is formed from a reduced image, a convolution operation (convolution) is performed with the filter coefficient set to F (x, y). The following expression represents the convolution operation.
F (x, y) * I (x, y)
The Retinex processed image is represented by R (x, y), and the output image is represented by Out (x, y).
Japanese Patent No. 3731577

  However, in the invention disclosed in Patent Document 1, in order to obtain a Retinex-processed image from an enlarged blurred image and an original image, a large number of logarithmic operations must be performed, which requires a long time for the calculation. It was. Further, when forming the output image, a memory for temporarily storing the Retinex processed image (R (x, y)) is necessary for normalization. In order to perform normalization, the maximum value and the minimum value of the values taken by the pixels of the Retinex processed image are extracted, and the frequencies of the pixels of each value in the range of the extracted maximum value and minimum value are aggregated. Create a histogram to clip upper and lower values. Therefore, the Retinex processed image has to be temporarily stored, and there is a disadvantage that the capacity of the memory becomes large.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing program and an image processing apparatus capable of speeding up the Retinex process.

  In order to achieve this object, an image processing program according to claim 1 causes a Retinex process to be executed by a computer, a peripheral average luminance image forming step for forming a peripheral average luminance image of an original image, and a periphery thereof A reduced Retinex image forming step for forming a reduced Retinex image having an image size smaller than the size of the original image based on the peripheral average luminance image formed by the average luminance image forming step and the original image, and the reduced Retinex image formation An enlargement processing step for enlarging the reduced Retinex image formed by the steps, an output image formation step for forming an output image based on the image information enlarged by the enlargement processing step, and the original image.

  The image processing program according to claim 2 is the image processing program according to claim 1, wherein the peripheral average luminance image forming step forms a reduced image having an image size smaller than the original image, and the peripheral average luminance image of the reduced image. A reduced peripheral average luminance image forming step, wherein the reduced Retinex image forming step is based on the reduced peripheral average luminance image and the reduced image formed by the reduced peripheral average luminance image forming step. Form.

  The image processing program according to claim 3 is the image processing program according to claim 2, wherein the reduced peripheral average luminance image forming step is configured to generate pixels by a nearest neighbor method in order to form a reduced image from the original image. A sampling step for extraction and a calculation step for performing a convolution operation between the pixel value extracted by the sampling step and the filter coefficient are provided.

  The image processing program according to claim 4 is the image processing program according to claim 2, wherein the reduced peripheral average luminance image forming step is reduced by an image reduction step of reducing the original image by an average pixel method and a reduction step thereof. And a calculation step for performing a convolution calculation between the pixel of the image and the filter coefficient.

  The image processing program according to claim 5 is the image processing program according to claim 1, wherein the peripheral average luminance image forming step includes a filter processing step for obtaining a peripheral average luminance for all pixels constituting the original image, and the filter processing thereof. A reduction step of reducing an image formed by the peripheral average luminance obtained in the step.

  An image processing program according to a sixth aspect is the image processing program according to any one of the first to fifth aspects, wherein the enlargement processing step is performed such that pixels of the reduced Retinex image corresponding to the pixels constituting the enlarged image are obtained. If it does not exist, a value is acquired based on the pixel values of the reduced Retinex image existing around the pixel position.

  The image processing program according to claim 7 is the image processing program according to claim 6, wherein the enlargement processing step performs an interpolation operation based on values of a plurality of pixels of the reduced Retinex image existing around the pixel position. An interpolation calculation step for obtaining a value is provided.

  An image processing program according to an eighth aspect of the present invention is the image processing program according to the seventh aspect, wherein the interpolation calculation step obtains a pixel value and a value of a reduced Retinex image existing around the position of the pixel whose value is to be obtained. Interpolation is performed by weighting the difference between the pixel value of the original image corresponding to the pixel and the value of the original image corresponding to the surrounding pixels.

  An image processing program according to a ninth aspect is the image processing program according to any one of the first to fifth aspects, wherein the enlargement processing step is performed such that pixels of the reduced Retinex image corresponding to pixels constituting the enlarged image are displayed. If it does not exist, it has a value close to the value of the pixel of the original image corresponding to the pixel, and a value based on the value of the pixel of the reduced Retinex image corresponding to the pixel of the original image existing around the pixel. get.

  An image processing program according to a tenth aspect is the image processing program according to any one of the first to fifth aspects, wherein the enlargement processing step includes a step of calculating a pixel of the reduced Retinex image corresponding to a pixel constituting the enlarged image. If it does not exist, it has a value close to the average value of the pixel value of the original image corresponding to the pixel and the pixel value in the vicinity of the pixel of the original image, and the original image existing around the pixel A value is acquired based on the pixel value of the reduced Retinex image corresponding to the pixel.

  The image processing apparatus according to claim 11 is a computer that executes Retinex processing by a computer, and is formed by a peripheral average luminance image forming unit that forms a peripheral average luminance image of an original image, and a peripheral average luminance image forming unit. A reduced Retinex image forming means for forming a reduced Retinex image having an image size smaller than the size of the original image based on the peripheral average luminance image and the original image, and a reduced Retinex formed by the reduced Retinex image forming means Enlarging processing means for enlarging the image, output image forming means for forming an output image based on the image information enlarged by the enlarging processing means, and the original image are provided.

  The image processing device according to claim 12 is the image processing device according to claim 11, wherein the peripheral average luminance image forming unit forms a reduced image having an image size smaller than the original image, and the peripheral average luminance image of the reduced image. Reduced peripheral average luminance image forming means, and the reduced Retinex image forming means is a reduced Retinex image based on the reduced peripheral average luminance image and the reduced image formed by the reduced peripheral average luminance image forming means. Form.

  The image processing apparatus according to claim 13, wherein the reduced peripheral average luminance image forming means extracts sampling pixels by a nearest neighbor method in order to form a reduced image from the original image. And an arithmetic means for performing a convolution operation between the pixel value extracted by the sampling means and the filter coefficient.

  The image processing device according to claim 14 is the image processing device according to claim 12, wherein the reduced peripheral average luminance image forming means is reduced by an image reduction means for reducing the original image by an average pixel method, and the reduction means. Arithmetic means for performing a convolution operation between the pixels of the image and the filter coefficient.

  The image processing device according to claim 15 is the image processing device according to claim 11, wherein the peripheral average luminance image forming unit is a filter processing unit that obtains the peripheral average luminance for all pixels constituting the original image, and the filter processing thereof. Reduction means for reducing the image formed by the peripheral average luminance obtained by the means.

  The image processing device according to claim 16 is the image processing device according to any one of claims 11 to 15, wherein the enlargement processing unit includes pixels of the reduced Retinex image corresponding to the pixels constituting the enlarged image. If it does not exist, a value is acquired based on the pixel values of the reduced Retinex image existing around the pixel position.

  The image processing device according to claim 17 is the image processing device according to claim 16, wherein the enlargement processing unit performs an interpolation operation based on values of a plurality of pixels of the reduced Retinex image existing around the pixel position. Interpolation calculation means for obtaining the value is provided.

  The image processing device according to claim 18, in the image processing device according to claim 17, wherein the interpolation calculation means obtains the pixel value of the reduced Retinex image existing around the position of the pixel whose value is to be obtained, and the value. Interpolation is performed by weighting the difference between the pixel value of the original image corresponding to the pixel and the value of the original image corresponding to the surrounding pixels.

  The image processing device according to claim 19 is the image processing device according to any one of claims 11 to 15, wherein the enlargement processing unit includes pixels of the reduced Retinex image corresponding to the pixels constituting the enlarged image. If it does not exist, it has a value close to the value of the pixel of the original image corresponding to the pixel, and a value based on the value of the pixel of the reduced Retinex image corresponding to the pixel of the original image existing around the pixel. get.

  The image processing device according to claim 20 is the image processing device according to any one of claims 11 to 15, wherein the enlargement processing unit includes pixels of a reduced Retinex image corresponding to pixels constituting the enlarged image. If it does not exist, it has a value close to the average value of the pixel value of the original image corresponding to the pixel and the pixel value in the vicinity of the pixel of the original image, and the original image existing around the pixel A value is acquired based on the pixel value of the reduced Retinex image corresponding to the pixel.

  According to the image processing program of claim 1, the peripheral average luminance image forming step for forming the peripheral average luminance image of the original image, the peripheral average luminance image formed by the peripheral average luminance image forming step, and the original image Based on this, the reduced Retinex image forming step forms a reduced Retinex image having an image size smaller than that of the original image. Next, the formed reduced Retinex image is enlarged by the enlargement processing step, and the output image is formed by the output image forming step based on the enlarged image information and the original image. Compared with the process of enlarging a luminance image and forming a Retinex image from the enlarged image and the original image, the number of logarithmic operations when forming a reduced Retinex image is small, and processing can be performed at high speed. There is an effect. Further, since the reduced Retinex image is small in size, there is an effect that the memory capacity can be reduced.

  According to the image processing program of claim 2, in addition to the effect of the image processing program of claim 1, the peripheral average luminance image forming step forms a reduced image having an image size smaller than that of the original image, and the reduced image The reduced peripheral average luminance image forming step for forming the peripheral average luminance image of the image is reduced, and the reduced Retinex image forming step is reduced based on the reduced peripheral average luminance image and the reduced image formed by the reduced peripheral average luminance image forming step. Since the Retinex image is formed, the number of convolution operations performed when forming the peripheral average luminance image of the reduced image can be reduced, and there is an effect that processing can be performed at high speed.

  According to the image processing program of claim 3, in addition to the effect produced by the image processing program of claim 2, the reduced peripheral average luminance image forming step includes a sampling step of extracting pixels from the original image by a nearest neighbor method, A calculation step for performing a convolution operation between the pixel value extracted by the sampling step and the filter coefficient, so that it is easy to extract pixels for forming a reduced image and a product to be performed by the convolution operation. There is an effect that the number of times of the sum operation is small and processing can be performed at high speed.

  According to the image processing program of claim 4, in addition to the effect produced by the image processing program of claim 2, the reduced peripheral average luminance image forming step includes an image reduction step of reducing the original image by an average pixel method, Since it has a calculation step that performs a convolution operation between the pixel of the image reduced by the reduction step and the filter coefficient, the value of the pixel used in the convolution operation reflects the value of all the pixels forming the image Therefore, there is an effect that a high-quality image can be obtained.

  According to the image processing program of claim 5, in addition to the effect produced by the image processing program of claim 1, the peripheral average luminance image forming step is a filter processing step for obtaining the peripheral average luminance for all the pixels constituting the original image. And a reduction step for reducing the image formed by the peripheral average luminance obtained by the filter processing step, the number of product-sum operations in the convolution operation is not reduced, but a higher quality image is obtained. There is an effect that can be.

  According to the image processing program of the sixth aspect, in addition to the effect produced by the image processing program according to any one of the first to fifth aspects, the enlargement processing step includes a reduction ratio corresponding to the pixels constituting the enlarged image. When the pixel of the nexus image does not exist, the value is acquired based on the value of the pixel of the reduced retinex image existing around the pixel position, so that the processing is simple and the processing can be performed at high speed. effective.

  According to the image processing program of the seventh aspect, in addition to the effect produced by the image processing program of the sixth aspect, the enlargement processing step converts the values of a plurality of pixels of the reduced Retinex image existing around the pixel position. Since an interpolation calculation step for obtaining a value by interpolation calculation is provided, it is possible to obtain a higher quality image by reflecting the values around the pixel as compared with the value of the pixel at the closest position. There is an effect.

  According to the image processing program of the eighth aspect, in addition to the effect produced by the image processing program of the seventh aspect, the interpolation calculation step includes the value of the pixel of the reduced Retinex image existing around the position of the pixel whose value is to be obtained. And the interpolation calculation is performed by weighting the difference between the value of the pixel of the original image corresponding to the pixel for which the value is obtained and the value of the original image corresponding to the pixel existing in the surrounding area. The pixel value and the values of the surrounding pixels are reflected, and there is an effect that a higher quality image can be obtained.

  According to the image processing program of the ninth aspect, in addition to the effect produced by the image processing program according to any one of the first to fifth aspects, the enlargement processing step includes a reduction ratio corresponding to the pixels constituting the enlarged image. When the pixel of the nexus image does not exist, the value of the pixel of the reduced Retinex image corresponding to the pixel of the original image existing around the pixel has a value close to the value of the pixel of the original image corresponding to the pixel Since the value is acquired based on the above, it is possible to perform a better approximation and obtain a higher quality image than to employ the pixel value of the reduced Retinex image that is close in position. .

  According to the image processing program of the tenth aspect, in addition to the effect produced by the image processing program according to any one of the first to fifth aspects, the enlargement processing step includes a reduction ratio corresponding to the pixels constituting the enlarged image. When the pixel of the nexus image does not exist, it has a value close to the average value of the value of the pixel of the original image corresponding to the pixel and the value of the pixel in the vicinity of the pixel of the original image, and around the pixel Since the value is acquired based on the pixel value of the reduced Retinex image corresponding to the pixel of the existing original image, the value of the pixel in the vicinity of the desired pixel can be reflected, and a higher quality image can be obtained. There is.

  According to the image processing apparatus of claim 11, the peripheral average luminance image forming unit that forms the peripheral average luminance image of the original image, the peripheral average luminance image formed by the peripheral average luminance image forming unit, and the original image Based on this, a reduced Retinex image forming unit forms a reduced Retinex image having an image size smaller than that of the original image. Next, the formed reduced Retinex image is enlarged by the enlargement processing means, and an output image is formed by the output image forming means based on the enlarged image information and the original image. Compared with the process of enlarging a luminance image and forming a Retinex image from the enlarged image and the original image, the number of logarithmic operations when forming a reduced Retinex image is small, and processing can be performed at high speed. There is an effect. Further, since the reduced Retinex image is small in size, there is an effect that the memory capacity can be reduced.

  According to the image processing device of the twelfth aspect, in addition to the effect produced by the image processing device of the eleventh aspect, the peripheral average luminance image forming unit forms a reduced image having an image size smaller than the original image, and the reduced image The reduced peripheral average luminance image forming means for forming the peripheral average luminance image of the image is reduced, and the reduced Retinex image forming means is reduced based on the reduced peripheral average luminance image and the reduced image formed by the reduced peripheral average luminance image forming means. Since the Retinex image is formed, the number of convolution operations performed when forming the peripheral average luminance image of the reduced image can be reduced, and there is an effect that processing can be performed at high speed.

  According to the image processing device of claim 13, in addition to the effect produced by the image processing device of claim 12, the reduced peripheral average luminance image forming means includes sampling means for extracting pixels from the original image by the nearest neighbor method, Since there is a calculation means for performing a convolution operation between the pixel value extracted by the sampling means and the filter coefficient, it is easy to extract pixels for forming a reduced image, and a product to be performed by the convolution calculation is used. There is an effect that the number of times of the sum operation is small and processing can be performed at high speed.

  According to the image processing device of the fourteenth aspect, in addition to the effect produced by the image processing device of the twelfth aspect, the reduced peripheral average luminance image forming unit includes an image reducing unit that reduces the original image by an average pixel method, Since it is provided with arithmetic means for performing a convolution operation between the pixel of the image reduced by the reduction means and the filter coefficient, the pixel values used in the convolution operation reflect the values of all the pixels forming the image. Therefore, there is an effect that a high-quality image can be obtained.

  According to the image processing device of claim 15, in addition to the effect achieved by the image processing device of claim 11, the peripheral average luminance image forming unit is a filter processing unit that calculates the peripheral average luminance for all the pixels constituting the original image. And a reduction means for reducing the image formed by the peripheral average luminance obtained by the filter processing means, the number of product-sum operations in the convolution operation is not reduced, but a higher quality image is obtained. There is an effect that can be.

  According to the image processing apparatus of the sixteenth aspect, in addition to the effect produced by the image processing apparatus according to any one of the eleventh to fifteenth aspects, the enlargement processing means includes a reduction ratio corresponding to the pixels constituting the enlarged image. When the pixel of the nexus image does not exist, the value is acquired based on the value of the pixel of the reduced retinex image existing around the pixel position, so that the processing is simple and the processing can be performed at high speed. effective.

  According to the image processing device of claim 17, in addition to the effect produced by the image processing device of claim 16, the enlargement processing means converts the values of the plurality of pixels of the reduced Retinex image existing around the pixel position. Interpolation calculation means for obtaining a value by interpolation calculation based on it is provided, so that a higher quality image can be obtained by reflecting the values around the pixel as compared with the value of the pixel at the nearest position. There is an effect.

  According to the image processing device of claim 18, in addition to the effect achieved by the image processing device of claim 17, the interpolation calculation means is configured to obtain the pixel value of the reduced Retinex image existing around the position of the pixel whose value is to be obtained. And the interpolation calculation is performed by weighting the difference between the value of the pixel of the original image corresponding to the pixel for which the value is obtained and the value of the original image corresponding to the pixel existing in the surrounding area. The pixel value and the values of the surrounding pixels are reflected, and there is an effect that a higher quality image can be obtained.

  According to the image processing device of the nineteenth aspect, in addition to the effect produced by the image processing device according to any one of the eleventh to fifteenth aspects, the enlargement processing means includes a reduction ratio corresponding to the pixels constituting the enlarged image. When the pixel of the nexus image does not exist, the value of the pixel of the reduced Retinex image corresponding to the pixel of the original image existing around the pixel has a value close to the value of the pixel of the original image corresponding to the pixel Since the value is acquired based on the above, it is possible to perform a better approximation and obtain a higher quality image than to employ the pixel value of the reduced Retinex image that is close in position. .

  According to the image processing device of the twentieth aspect, in addition to the effect produced by the image processing device according to any one of the eleventh to fifteenth aspects, the enlargement processing means includes a reduction ratio corresponding to the pixels constituting the enlarged image. When the pixel of the nexus image does not exist, it has a value close to the average value of the value of the pixel of the original image corresponding to the pixel and the value of the pixel in the vicinity of the pixel of the original image, and around the pixel Since the value is acquired based on the pixel value of the reduced Retinex image corresponding to the pixel of the existing original image, the value of the pixel in the vicinity of the desired pixel can be reflected, and a higher quality image can be obtained. There is.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a block diagram showing an electrical configuration of a printer 1 equipped with an image processing program according to the present embodiment. In the present embodiment, the image processing program installed in the printer 1 is image data (original image data or the like) input from the personal computer (hereinafter referred to as “PC”) 2, the digital camera 21, or the external medium 20. ), A Retinex process (Retinex process) is executed to correct a low image quality region such as a backlight image portion of the image data.

  As shown in FIG. 1, the printer 1 includes a CPU 11, a ROM 12, a RAM 13, a print head, and the like, and includes a printing unit 15 that performs printing (output) on a print medium (for example, a paper medium), an output image size, and the like. And an operation panel 16 having a user operation unit (for example, a numeric keypad) through which the user can input the input value.

  The printer 1 includes an interface (hereinafter referred to as “I / F”) 17 that can be connected to the PC 2 via the cable 5, an I / F 18 that can be connected to the digital camera 21 via the cable 6, and an external medium 20. And an external media slot 19 that can be detachably mounted (for example, a flash memory such as an SD memory card or a memory stick). In recent years, USB (Universal Serial Bus) is used as a communication method performed by these I / Fs 17 and 18.

  Therefore, the printer 1 can input the image data stored in the PC 2 via the cable 5 and the I / F 17 and also the image data photographed by the digital camera 21 via the cable 6 and the I / F 18. It is possible to input via. Further, it is possible to input image data stored in the external medium 21 from the external medium 20 mounted in the external media slot 19.

  The CPU 11 is an arithmetic processing device (arithmetic device) that controls the entire printer 1. The ROM 12 stores various control programs executed by the CPU 11 and fixed values used when executing the programs. In this embodiment, an image processing program that performs image processing such as Retinex processing is stored. An image processing program memory 12a for storing data, a Gaussian function memory 12b for storing Gaussian functions used in the processing, and the like are provided.

  The filter coefficient used for the filter processing (also referred to as mask processing) in the Retinex processing is obtained by calculation using a Gaussian function stored in the Gaussian function memory 12b.

  The RAM 13 is a rewritable memory that has a working area in which various register groups necessary for a control program executed by the CPU 11 are set, a temporary area that temporarily stores data being processed, and can be accessed at random. And an original image memory 13a for storing original image data, a reduced Retinex image memory 13b for storing reduced Retinex images obtained by Retinex processing, and the like.

  The original image data memory 13 a stores image data input from the PC 2, the digital camera 21, and the external media 20 via the I / F 17, I / F 18, and external media slot 19, respectively. In the present embodiment, both the original image data and the output image data are composed of RGB values, and these RGB values are values in the range of “0” to “255”.

  Note that the RGB value is a value having, as constituent components, an R value representing red, which is the three primary colors of light, a G value representing green, and a B value representing blue. Since various colors are generated by mixing the three primary colors of light, the color of each pixel of the input image is one color (hue, gradation, etc.) by a combination of R value, G value, and B value (RGB value). ) Is displayed. The luminance (lightness) increases as the RGB value increases.

  The reduced Retinex image memory 13b is a memory that stores an image formed by the average luminance image and the original image during the Retinex process. When the output image is formed, the maximum value and the minimum value of each pixel of the reduced Retinex image are detected, and normalization is performed by a clipping process for limiting the upper and lower values of the histogram.

  Next, the Retinex process executed by the CPU 11 of the printer 1 will be described with reference to FIG. FIG. 2 is a diagram schematically showing an outline of the Retinex process executed by the printer 1.

  First, a reduced image is formed from the original image. As a method of forming this reduced image, the nearest neighbor method (nearest neighbor method) that takes the value (density value) of pixels extracted (sampled) at predetermined intervals among the pixels constituting the original image, and the surrounding four methods Bilinear method for obtaining a value by linear interpolation from the values of one pixel, Bicubic method for obtaining a value by interpolation using the cubic function from the values of 16 surrounding pixels, and averaging the values in consideration of the area occupied by the original image An average pixel method for obtaining a value by doing so is known. Of these, the nearest neighbor method is simple because it simply extracts pixels at a predetermined position, and can be processed at high speed. On the other hand, since the average pixel method is a method of reducing using all the pixels, the reduced image has the highest image quality.

  Next, a peripheral average luminance image obtained by calculating the peripheral average luminance for each pixel of the reduced image is obtained. This processing is calculated by convolving a filter coefficient created using a Gaussian function and the pixel value of the reduced image. The abscissa of the pixel of the reduced image obtained by reducing the original image is represented by x, the ordinate is represented by y, the value of the pixel is represented by I (x, y), and the filter coefficient is similarly F (x, y). The convolution operation is represented as F (x, y) * I (x, y).

  Next, a reduced Retinex image is formed from the peripheral average luminance image formed by the convolution operation and the reduced image. This calculation is expressed by Equation 1 below, where R (x, y) is the pixel value of the Retinex image.

この数式が示すように、この処理では、各画素について、対数変換が行われる。

As shown in this equation, logarithmic conversion is performed for each pixel in this process.

  Next, the reduced Retinex image formed in this way is enlarged and normalized, and an output image is formed using the values of the original image. This calculation is expressed by Equation 2 below, where Out (x, y) is the pixel value of the output image.

ここで、ｒｅｆｌｅ（ｘ、ｙ）は、Ｒ（ｘ、ｙ）を正規化した値である。
なお、縮小レティネックス画像を拡大する方法には、各種の方法があり、後述する。

Here, refle (x, y) is a value obtained by normalizing R (x, y).
There are various methods for enlarging the reduced Retinex image, which will be described later.

  Next, a method for forming a peripheral average luminance image will be described with reference to FIGS. FIG. 3A is a table showing the pixels of the original image, which is composed of 16 horizontal pixels and 16 vertical pixels. In each pixel value I, a suffix x is applied from the left end to the right. .., 0, 1, 2,..., 0, 1, 2,.

  FIG. 3B is a table showing the configuration of the filter. Here, the table is composed of 5 pixels in the horizontal direction and 5 pixels in the vertical direction, and each filter coefficient F has subscripts x and y as in the case of the original image. It is added. These filter coefficients have a large value at the center using a Gaussian function, and a value that decreases as the distance from the center increases.

  A peripheral average luminance image is formed by performing a convolution operation on the filter shown in FIG. 3B and each pixel of the reduced image obtained by reducing the original image shown in FIG.

  Next, a reduced Retinex image shown in FIG. 4A is formed by calculating the peripheral average luminance image and the reduced image thus formed according to the above mathematical formula 1. FIG. 4A is a table showing a reduced Retinex image obtained by reducing the size of the image (decreasing the resolution). The table includes 5 pixels in the horizontal direction and 5 pixels in the vertical direction. The value Q of each pixel includes the value Q of the original image. Subscripts x and y are added as in the case.

  In the method of forming the reduced Retinex image shown in FIG. 4A, as described above, first, the original image is reduced, and the reduced average image is reduced by filtering the reduced image. A method of forming an image and forming a reduced Retinec image using Equation 1 from the reduced peripheral average luminance image and the reduced image, and forming the peripheral average luminance image at the size of the original image (original resolution) There is a method of reducing the peripheral average luminance image and forming a reduced Retinex image using Equation 1 based on the reduced image obtained by reducing the original image.

  When the original image is reduced and filter processing is performed on the reduced image, the number of product-sum operations in the convolution operation can be reduced, so that the operation can be performed at higher speed. Further, when the average pixel method is used when reducing the image, the values of all the pixels can be reflected, so that the image quality of the processed image can be improved.

  Further, since the number of times the logarithmic calculation of Formula 1 is performed is smaller than that in the case of forming a retinex image that is not reduced, the processing can be speeded up and a memory for storing a retinex image that needs to be temporarily stored is stored. The storage capacity can be reduced.

  In any case, the nearest neighbor method, bilinear method, bicubic method, average pixel method, or the like can be used as a method for reducing an image. When a reduced image is formed, the reduced image is stored, a peripheral average luminance image is formed based on the stored reduced image, and then reduced based on the peripheral average luminance image and the stored reduced image. A Retinex image may be formed, or a reduced image may be formed when a peripheral average luminance image is formed without storing a reduced image and when a reduced Retinex image is formed.

  Next, a method for enlarging the reduced Retinex image shown in FIG. 4A to the Retinex image shown in FIG. 4B will be described. FIG. 4B is a table showing Retinex images. Here, similarly to the original image, the table is composed of 16 pixels in the horizontal direction and 16 pixels in the vertical direction, and the value R of each pixel is the same as in the case of the original image. Similarly, subscripts x and y are added. As for the position of the pixel shown in gray, it indicates that a value corresponding to the reduced Retinex image shown in FIG.

There are the following five methods for enlarging the reduced Retinex image.
(1) First method A pixel value of a reduced Retinex image having the closest coordinate position is used.
For example, in the Retinex image shown in FIG. 4B, R (0,0), R (0,1), R (0,2), R (1,0), R (1,1), R The values of (1,2), R (2,0), R (2,1), R (2,2) are the values of Q (0,0) of the reduced Retinex image, and R (0,3) ), R (0,4), R (0,5), R (0,6), R (1,3), R (1,4), R (1,5), R (1,6) , R (2,3), R (2,4), R (2,5), R (2,6) are the values of Q (0,4) of the reduced Retinex image.

(2) Second method Obtained by linear interpolation from the values of the reduced Retinex images at four positions around the coordinate position.
For example, the value of R (1,1) is obtained by the following equation.

（３）第３の方法
第３の方法については、図５を参照して説明する。図５は、元画像の一部の画素の値を示すテーブルである。求めるレティネックス画像の座標位置に対応する元画像の画素の値に最も近い元画像の画素の値に対応する縮小レティネックス画像の画素の値とする。例えば、Ｒ（１，１）を求める場合に、図５に示すようにＩ（１，１）が１９０であり、
Ｉ（０，０）＝２００、Ｉ（０，４）＝１８０、Ｉ（４，０）＝１９５，Ｉ（４，４）＝２１０であるとすれば、１９０に最も近い値である１９５の値を有するＩ（４，０）に対応する縮小レティネックス画像の画素の値であるＱ（４，０）とする。
尚、図５において、説明に使用しない画素については、その値を「−」で示している。

(3) Third Method The third method will be described with reference to FIG. FIG. 5 is a table showing values of some pixels of the original image. The pixel value of the reduced Retinex image corresponding to the pixel value of the original image closest to the pixel value of the original image corresponding to the coordinate position of the Retinex image to be obtained is set. For example, when R (1,1) is obtained, I (1,1) is 190 as shown in FIG.
If I (0,0) = 200, I (0,4) = 180, I (4,0) = 195, I (4,4) = 210, the value of 195 closest to 190 Let Q (4,0) be the pixel value of the reduced Retinex image corresponding to I (4,0) having a value.
In FIG. 5, the value of a pixel that is not used for description is indicated by “−”.

(4) Fourth method This is a modification of the third method, in which the pixel of the original image closest to the average value of the pixel of the original image corresponding to the coordinate position of the Retinex image to be obtained and the pixels in the vicinity of the pixel is calculated. The pixel value of the reduced Retinex image corresponding to the value is used. For example, the average of four pixels adjacent to the left and right and up and down of the pixel of interest is taken, and, as in (3), I (0,0) = 200, I (0,4) = 180, I (4,0) = 195, I (4,4) = 210, and when R (1,1) is obtained, the value of I (1,1) is 190, and the value of I (0,1) is Assuming 200, the value of I (1,0) is 210, the value of I (1,2) is 215, and the value of I (2,1) is 220, the average value is (190 + 200 + 210 + 215 + 220) / 5 = 207 It becomes.
Therefore, since the value of I (0,4) = 210 is the closest, Q (0,4) is taken.

  Here, the average of the five pixels is taken, but the value of the pixels on the diagonal line may be added to take the average. Further, the minimum value and the maximum value of these pixel values may be detected, and an average value of the minimum value and the maximum value may be taken.

(5) Fifth method This is a modification of the second method, and uses the values of the four reduced Retinex images in the vicinity of the pixel to be obtained, the values of the original image corresponding to the pixel to be obtained, and the four Interpolation is performed using the difference between the pixel value and the original image as a weighting factor. That is, first, the absolute value of the difference between the pixel value of the original image corresponding to the pixel to be obtained and the pixel value of the original image corresponding to the four neighboring points is obtained.
For example, R (1,1) is calculated by the following equation.

ここで、
ｄｉｆｆ（０，０）＝ａｂｓ（Ｉ（０，０）−Ｉ（１，１））
ｄｉｆｆ（４，０）＝ａｂｓ（Ｉ（４，０）−Ｉ（１，１））
ｄｉｆｆ（０，４）＝ａｂｓ（Ｉ（０，４）−Ｉ（１，１））
ｄｉｆｆ（４，４）＝ａｂｓ（Ｉ（４，４）−Ｉ（１，１））
ただし、ａｂｓは、絶対値である。
次に、これらの絶対値、ｄｉｆｆ（０，０）、ｄｉｆｆ（４，０）、ｄｉｆｆ（０，４）、ｄｉｆｆ（４，４）のうちの最大値ｄｉｆｆ＿ｍａｘを求める。

here,
diff (0,0) = abs (I (0,0) -I (1,1))
diff (4,0) = abs (I (4,0) -I (1,1))
diff (0,4) = abs (I (0,4) -I (1,1))
diff (4,4) = abs (I (4,4) -I (1,1))
Here, abs is an absolute value.
Next, the maximum value diff_max among these absolute values, diff (0, 0), diff (4, 0), diff (0, 4), diff (4, 4) is obtained.

Next, the difference between the maximum value diff_max and each absolute value is obtained by the following equation:
diffm (0,0) = diff_max−diff (0,0)
difm (4,0) = diff_max−diff (4,0)
diffm (0,4) = diff_max−diff (0,4)
difm (4,4) = diff_max−diff (4,4)
Diff_sum, which is the sum of these, is calculated by the following equation.
diff_sum = difm (0,0) + difm (4,0) + difm (0,4) +
difm (4,4)
The weighting factors for each of the four points, W (0,0), W (4,0), W (0,4), and W (4,4) are the sum of the difference between the maximum value diff_max and each absolute value. Is obtained by the following equation.
W (0,0) = diffm (0,0) / diff_sum
W (4,0) = diffm (4,0) / diff_sum
W (0,4) = diffm (0,4) / diff_sum
W (4,4) = diffm (4,4) / diff_sum
According to the fifth method, since the density value of the pixel of the original image is reflected, a processed image with higher image quality can be obtained.

  FIG. 6 is an example showing an image that has been subjected to Retinex processing according to the present invention, and FIG. 6 (a) shows a shaggy in a portion indicated by a broken line A that shows an image when enlarged by the second method. Has occurred. On the other hand, FIG. 6B shows an image when enlargement is performed by the fifth method, and shows an image that has been subjected to Retinex processing without generating shaggy as shown in FIG.

  As described above, according to the above-described embodiment, when performing Retinex processing, a reduced Retinex image in which the resolution of the original image is reduced is formed, and then the reduced Retinex image is enlarged. Since the Retinex image is formed by normalization and the output image is formed with the original image, the calculation for forming the reduced Retinex image can be speeded up, and the reduced Retinex image has a size. Since it is small, the storage capacity of the memory can be reduced. Therefore, the image processing apparatus can be provided at a low cost.

  As described above, the present invention has been described based on the embodiments. However, the present invention is not limited to the above embodiments, and various improvements and modifications can be made without departing from the spirit of the present invention. It can be easily guessed.

  For example, in the above embodiment, the color representation system is the RGB format, but other color representation systems other than the RGB format, for example, a format in which brightness and color components are separated, such as Lab format and Ycc format. Since the present invention can be applied to the lightness component (L or Y), the present invention may be applied to the Lab format or the Ycc format.

  In the above embodiment, the filter coefficient is obtained by calculation using a Gaussian function. However, a filter with little deterioration in image quality according to the size and image quality of a frequently used image is stored in the memory in advance and stored. Filter processing may be performed by calculating the filter coefficient of the filter and the density value of the pixel.

  In the above embodiment, the image processing program of the present invention is executed by the CPU 11 incorporated in the printer 1, but is supplied as an application to the personal computer and executed by the CPU incorporated in the personal computer. You may make it do.

  The Retinex process may be SSR (single scale method) or MSR (multiscale method).

  In the image processing of the above embodiment, the Retinex processing is performed by the CPU 11, but may be performed by a DSP (Digital Signal Processor). If a DSP is used, processing such as a product-sum operation can be executed at a higher speed.

1 is a block diagram showing an electrical configuration of a printer equipped with an image processing program according to an embodiment of the present invention. It is a flowchart which shows the process performed by an image processing program. (A) shows the pixel of an original image, (b) is a table which shows the structure of a filter. (A) is a table showing a reduced Retinex image, and (b) is a table showing a Retinex image obtained by enlarging the reduced Retinex image shown in (a). It is a table which shows some pixels of an original image. It is an example which shows the image which performed the Retinex process by this invention, (a) shows the image at the time of enlarging by the 2nd method, (b) enlarged by the 5th method An image of the case is shown. It is a flowchart which shows the Retinex process in a prior art.

Explanation of symbols

1 Printer 2 Personal Computer 11 CPU
12 ROM
12a Image processing program memory 13 RAM
13b Reduced Retinex image memory

Claims (20)

In an image processing program for executing Retinex processing by a computer,
A peripheral average luminance image forming step for forming a peripheral average luminance image of the original image;
A reduced Retinex image forming step for forming a reduced Retinex image having an image size smaller than the size of the original image based on the peripheral average luminance image formed by the peripheral average luminance image forming step and the original image;
An enlargement processing step of enlarging the reduced Retinex image formed by the reduced Retinex image forming step;
An image processing program comprising: an output image forming step for forming an output image based on the image information enlarged by the enlargement processing step and the original image. The peripheral average luminance image forming step includes a reduced peripheral average luminance image forming step of forming a reduced image having an image size smaller than the original image and forming a peripheral average luminance image of the reduced image,
2. The reduced Retinex image forming step forms a reduced Retinex image based on the reduced peripheral average luminance image and the reduced image formed by the reduced peripheral average luminance image forming step. Image processing program.   The reduced peripheral average luminance image forming step includes a sampling step of extracting pixels by a nearest neighbor method to form a reduced image from the original image, a pixel value extracted by the sampling step, and a filter coefficient, The image processing program according to claim 2, further comprising an operation step for performing a convolution operation.   The reduced peripheral average luminance image forming step includes an image reduction step for reducing the original image by an average pixel method, and a calculation step for performing a convolution operation between the pixels of the image reduced by the reduction step and a filter coefficient. The image processing program according to claim 2, wherein:   The peripheral average luminance image forming step includes a filter processing step for obtaining the peripheral average luminance for all the pixels constituting the original image, and a reduction step for reducing the image formed by the peripheral average luminance obtained by the filter processing step. The image processing program according to claim 1, further comprising:   In the case where there is no pixel of the reduced Retinex image corresponding to the pixel constituting the enlarged image, the enlargement processing step is a value based on the value of the pixel of the reduced Retinex image existing around the pixel position. The image processing program according to claim 1, wherein the image processing program is acquired.   7. The enlargement processing step includes an interpolation calculation step of obtaining values by interpolation calculation based on values of a plurality of pixels of a reduced Retinex image existing around the pixel position. Image processing program.   The interpolation calculation step corresponds to the pixel value of the reduced Retinex image existing around the position of the pixel whose value is to be determined, the value of the pixel of the original image corresponding to the pixel whose value is to be determined, and the pixel existing around the pixel. 8. The image processing program according to claim 7, wherein the interpolation calculation is performed by weighting the difference from the value of the original image.   The enlargement processing step has a value close to the value of the pixel of the original image corresponding to the pixel when the pixel of the reduced Retinex image corresponding to the pixel constituting the enlarged image does not exist. 6. The image processing program according to claim 1, wherein a value is acquired based on a value of a pixel of a reduced Retinex image corresponding to a pixel of an original image existing around.   In the enlargement processing step, when there is no pixel of the reduced Retinex image corresponding to the pixel constituting the enlarged image, the value of the pixel of the original image corresponding to the pixel and the vicinity of the pixel of the original image The value is obtained based on a pixel value of a reduced Retinex image corresponding to a pixel of an original image existing around the pixel and having a value close to an average value of the pixel value. The image processing program according to any one of 1 to 5. In an image processing apparatus that executes Retinex processing by a computer,
A peripheral average luminance image forming means for forming a peripheral average luminance image of the original image;
A reduced Retinex image forming means for forming a reduced Retinex image having an image size smaller than the size of the original image based on the peripheral average brightness image formed by the peripheral average brightness image forming means and the original image;
Enlargement processing means for enlarging the reduced Retinex image formed by the reduced Retinex image forming means,
An image processing apparatus comprising: output image forming means for forming an output image based on image information enlarged by the enlargement processing means and an original image. The peripheral average luminance image forming unit includes a reduced peripheral average luminance image forming unit that forms a reduced image having a smaller image size than the original image and forms a peripheral average luminance image of the reduced image,
12. The reduced Retinex image forming unit forms a reduced Retinex image based on the reduced peripheral average luminance image and the reduced image formed by the reduced peripheral average luminance image forming unit. Image processing device.   The reduced peripheral average luminance image forming means is a sampling means for extracting pixels by a nearest neighbor method to form a reduced image from the original image, and a convolution operation between the pixel value extracted by the sampling means and the filter coefficient. The image processing apparatus according to claim 12, further comprising a calculation unit that performs the calculation.   The reduced peripheral average luminance image forming means includes an image reducing means for reducing the original image by the average pixel method, and an arithmetic means for performing a convolution operation between the pixels of the image reduced by the reducing means and the filter coefficient. The image processing apparatus according to claim 12.   The peripheral average brightness image forming means includes a filter processing means for obtaining the peripheral average brightness for all pixels constituting the original image, and a reduction means for reducing an image formed by the peripheral average brightness obtained by the filter processing means. The image processing apparatus according to claim 11, further comprising:   If there is no pixel of the reduced Retinex image corresponding to the pixel constituting the enlarged image, the enlargement processing unit determines a value based on the value of the pixel of the reduced Retinex image existing around the pixel position. The image processing apparatus according to claim 11, further comprising:   17. The enlargement processing means includes interpolation calculation means for obtaining values by interpolation calculation based on values of a plurality of pixels of a reduced Retinex image existing around the pixel position. Image processing apparatus.   The interpolation calculation means corresponds to the pixel value of the reduced Retinex image existing around the position of the pixel whose value is to be determined, the value of the pixel of the original image corresponding to the pixel whose value is to be determined, and the pixel existing around the pixel. 18. The image processing apparatus according to claim 17, wherein interpolation is performed by weighting the difference from the value of the original image.   The enlargement processing unit has a value close to the value of the pixel of the original image corresponding to the pixel when the pixel of the reduced Retinex image corresponding to the pixel constituting the enlarged image does not exist, The image processing apparatus according to claim 11, wherein a value is acquired based on a value of a pixel of a reduced Retinex image corresponding to a pixel of an original image existing around.   If there is no pixel in the reduced Retinex image corresponding to the pixel constituting the enlarged image, the enlargement processing unit determines the value of the pixel in the original image corresponding to the pixel and the vicinity of the pixel in the original image. The value is obtained based on a pixel value of a reduced Retinex image corresponding to a pixel of an original image existing around the pixel and having a value close to an average value of the pixel value. The image processing device according to any one of 11 to 15.

Images