JP2014155167A - Image processing apparatus, method, and program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of properly improving image quality.SOLUTION: An image processing apparatus comprises an input unit, a first color casting correction unit, a food determination unit, and a food correction unit. The input unit receives input of an image. The first color casting correction unit corrects color cast of the image. The food determination unit determines whether or not objects in the image corrected by the first color casting correction unit include food. When the food determination unit determines that food is included in the objects, the food correction unit corrects color components of the image corrected by the first color casting correction unit.

Description

  The present invention relates to an image processing apparatus, an image processing method, an image processing program, and a recording medium.

  2. Description of the Related Art Conventionally, as an image processing apparatus, an apparatus that performs correction processing on an input image is known (see, for example, Patent Document 1). The apparatus described in Patent Literature 1 executes a combination of luminance / saturation correction processing, enlargement processing, and color noise reduction processing on an input image. In this apparatus, after performing the color noise reduction process, the brightness / saturation correction process is performed.

JP 2001-186366 A

  With the recent spread of digital cameras and camera-equipped mobile terminals, interest in image quality such as captured images has increased. For this reason, it is desired to improve the image quality by appropriately correcting an image that cannot be re-taken. There are many image quality to be improved, such as the image quality related to luminance, the image quality related to saturation, the image quality related to edge and the image quality related to noise described in Patent Document 1, as well as the image quality related to brightness and the image quality related to color covering. And when improving several image quality simultaneously, it is necessary to perform each correction process sequentially.

However, when an attempt is made to improve a plurality of image quality, the preceding correction process adversely affects the subsequent correction process, and as a result, the image quality may be degraded. Furthermore, since the quality of the image felt by a person varies depending on the subject, an appropriate correction process is required for each subject. For this reason, the apparatus described in Patent Document 1 may not be able to perform a sufficiently satisfactory correction.
Therefore, in this technical field, an image processing apparatus, an image processing method, an image processing program, and a recording medium on which the program can be recorded that can appropriately improve image quality are desired.

  That is, an image processing apparatus according to one aspect of the present invention is an apparatus that corrects an image. The apparatus includes an input unit, a first color covering correction unit, a food determination unit, and a food correction unit. The input unit inputs an image. The first color fog correction unit corrects the color fog of the image. The food determination unit determines whether food is included in the subject of the image after being corrected by the first color covering correction unit. The food correction unit corrects the color components of the image corrected by the first color covering correction unit when the food determination unit determines that the subject contains food.

  In this image processing apparatus, the first color covering correction unit corrects the color covering of the image, and the food determination unit determines whether food is included in the subject of the image after the color covering correction. When it is determined that food is included, the color component of the image after color covering correction is corrected by the food correction unit. In this way, when improving the image quality related to color covering and the image quality related to food at the same time, it is necessary to determine whether or not the subject contains food. Therefore, the accuracy of food determination can be improved by performing color coverage correction before the determination. Therefore, it is possible to improve the image quality as compared with the case where the color covering correction is performed after the food correction.

  In one embodiment, the food determination unit obtains food features in advance by learning using a teacher image, the image features after being corrected by the first color covering correction unit, and the food features obtained by learning. , It may be determined whether or not food is included in the subject of the image after being corrected by the first color covering correction unit. Thus, by defining food by prior learning, it can be determined whether it is food.

  In one embodiment, the food correction unit may emphasize the brightness of each pixel at a certain ratio. In one embodiment, the food correction unit corrects the saturation of each pixel by weighting so that the saturation of the pixels of the warm hue hue is emphasized more than the saturation of the pixels of the non-warm hue hue. May be. By correcting in this way, the appearance of food can be improved.

  In one embodiment, the first color covering correction unit determines that the color of a pixel whose luminance value is equal to or larger than a predetermined value and whose ratio of the color difference value to the luminance value is equal to or smaller than the predetermined value is white. The degree of covering may be estimated. With this configuration, it is possible to estimate the degree of color coverage while appropriately determining pixels that were originally white.

  In one embodiment, the image processing apparatus further includes a first noise correction unit that corrects noise of the image input by the input unit, and the first color fog correction unit is corrected by the first noise correction unit. The color coverage of the image may be corrected. In one embodiment, the first noise correction unit may expand the bit of each pixel after correcting the noise of the image input by the input unit. By expanding the bits, the calculation accuracy of the subsequent processing can be improved.

  In one embodiment, the first noise correction unit sets a noise correction intensity based on a correlation between a color component of a correction target pixel and a color component of a pixel adjacent to the correction target pixel. Also good. With such a configuration, excessive correction can be avoided, so that the image quality can be appropriately improved.

  In one embodiment, the image processing apparatus further includes a lightness and saturation correction unit that corrects the lightness or saturation of the image after being corrected by the first noise correction unit, and the first color covering correction unit includes the lightness and saturation. You may correct | amend the color coverage of the image after correct | amending by the correction | amendment part. In this way, by correcting the color coverage after correcting the lightness and saturation, it is possible to avoid erroneously determining that the pixel in the bright gradation portion is a color-covered pixel. It is possible to improve image quality.

  Alternatively, in one embodiment, when the food determination unit determines that the subject does not contain food, the image processing apparatus calculates the lightness or saturation of the image corrected by the first noise correction unit. You may further provide the lightness saturation correction part which correct | amends, and the 2nd color fog correction part which correct | amends the color cast of the image after being corrected by the lightness saturation correction part. If the color of the food is corrected after the lightness / saturation correction, for example, the brightness may be overemphasized. For this reason, no lightness saturation correction is required for food, and lightness saturation correction is performed on subjects other than food, thereby improving the image quality of food images and improving the image quality of non-food images appropriately. be able to. At this time, in the case of an image other than food, instead of using the result of color covering correction for the purpose of improving the accuracy of food determination, the color covering is performed after lightness saturation correction by using the result of noise correction. Since the correction can be performed, it is possible to avoid erroneously determining that the pixel of the bright gradation portion is a color-covered pixel.

  In one embodiment, the lightness / saturation correction unit may correct the lightness based on a tone curve using a histogram in the color space of the image. In one embodiment, the lightness / saturation correction unit may perform saturation correction using the lightness correction amount as a coefficient. In one embodiment, the lightness saturation correction unit may change the intensity of saturation correction based on a difference from a predetermined reference value. By correcting in this way, brightness or saturation can be corrected automatically and appropriately.

  In one embodiment, the image processing apparatus may further include an edge correction unit that corrects an edge of the image after being corrected by the food correction unit. When the edge is corrected, an image with increased roughness is obtained, and the color may not be corrected appropriately. For this reason, it is possible to correct the edge while adjusting the color by performing edge correction after adjusting the overall color. Therefore, a well-balanced correction can be performed.

  In one embodiment, the edge correction unit may set the edge strength based on the difference between the luminance value of the pixel to be corrected and the luminance value of a pixel included in a predetermined area including the pixel to be corrected. . In one embodiment, the image processing apparatus further includes a second noise correction unit that corrects noise of the image after correction by the food correction unit, and the edge correction unit is corrected by the second noise correction unit. You may correct | amend the edge of a subsequent image. By correcting in this way, the edge can be corrected automatically and appropriately.

  An image processing method according to another aspect of the present invention is a method for correcting an image. The method includes an input step, a first color covering correction step, a food determination step, and a food correction step. In the input step, an image is input. The first color covering correction step corrects the color covering of the image. The food determination step determines whether food is included in the subject of the image after being corrected by the first color covering correction step. In the food correction step, when it is determined in the food determination step that the subject contains food, the color component of the image corrected by the first color covering correction step is corrected.

  An image processing program according to another aspect of the present invention is a program that causes a computer to operate so as to correct an image. The program causes the computer to operate as an input unit, a first color covering correction unit, a food determination unit, and a food correction unit. The input unit inputs an image. The first color fog correction unit corrects the color fog of the image. The food determination unit determines whether food is included in the subject of the image after being corrected by the first color covering correction unit. The food correction unit corrects the color components of the image corrected by the first color covering correction unit when the food determination unit determines that the subject contains food.

  Furthermore, a recording medium according to another aspect of the present invention is a computer-readable recording medium on which an image processing program that causes a computer to operate so as to correct an image is recorded. The program causes the computer to operate as an input unit, a first color covering correction unit, a food determination unit, and a food correction unit. The input unit inputs an image. The first color fog correction unit corrects the color fog of the image. The food determination unit determines whether food is included in the subject of the image after being corrected by the first color covering correction unit. The food correction unit corrects the color components of the image corrected by the first color covering correction unit when the food determination unit determines that the subject contains food.

  According to the image processing method, the image processing program, and the recording medium on which the image processing program according to the present invention is recorded, the same effects as those of the image processing apparatus according to the present invention described above can be obtained.

  As described above, according to various aspects and embodiments of the present invention, image quality can be appropriately improved.

It is a functional block diagram of the portable terminal carrying the image processing apparatus which concerns on embodiment. It is a hardware block diagram of the portable terminal in which the image processing apparatus in FIG. 1 is mounted. It is a tone curve used for lightness correction processing. It is a graph explaining the correction intensity | strength of saturation. It is a graph explaining correction | amendment of edge strength. 4 is a flowchart for explaining the operation of the image processing apparatus according to the first embodiment. It is a flowchart explaining operation | movement of the image processing apparatus which concerns on 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted.

(First embodiment)
The image processing apparatus according to the present embodiment is an apparatus that corrects an image, and is mounted on, for example, a mobile phone, a digital camera, a PDA (Personal Digital Assistant), or a normal computer system. In the following, an image processing apparatus mounted on a portable terminal will be described as an example of the image processing apparatus according to the present invention in consideration of ease of understanding.

  FIG. 1 is a functional block diagram of a mobile terminal 2 including an image processing apparatus 1 according to the present embodiment. A mobile terminal 2 shown in FIG. 1 is a mobile terminal carried by a user, for example, and has a hardware configuration shown in FIG. FIG. 2 is a hardware configuration diagram of the mobile terminal 2. As shown in FIG. 2, the portable terminal 2 physically includes a main storage device such as a CPU (Central Processing Unit) 100, a ROM (Read Only Memory) 101, and a RAM (Random Access Memory) 102, a camera, a keyboard, and the like. The input device 103, the output device 104 such as a display, the auxiliary storage device 105 such as a hard disk, and the like are configured as a normal computer system. Each function of the portable terminal 2 and the image processing apparatus 1 to be described later causes the input device 103 and the output device 104 to be controlled under the control of the CPU 100 by reading predetermined computer software on hardware such as the CPU 100, the ROM 101, and the RAM 102. This is realized by operating and reading and writing data in the main storage device and the auxiliary storage device 105. Although the above description has been given as a hardware configuration of the mobile terminal 2, the image processing apparatus 1 normally includes a CPU 100, a main storage device such as the ROM 101 and the RAM 102, an input device 103, an output device 104, an auxiliary storage device 105, and the like. It may be configured as a computer system. The mobile terminal 2 may include a communication module or the like.

  As shown in FIG. 1, the mobile terminal 2 includes a camera 20, an image recording unit 21, an image processing device 1, a temporary image recording unit 22, a display unit 23, and a corrected image recording unit 24.

  The camera 20 has a function of capturing an image. For example, an image sensor or the like is used as the camera 20. For example, the camera 20 has a function of outputting a captured image to the image processing apparatus 1 every time it is captured.

  The image recording unit 21 has a function of recording an image. The image stored in the image recording unit 21 may be an image captured by the camera 20 or an image acquired via communication such as the Internet.

  The image processing apparatus 1 includes an input unit 10 and an image processing unit 11. The input unit 10 inputs an image to be corrected from the camera 20 or the image recording unit 21. The input image is, for example, an image expressed in an RGB color space. Then, the input unit 10 outputs an image to the image processing unit 11.

  The image processing unit 11 includes a brightness / saturation correction unit (brightness / saturation correction unit) 12, a color fog correction unit (first color fog correction unit) 13, a food determination unit 14, a food correction unit 15, an edge correction unit 16, and A noise correction unit (first noise correction unit) 17 is provided. The image processor 11 automatically and appropriately corrects the image input by the input unit 10 by appropriately operating these correction units and determination units and causing them to function in combination. Hereinafter, details of each correction unit and determination unit will be described.

  The lightness / saturation correction unit 12 corrects the lightness or saturation of the pixels included in the image. The lightness / saturation correction unit 12 corrects the lightness or saturation to eliminate an exposure mistake that is in a state shifted from the appropriate exposure. The lightness / saturation correction unit 12 not only makes a noticeable shift from proper exposure such as overexposure or blackout (a part where the brightness level is extremely high or low), but also a slight shift such as dim / slightly bright. Eliminate and improve image quality. Further, the lightness / saturation correction unit 12 improves the image quality by bringing the blue of “sky” or the green of “grass” closer to the memory color that a person has stored as “sky” or “grass”. . The memory color is a color that is generally stored as an image by a person. In many cases, the memory color is a brighter color than the actual color. For example, in the case of “sky”, the actual color is light blue that is close to light blue, but the memory color is pure blue. As described above, since the actual color may be different from the color imaged by the person, the image quality can be improved by bringing the predetermined color closer to the memory color.

  The lightness / saturation correction unit 12 adjusts the lightness in order to eliminate an image exposure error. If the intensity of lightness correction is increased too much in order to brighten a dark image, overexposure and color noise will become conspicuous and the image quality may be degraded. For this reason, the lightness / saturation correction unit 12 performs lightness correction using a tone curve in order to eliminate an exposure error. By using the tone curve, brightness and contrast can be corrected smoothly. The tone curve is generated based on a cumulative histogram of color components in the color space. In other words, a cumulative histogram of color components before correction is created, and a tone curve is generated that moves the center of the cumulative histogram to the center so that it is not overexposure and blackout. Further, the lightness / saturation correction unit 12 handles the image in the HSV color space. Since the hue of the HSV color space is independent, the brightness and the saturation can be corrected without changing the hue by adopting the HSV color space. Therefore, when the image before processing is an RGB color space or the like, the lightness / saturation correction unit 12 converts the color space of the image into an HSV color space.

  FIG. 3 is an example of a tone curve generated by the lightness / saturation correction unit 12. FIG. 3A shows a tone curve for correcting an overexposed image. FIG. 3B is a tone curve for correcting a blackout image. 3A and 3B, the horizontal axis is the input brightness value, and the vertical axis is the output brightness value (corrected brightness value), which is in the range of 0-255. The straight line indicated by L1 in the figure is not corrected, and the curve indicated by L2 in the figure is the tone curve. As shown in FIG. 3A, in the case of an overexposed image, a tone curve is generated so as to lower the brightness based on the HVS cumulative histogram. On the other hand, as shown in FIG. 3B, in the case of a blackout image, a tone curve is generated so as to increase the brightness based on the HVS cumulative histogram.

  Note that when the lightness is corrected using the tone curve, if the lightness is lowered, the saturation may appear low for human senses. For this reason, the lightness / saturation correction unit 12 increases the saturation when the lightness is lowered. Similarly, the lightness / saturation correction unit 12 decreases the saturation when the lightness is increased. At this time, the saturation is corrected using the change in brightness corrected by the tone curve as a coefficient. For example, the coefficient is set larger as the amount of change in brightness is larger. For example, when the saturation is increased by a predetermined value, the saturation is increased by multiplying or adding the predetermined value to the coefficient. Similarly, when the saturation is lowered by a predetermined value, the saturation is lowered by multiplying or adding the predetermined value to the coefficient. In this way, by adjusting the amount of increase and decrease in saturation according to the amount of change in lightness, it is possible to appropriately eliminate an exposure error and correct the image with good image quality.

  Further, the lightness / saturation correction unit 12 additionally performs saturation correction so that colors such as “sky”, “grass”, and “human skin” approach the memory color in addition to saturation correction according to lightness. May be performed. For example, in the case of “sky” blue, the higher the blue saturation, the closer to the memory color. However, if the saturation is emphasized too much in order to emphasize the “sky” blue, color unevenness may occur. For this reason, the lightness / saturation correction unit 12 enhances the saturation to such an extent that no color unevenness occurs. FIG. 4 is a graph for explaining the saturation correction strength. The horizontal axis is the hue value, and the vertical axis is the saturation correction intensity. The reference value D1 is a memory color. As shown in FIG. 4, the lightness / saturation correction unit 12 changes the intensity of saturation correction according to the distance in the hue space from the reference value D1 (or a reference region including the reference value D1). For example, the saturation correction strength is increased as the pixel has a hue value closer to the reference value D1, and the saturation correction strength is decreased as the pixel has a hue value farther from the reference value D1. By correcting the saturation in this way, it is possible to avoid blue color unevenness and reproduce blue close to the memory color. Furthermore, since the bluish color may become too strong in the sky with vivid colors, correction by a gamma curve may be performed.

  As described above, the lightness / saturation correction unit 12 performs saturation correction according to lightness for each pixel, and additionally distinguishes a region for each predetermined color to further enhance the saturation. Note that the predetermined color area can be easily determined by generating a map image of a predetermined color.

  The color fog correction unit 13 corrects the color fog of the image. The color covering is a state in which the image is biased to a specific color due to the influence of illumination or the environment. The color fog correction unit 13 adjusts the white balance and corrects the image quality so as to reproduce the original color by adjusting, for example, the complementary colors of the entire image (R value, G value, B value). Improve. For example, the color fog correction unit 13 determines a pixel that is originally expected to be white in the image and estimates how much color fog is generated. The color fog correction unit 13 uses an image expressed in the YUV color space when determining color fog. For this reason, when the image before processing is an RGB color space or the like, the color fog correction unit 13 converts the color space of the image into a YUV color space. Then, for example, the color fog correction unit 13 determines that the color of a pixel whose luminance value Y is equal to or greater than a predetermined value and whose color difference value (UV component value) with respect to the luminance value Y is equal to or smaller than a predetermined value is originally white. judge. Then, the color fog correction unit 13 estimates the color fog degree from the difference from the color of the image. Then, the color covering correction unit 13 samples the pixel determined to be white and adjusts the white balance so that the color of the pixel becomes white and does not become monotone.

  The food determination unit 14 determines whether food is captured as the subject of the image. Here, “food” is edible by humans and animals, and is defined here by learning before image correction. That is, the food determination unit 14 acquires in advance an image feature of food through learning using a teacher image. Then, the food determination unit 14 determines whether or not the subject contains food by an object detection technique using image features. For example, the food determination unit 14 compares the image characteristics to determine whether the subject contains food.

  The food correction unit 15 changes the color components of the pixels included in the image. The food correction unit 15 improves the appearance of the food, for example, by changing brightness and saturation. For example, when the subject includes food, the food correction unit 15 performs correction to emphasize the brightness of each pixel at a uniform (constant) ratio. Further, the food correction unit 15 corrects the saturation of each pixel by weighting so that the saturation of the pixel of the warm hue is emphasized more than the saturation of the pixel of the non-warm hue. As a result, the food is a bright and reddish image as a whole. In addition, when the food classified into the cold color type | system | groups, such as a salad, is contained in a part of image, image quality may fall by making it reddish. For this reason, the food correction unit 15 may not perform saturation correction for regions that are not warm colors. In this way, by performing different saturation correction for each color region, it is possible to improve the appearance of food even when, for example, a wide variety of foods are imaged side by side. Note that the food correction unit 15 performs correction in, for example, the HSV color space.

  The edge correction unit 16 enhances the edge of the image. For example, the edge correction unit 16 appropriately corrects the blurring of the outline, etc., by performing correction by adaptively changing the intensity of the edge with respect to a general unsharp mask, for example. Improve image resolution. The edge correction unit 16 performs correction in, for example, a YUV color space. For example, the edge correction unit 16 sets the edge strength based on the difference between the luminance value of the correction target pixel and the luminance value of the pixel included in the predetermined area including the correction target pixel. For example, a pixel to be corrected and eight neighborhoods surrounding the pixel are set as a predetermined region. If the pixel value of the correction target pixel is a and the average pixel value in the vicinity of 8 is b, the contrast cont of the correction target pixel is cont = (ab) · (ab). In proportion to the value of cont, the edge strength is varied from a predetermined minimum strength to a maximum strength. FIG. 5 is a graph illustrating correction of edge strength. The horizontal axis is the luminance difference, and the vertical axis is the edge strength. K1 in the figure is the minimum value of the edge strength (that is, edge correction amount), and K2 in the figure is the maximum value of the edge strength (that is, edge correction amount). Thus, when the luminance difference is smaller than S1, the edge strength is set to the minimum value K1, and when the luminance difference is larger than S1, the edge strength is set to the maximum value K2. Here, in the range where the luminance difference is not less than S1 and not more than S2, the edge intensity is linearly changed. In this way, the brightness difference is calculated in units of pixels, and the edges are set and corrected based on the graph shown in FIG. 5, so that appropriate edges are set in units of pixels. By appropriately correcting, it is possible to improve the resolution of the image.

  The noise correction unit 17 removes color noise and luminance noise. The noise correction unit 17 determines the correction strength using, for example, the correlation of neighboring pixels. Note that the noise correction unit 17 performs correction in, for example, an RBG color space. For example, the noise correction unit 17 sets the intensity of noise correction using the correlation between the R value of a predetermined pixel and the G value of one of the pixels adjacent to the pixel. For example, if the correlation is high, the correction strength is set to be weak, and if the correlation is low, the correction strength is set to be high. And the average value of the whole is calculated. The kernel size of the noise filter is changed from the set minimum value to the maximum value so as to be proportional to the average value. Thus, by removing noise according to the noise level, it is possible to avoid excessive noise removal and obtain an image with an appropriate detail feeling.

  Furthermore, the noise correction unit 17 expands the bits of the result data. The noise correction unit 17 expands the bits of the image after removing noise, for example. For example, the noise correction unit 17 extends the pixel data from 8 bits to 16 bits. Thereby, it is possible to reduce the quantization error in the processing after the noise correction.

  The image processing unit 11 is configured to be able to write and refer to the temporary image recording unit 22. The correction unit and the determination unit may record the image being processed in the temporary image recording unit 22. Then, the image processing unit 11 may output the corrected image to the display unit 23. The display unit 23 is a display device such as a display. Alternatively, the image processing unit 11 may output and record the corrected image to the corrected image recording unit 24.

  Next, the operation of the image processing apparatus 1 will be described. FIG. 6 is a flowchart showing the operation of the image processing apparatus 1. The control process shown in FIG. 6 is executed, for example, when the start button is turned on by a user operation.

  As shown in FIG. 6, first, image input processing is started (S10: input step). In the process of S10, the input unit 10 inputs an image. When the process of S10 ends, the process proceeds to a noise level determination process (S12).

  In the process of S12, the noise correcting unit 17 determines the noise level of the image input in the process of S10. The noise correction unit 17 determines the correction strength using, for example, the correlation of neighboring pixels. When the process of S12 ends, the process proceeds to a noise correction process (S14).

  In the process of S14, the noise correction unit 17 corrects the noise of the image input in the process of S10. For example, the noise correction unit 17 corrects the image of the image input in the process of S10 based on the noise intensity determined in the process of S12. Furthermore, the noise correction unit 17 extends the pixel data from 8 bits to 16 bits. When the processing of S14 is completed, the routine proceeds to lightness / saturation correction processing (S16).

  In the process of S16, the lightness / saturation correction unit 12 corrects the lightness and saturation of the image subjected to noise correction in the process of S14. For example, the lightness / saturation correction unit 12 corrects the lightness using the tone curve, and corrects the saturation while adjusting the increase and decrease widths according to the amount of change in lightness. Further, saturation correction is additionally performed so that the specific color approaches the memory color. When the process of S16 ends, the process proceeds to a color covering degree determination process (S18).

  In the process of S18, the color covering correction unit 13 determines the degree of color covering of the image that has been subjected to the lightness / saturation correction in the process of S16. For example, the color fog correction unit 13 determines a pixel that is originally expected to be white in the image that has been subjected to lightness / saturation correction in the process of S16, and estimates how much color fog occurs. When the process of S18 is completed, the process proceeds to a color fog correction process (S20: first color fog correction step).

  In the process of S20, the color covering correction unit 13 corrects the color covering of the image that has been lightness / saturated in the process of S16, based on the color covering degree determined in the process of S18. The color covering correction unit 13 samples a pixel determined to be white and adjusts the white balance so that the color of the pixel becomes white and does not become monotone. When the process of S20 ends, the process proceeds to a food determination process (S22: food determination step).

  In the process of S22, the food determination unit 14 determines whether or not the subject includes food using the image subjected to the color covering correction in the process of S20. The food determination unit 14 determines whether food is included in the subject by an object detection technique using image features. If it is determined in step S22 that the subject contains food, the process proceeds to food correction processing (S24: food correction step).

  In the process of S24, the food correction unit 15 corrects the image subjected to the color covering correction in the process of S20. The food correction unit 15 performs correction to emphasize the lightness of each pixel of the image subjected to color covering correction in the process of S20 at a uniform (constant) ratio, and the saturation of pixels of a warm color hue is a non-warm color system. The saturation of each pixel is corrected by weighting so that it is more emphasized than the saturation of the pixel of the hue. When the processing of S24 is completed, the process proceeds to edge strength determination processing (S26).

  In the process of S26, the edge correction unit 16 determines the edge strength of the image subjected to the food correction in the process of S24. For example, the edge correction unit 16 sets the edge strength based on the difference between the luminance value of the pixel of the image that has been food-corrected in the process of S24 and the luminance value of the eight neighboring pixels of the pixel. When the processing of S26 ends, the process proceeds to edge strength correction processing (S28).

  In the process of S28, the edge correction unit 16 corrects the image subjected to the food correction in the process of S24, based on the edge strength calculated in the process of S26. When the process of S28 ends, the process proceeds to a corrected image output process (S30).

  In the process of S30, the image processing unit 11 outputs the image whose edge has been corrected in the process of S28. For example, the image processing unit 11 records the image whose edge has been corrected in the process of S28 in the corrected image recording unit 24. When the process of S30 ends, the control process shown in FIG. 6 ends.

  On the other hand, if it is determined in step S22 that the subject does not contain food, the process proceeds to edge strength determination processing (S26).

  Thus, the control process shown in FIG. 6 is finished. By executing the control process shown in FIG. 6, it is possible to automatically obtain an image with appropriate correction. For example, if noise correction is performed after enhancing an edge, the enhanced edge may be crushed by noise correction. For this reason, as shown in FIG. 6, an appropriate correction can be performed by performing the edge correction process (S28) after the noise correction process (S14). When the data is 8 bits, so-called tone jump in which the output brightness changes rapidly with respect to the change in input brightness is likely to occur during brightness correction using the tone curve. For this reason, as shown in FIG. 6, after the noise correction process (S14) is expanded to 16 bits, the brightness / saturation correction process (S16) and the food correction process (S24) are performed, so that appropriate correction can be performed. Yes. Also, bright color gradations such as sky and petals are likely to be sampled as pixels that are accidentally colored. For this reason, as shown in FIG. 6, appropriate correction can be performed by performing color fog correction processing (S20) after lightness / saturation correction processing (S16). Also, food images are often covered with colors due to the effects of room lighting and the like. For this reason, as shown in FIG. 6, an appropriate determination can be performed by performing the food determination process (S22) after the color covering correction process (S20). Further, when the color correction is performed after the edge correction is performed, the brightness and the saturation are emphasized with respect to the image with the increased roughness, so that a tone jump and an increase in noise may be remarkable. For this reason, as shown in FIG. 6, the edge correction process (S28) is performed locally after the correction related to the entire color is completed, whereby appropriate correction can be performed.

  Next, an image processing program for causing the portable terminal 2 (computer) to function as the image processing apparatus 1 will be described.

  The image processing program includes a main module, an input module, and an arithmetic processing module. The main module is a part that comprehensively controls image processing. The input module operates the mobile terminal 2 so as to acquire an input image. The arithmetic processing module includes a lightness / saturation correction module, a color fog correction module, a food determination module, a food correction module, an edge correction module, and a noise correction module. Functions realized by executing the main module, the input module, and the arithmetic processing module are the lightness / saturation correction unit 12, the color fog correction unit 13, the food determination unit 14, and the food correction unit 15 of the image processing apparatus 1 described above. The functions of the edge correction unit 16 and the noise correction unit 17 are the same.

  The image processing program is provided by a storage medium such as a ROM or a semiconductor memory, for example. The image processing program may be provided as a data signal via a network.

  As described above, according to the image processing apparatus 1 according to the first embodiment, the color covering correction unit 13 corrects the color covering of the image, and the food determination unit 14 includes food in the image subject after the color covering correction. When it is determined that the subject contains food, the food correction unit 15 corrects the color component of the image after color covering correction. In this way, when improving the image quality related to color covering and the image quality related to food at the same time, it is necessary to determine whether or not the subject contains food. Therefore, the accuracy of food determination can be improved by performing color coverage correction before the determination. Therefore, it is possible to improve the image quality as compared with the case where the color covering correction is performed after the food correction.

(Second Embodiment)
The image processing apparatus 1 according to the second embodiment is configured in the same manner as the image processing apparatus according to the first embodiment. That is, as shown in FIG. 1, the image processing unit 11 includes a lightness / saturation correction unit (lightness / saturation correction unit) 12, a color fog correction unit (first color fog correction unit, second color fog correction unit) 13. , Food determination unit 14, food correction unit 15, edge correction unit 16, and noise correction unit (first noise correction unit, second noise correction unit) 17. In the second embodiment, only the operation order of each correction unit and determination unit is different. For this reason, below, it demonstrates centering around difference with 1st Embodiment, and the overlapping description is abbreviate | omitted.

  FIG. 7 is a flowchart showing the operation of the image processing apparatus 1. The control process shown in FIG. 7 is executed, for example, when the start button is turned on by a user operation.

  As shown in FIG. 7, first, image input processing is started (S40). The process of S40 is the same as the process of S10 of FIG. When the process of S40 ends, the process proceeds to a noise level determination process (S42).

  In the process of S42, the noise correcting unit 17 determines the noise level of the image input in the process of S40. The process of S42 is the same as the process of S12 of FIG. When the process of S42 ends, the process proceeds to a noise correction process (S44).

  In the process of S44, the noise correction unit 17 corrects the noise of the image input in the process of S40. The process of S44 is the same as the process of S14 of FIG. When the process of S44 is completed, the process proceeds to the correction result recording process (S46).

  In the process of S46, the noise correction unit 17 records the image whose noise has been corrected in the process of S44 in the temporary image recording unit 22. When the process of S46 is completed, the process proceeds to a color covering degree determination process (S48).

  In the process of S48, the color fog correction unit 13 determines the degree of color fog of the image whose noise has been corrected in the process of S44. This process is the same as the process of S18 of FIG. When the processing of S48 is completed, the process proceeds to color fog correction processing (S50).

  In the process of S50, the color fog correction unit 13 corrects the color fog of the image whose noise has been corrected in the process of S44 based on the color fog degree determined in the process of S48. This process is the same as the process of S20 of FIG. When the process of S50 ends, the process proceeds to the food determination process (S52).

  In the process of S52, the food determination unit 14 determines whether or not the subject includes food using the image subjected to the color covering correction in the process of S50. The process of S52 is the same as the process of S22 of FIG. If it is determined in step S52 that the subject contains food, the process proceeds to food correction processing (S54).

  In the process of S54, the food correction unit 15 corrects the image subjected to the color covering correction in the process of S50. The process of S54 is the same as the process of S24 of FIG. When the processing of S54 is completed, the routine proceeds to noise level determination processing (S56).

  In the process of S56, the noise correcting unit 17 determines the noise level of the image that has been food-corrected in the process of S54. The process of S56 is the same as the process of S42. When the processing of S56 is completed, the routine proceeds to noise correction processing (S58).

  In the process of S58, the noise correction unit 17 corrects the noise of the image subjected to the food correction in the process of S54. The process of S58 is the same as the process of S44. When the process of S58 is completed, the process proceeds to an edge strength determination process (S60).

  In the process of S60, the edge correction unit 16 determines the edge strength of the image whose noise has been corrected in the process of S58. This process is the same as the process of S26 of FIG. When the processing of S60 is completed, the process proceeds to edge strength correction processing (S62).

  In the process of S62, the edge correction unit 16 corrects the image whose noise has been corrected in the process of S58, based on the edge intensity calculated in the process of S60. When the process of S62 ends, the process proceeds to a corrected image output process (S64).

  In the process of S64, the image processing unit 11 outputs the image whose edge has been corrected in the process of S62. When the process of S64 ends, the control process shown in FIG. 6 ends.

  On the other hand, if it is determined in step S52 that the subject does not contain food, the process proceeds to a correction result input process (S66). In the process of S66, the image processing unit 11 refers to the temporary image recording unit 22 and acquires a noise-corrected image. When the processing of S66 is completed, the routine proceeds to lightness / saturation correction processing (S68).

  In the process of S68, the lightness / saturation correction unit 12 corrects the lightness and saturation of the image input in the process of S66. This process is the same as the process of S16 of FIG. When the process of S68 ends, the process proceeds to a color covering degree determination process (S70).

  In the process of S70, the color fog correction unit 13 determines the degree of color fog of the image input in the process of S66. This process is the same as the process of S48. When the processing of S70 is completed, the process proceeds to color fog correction processing (S72).

  In the process of S72, the color fog correction unit 13 corrects the color fog of the image subjected to the noise correction in the process of S66 based on the color fog degree determined in the process of S70. This process is the same as the process of S50. When the process of S72 ends, the control process shown in FIG. 6 ends.

  The control process shown in FIG. By executing the control process shown in FIG. 7, an image with appropriate correction can be automatically obtained. For example, when the subject contains food, the brightness may be overemphasized due to an increase in contrast. For this reason, as shown in FIG. 7, when performing the food correction process (S52), the contrast of food is excessively emphasized by not performing the lightness / saturation correction process (S68). It can be avoided. Along with this, in order to perform color fog correction processing after lightness / saturation correction processing, in the case of an image other than food, color fog correction processing for the purpose of improving the accuracy of food determination is performed. Appropriate correction can be performed by discarding the result image, obtaining a noise-corrected image (S66), and sequentially performing lightness / saturation correction processing (S68) and color fog correction processing (S72). Furthermore, when the color is whitened by the color covering correction, the original noise may be conspicuous. Therefore, as shown in FIG. 7, appropriate correction can be performed by performing the noise correction process (S58) again after the color fog correction process (S50).

  As described above, according to the image processing apparatus 1 according to the second embodiment, the same effects as those of the first embodiment can be obtained, and the correction processing order can be changed to perform food correction processing or correction processing for subjects other than food. It is possible to appropriately perform and reduce noise.

  The above-described embodiment shows an example of the image processing apparatus, the image processing method, the image processing program, and the recording medium according to the present invention, and is not limited to the apparatus, the method, the program, and the recording medium according to the embodiment. However, it may be modified or applied to others.

  DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus, 10 ... Input part, 11 ... Image processing part, 12 ... Lightness / saturation correction part (lightness saturation correction part), 13 ... Color fog correction part (1st color fog correction part, 2nd color) (Covering correction unit), 14 ... food determination unit, 15 ... food correction unit, 16 ... edge correction unit, 17 ... noise correction unit (first noise correction unit, second noise correction unit).

Claims (19)

An image processing apparatus for correcting an image,
An input unit for inputting the image;
A first color covering correction unit for correcting the color covering of the image;
A food determination unit for determining whether food is included in the subject of the image after being corrected by the first color covering correction unit;
A food correction unit that corrects the color components of the image after correction by the first color covering correction unit when the food determination unit determines that the subject contains food;
An image processing apparatus comprising:   The food determination unit obtains food characteristics in advance by learning using a teacher image, the image characteristics after being corrected by the first color covering correction unit, and the food characteristics obtained by learning. The image processing apparatus according to claim 1, wherein the image processing apparatus determines whether food is included in the subject of the image after being corrected by the first color covering correction unit by comparing.   The image processing apparatus according to claim 1, wherein the food correction unit emphasizes the brightness of each pixel at a certain ratio.   The food correction unit corrects the saturation of each pixel by weighting so that the saturation of a pixel having a warm hue is emphasized more than the saturation of a pixel having a non-warm hue. The image processing apparatus according to any one of the above.   The first color covering correction unit determines that the color of a pixel whose luminance value is equal to or greater than a predetermined value and whose ratio of the color difference value to the luminance value is equal to or smaller than the predetermined value is white, and determines the degree of color covering The image processing device according to claim 1, wherein the image processing device is estimated. A first noise correction unit that corrects noise of the image input by the input unit;
The image processing apparatus according to claim 1, wherein the first color fog correction unit corrects the color fog of the image after being corrected by the first noise correction unit.   The image processing apparatus according to claim 6, wherein the first noise correction unit expands a bit of each pixel after correcting the noise of the image input by the input unit.   The first noise correction unit sets a noise correction intensity based on a correlation between a color component of a correction target pixel and a color component of a pixel adjacent to the correction target pixel. An image processing apparatus according to 1. A lightness / saturation correction unit that corrects the lightness or saturation of the image after being corrected by the first noise correction unit;
The image processing apparatus according to claim 6, wherein the first color covering correction unit corrects the color covering of the image after being corrected by the lightness / saturation correcting unit. A brightness saturation correction unit that corrects the brightness or saturation of the image after being corrected by the first noise correction unit when the food determination unit determines that the subject does not contain food;
A second color fog correction unit that corrects the color fog of the image after being corrected by the lightness saturation correction unit;
The image processing apparatus according to any one of claims 6 to 8, further comprising:   The image processing apparatus according to claim 9, wherein the lightness saturation correction unit corrects the lightness based on a tone curve using a histogram in a color space of the image.   The image processing apparatus according to claim 11, wherein the lightness saturation correction unit performs saturation correction using a lightness correction amount as a coefficient.   The image processing apparatus according to claim 9, wherein the lightness saturation correction unit changes the intensity of saturation correction based on a difference from a predetermined reference value.   The image processing apparatus according to claim 1, further comprising an edge correction unit that corrects an edge of the image after being corrected by the food correction unit.   The image according to claim 14, wherein the edge correction unit sets the edge strength based on a difference between a luminance value of a pixel to be corrected and a luminance value of a pixel included in a predetermined area including the correction target pixel. Processing equipment. A second noise correction unit for correcting noise of the image after correction by the food correction unit;
The image processing apparatus according to claim 14, wherein the edge correction unit corrects an edge of the image after being corrected by the second noise correction unit. An image processing method for correcting an image,
An input step for inputting the image;
A first color covering correction step for correcting the color covering of the image;
A food determination step of determining whether food is included in the subject of the image after being corrected by the first color covering correction step;
A food correction step of correcting the color component of the image after being corrected by the first color covering correction step when the food determination step determines that the subject contains food;
An image processing method comprising: An image processing program for operating a computer to correct an image,
An input unit for inputting the image;
A first color covering correction unit for correcting the color covering of the image;
A food determination unit for determining whether food is included in the subject of the image after being corrected by the first color covering correction unit; and
When the food determination unit determines that the subject contains food, the computer is operated as a food correction unit that corrects the color component of the image after correction by the first color covering correction unit. An image processing program to be executed. A computer-readable recording medium on which an image processing program for operating a computer to correct an image is recorded,
An input unit for inputting the image;
A first color covering correction unit for correcting the color covering of the image;
A food determination unit for determining whether food is included in the subject of the image after being corrected by the first color covering correction unit; and
When the food determination unit determines that the subject contains food, the computer is operated as a food correction unit that corrects the color component of the image after correction by the first color covering correction unit. A recording medium on which an image processing program to be recorded is recorded.