JP2016213596A - Color correction device, color correction method and color correction program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a color correction device capable of separating infrared light into a part of high spectral reflectance and a part of low spectral reflectance.SOLUTION: A color correction device 100 includes: an acquisition section that acquires an image; a conversion section that converts the image into color space components including brightness and chromaticity; and a determination section that determines, with respect to each of pixels included in the image converted into a component of color space, whether the pixel corresponds to a first subject which has a high spectral reflectance of infrared light or to another subject other than the first subject depending on whether or not the pixel satisfies a separation conditions relevant to the chromaticity component and the brightness component.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a color correction apparatus, a color correction method, and a color correction program.

  Cameras are used in a wide variety of applications. For example, the camera may be implemented as a surveillance camera or an in-vehicle camera, or may be implemented as an image capturing device used for various types of biometric authentication or gaze detection using corneal reflection.

  For example, in the case of a surveillance camera or an in-vehicle camera, it may be used both day and night. In this case, by installing an infrared cut filter that removes infrared light in the daytime, visible light that can be recognized by the human eye is imaged on the light receiving element, while the infrared cut filter is removed at night and in the dark. As a result, infrared light such as near infrared may be used as illumination. Further, in the case of an information processing terminal equipped with functions such as biometric authentication and line-of-sight detection, an image used for biometric authentication and line-of-sight detection and an image used for other purposes are captured by one camera, thereby There are also trends aimed at reducing scale and costs.

  When the imaging device receives visible light and infrared light in this way, the image captured by the camera is different from the color that a person recognizes with eyes under visible light. From this, by correcting the color of an image captured by an image sensor that receives visible light and infrared light, visible light is received, in other words, the same color as when infrared light is not received. Techniques for reproducing the taste have been proposed.

  As an example, there is a method of correcting an input image by matrix calculation. In this method, matrix coefficients used for color correction are calculated from two images taken by different image input systems of a plurality of colors, for example, a color chart including 24 colors. That is, from two images of an image taken with an infrared cut filter incorporated on the optical path and an image taken with no infrared cut filter incorporated on the optical path, A matrix coefficient that approximates the spectral sensitivity characteristic to the color matching function is obtained by the method of least squares. The color correction of the input image is performed by the calculation using the matrix coefficient.

  However, when color correction is performed by the above matrix calculation, color reproducibility can be sufficiently exerted on a specific subject having a high spectral reflectance of infrared light, such as a plant leaf or a chemical fiber cloth. There are cases where it is not possible. For example, in the case of plant leaves, the spectral reflectance increases at a wavelength corresponding to near-infrared light exceeding the upper limit of visible light due to the influence of chlorophyll contained in the leaves. For this reason, the influence of the color which infrared light included in environmental light such as natural light or illumination has on the image is greater than that of other subjects other than the leaves of plants. Therefore, even if color correction is performed by the above matrix calculation, the leaf color of the plant cannot be sufficiently reproduced as compared with other subjects. As a result, the color of the leaf, which is recognized as green under visible light, is recognized as a color different from the green color of the color chart that is also recognized as green with visible light.

  In order to suppress such a decrease in color reproducibility, the following correction data generation apparatus has also been proposed. In the correction data generation device, the first sample image in which the specific subject is imaged without the infrared cut filter and the first sample image in which the specific subject is imaged with the infrared cut filter attached. Correction data for correcting the color of the specific subject is generated using the two sample images.

  In such a correction data generation device, the knowledge that colors included in a specific subject are concentrated in a local region close to an achromatic color in the chromaticity space is used. Based on such knowledge, the correction data generation device allows an ellipse region in which the color included in the specific subject is distributed in the chromaticity space by designating a range in which the specific subject appears on the first sample image. Set. Then, the correction data generation device calculates, as a correction amount, a shift amount that shifts the elliptical region set using the first sample image to the elliptical region in which the color of a specific subject included in the second sample image is distributed. To do. In addition, the spectral reflectance of infrared light is high depending on whether or not the color of the pixel included in the input image is included in the elliptical region on the chromaticity space set using the first sample image. Separated into parts and other parts. That is, correction for shifting the color using the above correction data is performed on the portion where the spectral reflectance of infrared light is high, while color correction is performed on the other portions by the above matrix calculation. .

JP 2005-303702 A JP 2008-288851 A JP 2011-015086 A

Yoshitaka Toyoda, Tetsuya Kuno, Shinzo Ishida, Hiroaki Sugiura “Study on Near-Infrared Imaging Device Capable of Color Imaging”, Journal of the Institute of Image Information and Television Engineers, Vol. 64, no. 1, pp. 101-110 2010

  However, with the above technique, there may be a case where a portion having a high spectral reflectance of infrared light and a portion other than that cannot be sufficiently separated.

  In other words, the elliptical region set in the first sample image by the correction data generation device described above has a color other than a specific object such as a plant leaf or a chemical fiber cloth having a high spectral reflectance of infrared light. However, as an example, colors such as brown included in the color chart are also included. Therefore, in the above correction data generation device, the color of the specific subject and the brown color chart cannot be separated, and the color chart corresponding to the brown color is corrected to green.

  In one aspect, an object of the present invention is to provide a color correction device, a color correction method, and a color correction program that can separate a portion having a high spectral reflectance of infrared light from other portions.

  A color correction apparatus according to an aspect includes an acquisition unit that acquires an image, a conversion unit that converts the image into a component of a color space including luminance and chromaticity, and each pixel included in the image converted into the component of the color space In addition, depending on whether or not the pixel satisfies the separation condition regarding the chromaticity component and the luminance component, either the first subject having a high spectral reflectance of infrared light or the other subject other than the first subject is selected. And a determination unit for determining whether or not it corresponds.

  A portion having a high spectral reflectance of infrared light can be separated from other portions.

FIG. 1 is a block diagram illustrating a functional configuration of each device included in the color correction system according to the first embodiment. FIG. 2A is a diagram illustrating an example of a sample image. FIG. 2B is a diagram illustrating an example of a sample image. FIG. 3 is a diagram illustrating an example of a color distribution on the UV plane. FIG. 4 is a diagram illustrating an example of a color distribution on the YU plane. FIG. 5 is a diagram illustrating an example of a histogram of luminance Y. FIG. 6 is a flowchart illustrating the procedure of the setting process according to the first embodiment. FIG. 7 is a flowchart illustrating the procedure of the color correction process according to the first embodiment. FIG. 8A is a diagram illustrating an example of a histogram of the color component U. FIG. FIG. 8B is a diagram illustrating an example of a histogram of the color component V. FIG. 9 is a diagram illustrating a calculation example of the probability density regarding the chromaticity space. FIG. 10 is a diagram illustrating an example of the third separation condition. FIG. 11 is a diagram illustrating a hardware configuration example of a computer that executes a color correction program according to the first and second embodiments.

  A color correction device, a color correction method, and a color correction program according to the present application will be described below with reference to the accompanying drawings. Note that this embodiment does not limit the disclosed technology. Each embodiment can be appropriately combined within a range in which processing contents are not contradictory.

[Configuration of color correction system]
FIG. 1 is a block diagram illustrating a functional configuration of each device included in the color correction system according to the first embodiment. A color correction system 1 shown in FIG. 1 realizes a color correction process for correcting the color of an image captured including light corresponding to wavelengths of visible light and infrared light in accordance with human color vision characteristics. . Here, as an example, a case where the image is taken without a filter that removes infrared rays, that is, a so-called infrared cut filter, is exemplified. However, as described later, one of the wavelengths corresponding to infrared light is used. Needless to say, the scope of application includes a case where an image photographed in a state where an optical filter that transmits light in a certain wavelength region is attached is input.

  As part of such color correction processing, the color distribution of the first subject in the color space and the first subject having a high spectral reflectance of infrared light included in the image photographed without the infrared cut filter attached. Separation processing for separating a second subject with similar brightness depending on whether the luminance is high is also realized. That is, the brightness corresponding to the first subject tends to be high because the amount of reflected light itself is larger than that of the second subject. Based on such knowledge, the second subject whose color distribution is similar to that of the first subject in the color space is similar to the location of the first subject represented by the leaves of a plant having high spectral reflectance of infrared light. An object, for example, a brown portion of a color chart is separated according to a luminance separation condition. As a result, it is possible to sufficiently separate the portion where the spectral reflectance of infrared light is high and the other portions.

  As shown in FIG. 1, the color correction system 1 includes a setting device 10 and a color correction device 100. The setting device 10 and the color correction device 100 are connected to be communicable via a predetermined network. As an example, any type of communication network such as the Internet (Internet), LAN (Local Area Network), and VPN (Virtual Private Network) can be adopted as such a network, regardless of whether it is wired or wireless.

  The setting device 10 is a device that sets various parameters relating to color correction.

  As an embodiment, the setting device 10 sets correction data related to color correction to the color correction device 100 and sets separation conditions for separating the first subject from other subjects.

  Although FIG. 1 shows an example in which various parameters relating to color correction are set through the setting device 10, they may be set in advance in the color correction device 100 before the color correction device 100 is shipped. Further, the setting device 10 and the color correction device 100 do not necessarily need to be constructed separately, and the color correction device 100 can also have the function of the setting device 10.

  The color correction apparatus 100 is a computer that realizes the color correction process.

  As one embodiment, the color correction apparatus 100 can be implemented by installing a color correction program for executing the above color correction processing as package software or online software on a desired computer. For example, the information processing apparatus can function as the color correction apparatus 100 by causing the information processing apparatus to execute the color correction program. The information processing apparatus mentioned here includes a desktop or notebook personal computer, a mobile communication terminal such as a smartphone, a mobile phone or a personal handyphone system (PHS), or a slate such as a PDA (Personal Digital Assistants). Terminals are included in the category. Further, the terminal device used by the user can be used as a client, and the client can be implemented as a server device that provides the client with services related to the color correction processing. For example, the color correction apparatus 100 provides a color correction service that receives an image or identification information that can call the image via a network or a storage medium, and outputs the execution result of the color correction processing for the image. It is implemented as a server device. In this case, the color correction apparatus 100 may be implemented as a Web server, or may be implemented as a cloud that provides a service related to the color correction processing described above by outsourcing.

[Configuration of Setting Device 10]
Subsequently, a functional configuration of the setting apparatus 10 according to the present embodiment will be described. As illustrated in FIG. 1, the setting device 10 includes a first acquisition unit 11, a second acquisition unit 12, a specification unit 13, a conversion unit 14, a correction data setting unit 15, and a separation condition setting unit 16. Have. Note that the setting device 10 may include various functional units included in known computers, for example, functional units such as various input devices and audio output devices, in addition to the functional units illustrated in FIG.

  Among these, the first acquisition unit 11 and the second acquisition unit 12 are both processing units that acquire sample images as materials for generating various parameters relating to color correction. Hereinafter, the sample image acquired by the first acquisition unit 11 is referred to as a “first sample image”, and the sample image acquired by the second acquisition unit 12 is referred to as a “second sample image”. May be described.

  As one embodiment, the first acquisition unit 11 and the second acquisition unit 12 acquire a sample image captured by a camera equipped with an image sensor such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). Can do. The first acquisition unit 11 and the second acquisition unit 12 acquire the sample image by reading a sample image stored in an auxiliary storage device such as a hard disk or an optical disk or a removable medium such as a memory card or a USB (Universal Serial Bus) memory. You can also In addition, the 1st acquisition part 11 and the 2nd acquisition part 12 can also acquire a sample image by receiving via a network from an external device.

  2A and 2B are diagrams illustrating examples of sample images. Here, as an example, the following description will be made on the assumption that an image having three RGB light receiving elements is used to capture an image in which each pixel has RGB luminance values. Among these, the red component R receives the red wavelength range and the infrared wavelength range among the visible light wavelengths, and the green component G includes the green wavelength range and the infrared light among the visible light wavelengths. The blue component B receives the blue wavelength range and the infrared wavelength range among the visible light wavelengths.

  The first sample image 20A shown in FIG. 2A and the second sample image 20B shown in FIG. 2B are common in that the same target is photographed at the same position and the same angle of view by the same camera. However, a part of the image input system in which images of each other are taken is different. That is, the first sample image 20A shown in FIG. 2A is an image taken without an infrared cut filter, while the second sample image 20B shown in FIG. 2B has an infrared cut filter attached. The difference is that the image is taken in a state. Therefore, the first sample image 20A includes the influence of light having the wavelength of infrared light in addition to light having the wavelength of visible light, while the second sample image 20B includes infrared light. This does not include the influence of light having a wavelength of. In addition, although the case where it image | photographed in the state which is not mounting | wearing with an infrared cut filter was illustrated as an example of the imaging | photography form of a 1st sample image here, the bandwidth of the visible light region and the infrared wavelength region to be used, for example, An image photographed with an optical filter that transmits a part of the wavelength of near-infrared light being inserted may be used as the first sample image.

  When the first sample image 20A and the second sample image 20B are photographed, as an example, the photographing is performed in a state where the first subject is included in the imaging range. An example of such a first subject is a plant containing a chemical substance such as chlorophyll in the leaf. When the plant is photographed in this way, the shape of the plant image 21A and the plant image 21B is reflected in the same manner between the first sample image 20A and the second sample image 20B. Is different. That is, in the case of a plant, the spectral reflectance of infrared light becomes larger than other subjects due to the influence of chlorophyll contained in the leaves of the plant. Therefore, the plant image 21B appears green like the color recognized by the human eye, while the plant image 21A is grayish celadon or grayish pale green due to the influence of reflected light of infrared light. Reflected in.

  Further, in addition to the above-described plant, shooting is performed in a state where the second subject is included in the imaging range. An example of such a second subject is a color chart in which 24 different colors are arranged in 6 rows and 4 columns, as shown in FIGS. 2A and 2B. For example, the color chart may include a color having a color distribution similar to a plant in the color space, such as brown. Accordingly, it is possible to set a luminance separation condition for separating the portion corresponding to the plant and the brown portion of the color chart in the subsequent processing unit.

  Here, a plant is illustrated as an example of the first subject. However, a subject other than a plant is a subject as long as the spectral reflectance increases with a predetermined wavelength, for example, about 700 nm as a boundary. It doesn't matter. That is, any object, for example, a chemical fiber, can be used as long as the spectral reflectance rises more rapidly than the spectral reflectance in the wavelength range corresponding to visible light near the upper limit of visible light or in the near infrared wavelength range. It can be cloth.

  The specifying unit 13 specifies a first region that forms part or all of the first subject on each sample image and a second region that forms part or all of the second subject on each sample image. Is a processing unit.

  As one embodiment, the specifying unit 13 causes the display unit of the setting device 10 or an external device connected to the setting device 10 to display at least one sample image of the first sample image and the second sample image. . Then, the specifying unit 13 causes a first object, that is, a range including one or a plurality of pixels forming a plant to be designated on the sample image through an input unit (not shown). For example, the specifying unit 13 can specify the first region corresponding to the first subject on the sample image by receiving the drag and drop range specification via the pointing device. The specifying unit 13 specifies a second region that forms part or all of the second subject on the sample image in the same manner as in the case of specifying the first region. In addition, although the case where the first area and the second area are specified by an operation input is illustrated here, it does not necessarily have to be specified by an operation input. For example, by applying image recognition such as template matching to the sample image, the region similar in shape to the first subject among the regions labeled on the sample image is specified as the first region, An area having a shape similar to that of the second subject may be specified as the second area.

  The conversion unit 14 is a processing unit that converts the data format of the sample image.

  As an embodiment, the conversion unit 14 converts the data format of the first sample image acquired by the first acquisition unit 11 and the second sample image acquired by the second acquisition unit 12 into a space including luminance and chromaticity. Convert to the data format represented by. For example, the conversion unit 14 includes pixels represented by luminance Y and two color differences from the RGB pixel values represented in the RGB space for the pixel values of the pixels included in the first sample image and the second sample image. Convert to value. Here, the case where each sample image is converted to the YUV space is illustrated, but the chromaticity space of the conversion destination is not limited to the YUV space, and the sample image is converted to another color space, for example, the L * a * b space. It does not matter as a conversion.

  The correction data setting unit 15 is a processing unit that sets correction data related to color correction in the color correction apparatus 100.

  As one embodiment, the correction data setting unit 15 includes first correction data used for color correction related to the first subject and second correction used for color correction related to a subject other than the first subject. Calculate the data.

  Here, the first correction data can be calculated by using a method disclosed in Japanese Unexamined Patent Application Publication No. 2014-200045 as an example. In other words, the correction data setting unit 15 includes, in the chromaticity space, pixels included in the first region specified as a range for forming the first subject by the specifying unit 13 among the pixels included in the first sample image. The shift amount in the chromaticity space is used as the first correction data from the position where the pixel is distributed and the position where the pixel on the second sample image corresponding to the pixel included in the first region is distributed in the chromaticity space. calculate.

  More specifically, the correction data setting unit 15 defines a distribution of coordinates in which the color of each pixel included in the first region among the pixels included in the first sample image is scattered in the chromaticity space. . For example, the correction data setting unit 15 defines a distribution of coordinates in which the UV components of each pixel included in the first region are scattered on the UV plane by parameters such as the ellipse center point, the ellipse axis length, and the ellipse rotation angle. Thus, it can be approximated as an elliptical region on the UV plane. At this time, the specifying unit 13 defines the elliptical region by narrowing down to a location where the pixels forming the first region are concentrated and distributed on the UV plane of the chromaticity space in the first sample image. Pixels having a distance between them within a predetermined value can be included, and pixels exceeding a predetermined value can be excluded from the target. In this manner, an elliptical area representing a distribution in which the UV components of the pixels forming the first subject are scattered on the UV plane on the first sample image is calculated. Here, the case where the distribution is approximated to an elliptical area is illustrated, but it may be approximated to another shape, for example, a polygon.

  Thereafter, the correction data setting unit 15 determines, for each pixel forming the first region on the first sample image, a color difference between the pixel on the second sample image corresponding to the pixel, that is, on the UV plane. The distance is calculated. Then, the correction data setting unit 15 calculates a representative value by executing predetermined statistical processing on the color difference calculated for each pixel forming the first region. As an example of such statistical processing, an arithmetic average or a weighted average of color differences can be calculated as a representative value, or a median value of color differences can be calculated as a representative value.

  The representative value of the color difference calculated in this way, that is, the shift amount Sm on the UV plane is registered in the correction data storage unit 160 of the color correction apparatus 100 as the first correction data, whereby the first correction data is stored. The color correction apparatus 100 is set. Here, the case where the shift amount Sm on the UV plane is calculated is exemplified, but the shift amount on the ab plane in the L * a * b space may be calculated.

  Further, the second correction data can be calculated according to the method disclosed in Non-Patent Document 1 “Research of near-infrared imaging device capable of color imaging” as an example. That is, the correction data setting unit 15 uses the RGB pixel values of the pixels not included in the first region among the pixels included in the two sample images of the first sample image and the second sample image to calculate the image sensor. The matrix coefficient that approximates the spectral sensitivity characteristic of the image to the color matching function can be obtained by the method of least squares.

  The matrix coefficient calculated in this way is registered as the second correction data in the correction data storage unit 160 of the color correction apparatus 100, whereby the second correction data is set in the color correction apparatus 100.

  The separation condition setting unit 16 is a processing unit that sets a separation condition for separating the first subject and other subjects in the color correction apparatus 100.

  As an embodiment, the separation condition setting unit 16 separates the first subject and the second subject and subjects other than the first subject and the second subject on the UV plane as an example of the separation condition. Are used in two stages: a “first separation condition”, that is, a separation condition relating to chromaticity, and a “second separation condition” for separating the first subject and the second subject, ie, the above-described separation condition relating to luminance. Calculate separation conditions.

  For example, the separation condition setting unit 16 can set an elliptical area corresponding to the first area calculated by the correction data setting unit 15 in the color correction apparatus 100 as the first separation condition.

  FIG. 3 is a diagram illustrating an example of a color distribution on the UV plane. The vertical axis of the graph shown in FIG. 3 indicates the color difference V, and the horizontal axis indicates the color difference U. FIG. 3 shows a graph in which the distribution of pixels included in the first sample image and the second sample image is plotted on the UV plane. That is, in FIG. 3, the distribution of the pixels forming the first region among the pixels included in the first sample image is plotted with a circle mark, and the first of the pixels included in the second sample image is plotted. The distribution of the pixels forming the region is plotted with rhombus marks, and the distribution of the pixels corresponding to the second region among the pixels included in the first sample image is plotted with triangular marks.

  As shown in FIG. 3, the elliptical region 31 corresponding to the first region on the first sample image includes the color corresponding to the leaf of the plant, that is, the whole or most of the circle mark in the drawing. Therefore, by using the elliptical region 31 as the first separation condition, the first subject and other subjects can be separated with a certain degree of accuracy. On the other hand, the ellipse area 31 does not include only the color of the pixel forming the first area, but also includes the color of the pixel forming the second area. That is, the oval region 31 includes a color distribution corresponding to the leaf of the plant, while the color corresponding to the brown portion of the color chart whose distribution is similar to the color of the plant leaf on the UV plane, that is, in the drawing. The distribution of triangular marks is also included.

  Therefore, the separation condition setting unit 16 further calculates a second separation condition for separating the first subject and the second subject in addition to the first separation condition.

  FIG. 4 is a diagram illustrating an example of a color distribution on the YU plane. The vertical axis of the graph shown in FIG. 4 indicates the color difference U, and the horizontal axis indicates the luminance Y. FIG. 4 shows a graph in which the distribution of pixels included in the first sample image and the second sample image is plotted on the YU plane. That is, in FIG. 4, the distribution of the pixels forming the first region among the pixels included in the first sample image is plotted with a circle mark, and the first of the pixels included in the second sample image is plotted. The distribution of the pixels forming the region is plotted with rhombus marks, and the distribution of the pixels forming the second region among the pixels included in the first sample image is plotted with triangular marks.

  As shown in FIG. 4, the separation condition setting unit 16 is on the YU plane corresponding to the first area on the first sample image in the same manner as in the case of setting the elliptical area on the UV plane. An elliptical area 41 and an elliptical area 42 on the YU plane corresponding to the second area on the first sample image are set. Among these, the ellipse area 41 indicates an area in which the distribution of coordinates in which the color difference U of the pixels forming the plant that is the first subject is scattered on the YU plane including the luminance Y is approximated to an ellipse. On the other hand, the ellipse area 42 is an area in which the distribution of coordinates in which the color difference U of the pixels forming the brown portion of the color chart as the second subject is scattered on the YU plane including the luminance Y is approximated to an ellipse.

  As shown in FIG. 4, on the YU plane including luminance Y, unlike the case of the UV plane shown in FIG. 3, the color distribution of the pixels forming the first area and the second area are formed. The color distribution of the pixels is not mixed in the same range, and the elliptical region 41 and the elliptical region 42 do not overlap. This is because the color distribution of the pixels forming the leaves of the plant on the first sample image and the color distribution of the pixels forming the brown part of the color chart on the first sample image include luminance Y. It means the fact that they do not overlap on a plane.

  More specifically, since the plant leaves have higher infrared spectral reflectance than the color chart, the amount of received infrared light received by each of the R, G, and B image sensors increases. doing. Therefore, when receiving visible light, the amount of light received by the green component G is relatively large, but when receiving both visible light and infrared light, each of R, G, and B is infrared. Since the amount of received light increases, the total amount of received light increases, and as a result, the ratio of the amount of received light among R, G, and B decreases relatively. For this reason, it approaches an achromatic color, and changes in the chromaticity space. On the other hand, the color chart has a smaller amount of reflected infrared light than leaves of plants, so even when both visible light and infrared light are received, the amount of change in the ratio of R, G, and B received light amounts is small. small. As a result, when both visible light and infrared light are received, the leaves of the plant greatly change in the chromaticity space, and are located in the same chromaticity space as, for example, brown in the color chart.

  Here, in the case of plant leaves, when infrared light is received, the total amount of received light increases, so that not only the chromaticity space changes but also the luminance changes. We focused on the fact that this brightness also changes greatly. From this fact, it is found that the first subject and the second subject having similar color distributions on the UV plane can be separated by setting the threshold value Th1 of the luminance Y, that is, the boundary line parallel to the U axis. Is obtained.

  Under such knowledge, the separation condition setting unit 16 can derive the threshold value Th1 of the luminance Y as the second separation condition as an example. For example, the separation condition setting unit 16 receives an input of the threshold value Th1 of the luminance Y that separates the elliptical region 41 and the elliptical region 42 via an input unit (not shown). In the example shown in FIG. 4, a value with a luminance Y of around 40 can be input as the threshold Th1. For example, the threshold value Th1 for luminance Y can be designated by numerical input by pressing a numeric keypad, or the threshold value Th1 for luminance Y can be designated by inputting a desired position on the graph with a pointing device.

  In addition to the threshold value Th1 of the luminance Y, the separation condition setting unit 16 can further calculate the threshold value Th2 of the luminance Y that defines the upper limit of the luminance of the pixels forming the first subject as the second separation condition. . In the example shown in FIG. 4, a value where the luminance Y is about 130 can be input as the threshold Th2. By using the threshold value Th2 of the luminance Y as the second separation condition, color correction for the first subject is performed on a portion where the luminance is larger than the threshold value Th2, that is, a subject other than an achromatic leaf having a high luminance. This can be suppressed.

  Here, the case where the threshold value Th1 and the threshold value Th2 of the luminance Y are input on the YU plane is exemplified, but the threshold value Th1 and the threshold value Th2 of the luminance Y are set similarly on the plane including the luminance Y, that is, the YV plane. You can also.

  After the first separation condition and the second separation condition are calculated in this way, the separation condition setting unit 16 sends the first separation condition and the second separation condition to the separation condition storage unit 140 of the color correction apparatus 100. By registering, the separation condition is set in the color correction apparatus 100. Although the case where the threshold value Th2 for the luminance Y is included in the second separation condition is illustrated here, the threshold value Th2 for the luminance Y may not necessarily be included in the second separation condition.

  Here, in the example of FIG. 4, the case where the second separation condition is designated via the operation input is illustrated, but the second separation condition may not necessarily be designated via the operation input. Separation conditions can also be set automatically.

  FIG. 5 is a diagram illustrating an example of a histogram of luminance Y. The vertical axis of the graph shown in FIG. 5 indicates the frequency, and the horizontal axis indicates the luminance Y that is a class value. As shown in FIG. 5, the separation condition setting unit 16 determines the luminance value of the pixels forming the first region on the first sample image and the pixels forming the second region on the first sample image. A brightness histogram is generated from the brightness values. In FIG. 5, the luminance frequency relating to the pixels forming the first region is shown in white, and the luminance frequency relating to the pixels forming the second region is shown in black. In this way, the luminance histogram is divided into two peaks, the luminance distribution of the pixels forming the first region and the luminance distribution of the pixels forming the second region. I understand that I have it. Therefore, the separation condition setting unit 16 calculates the rank value of the valley formed by the two distributions by applying a method for calculating a binarization threshold, such as the mode method or the P tile method. Since the valley class value calculated in this way is equivalent to the threshold value Th1 of the luminance Y shown in FIG. 4, the separation condition setting unit 16 performs color correction using the threshold value Th1 of the luminance Y as the second separation condition. Set to device 100.

  The processing units such as the first acquisition unit 11, the second acquisition unit 12, the identification unit 13, the conversion unit 14, the correction data setting unit 15, and the separation condition setting unit 16 can be implemented as follows. For example, in a central processing unit, a so-called CPU (Central Processing Unit), a process that performs the same function as each processing unit described above is developed in various semiconductor memory devices, such as RAM (Random Access Memory) and flash memory. It can be realized by executing. These processing units do not necessarily have to be executed by the central processing unit, but may be executed by an MPU (Micro Processing Unit). Each functional unit described above can also be realized by a hard wired logic such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[Configuration of Color Correction Apparatus 100]
Next, a functional configuration of the color correction apparatus 100 according to the present embodiment will be described. As illustrated in FIG. 1, the color correction apparatus 100 includes an acquisition unit 110, a conversion unit 120, a determination unit 130, a separation condition storage unit 140, a correction unit 150, and a correction data storage unit 160.

  The acquisition unit 110 is a processing unit that acquires an image. The “image” here refers to an image to which the above color correction processing is applied, unlike the sample image acquired by the first acquisition unit 11 and the second acquisition unit 12 of the setting device 10. In the following description, an “original image” may be described from one aspect of an image from which color correction is performed. Here, as an example of the form of the original image, it is assumed that the image is captured without an infrared cut filter, but the bandwidth of the visible light region and the infrared wavelength region to be used, for example, near infrared An image captured with an optical filter that transmits a part of the wavelength of light being inserted may be used as an original image.

  As one embodiment, the acquisition unit 110 can acquire an original image captured by a camera equipped with an image sensor such as a CCD or a CMOS. The acquisition unit 110 can also acquire an original image stored in an auxiliary storage device such as a hard disk or an optical disk or a removable medium such as a memory card or a USB (Universal Serial Bus) memory. In addition, the acquisition unit 110 can also acquire an original image by receiving it from an external device via a network.

  The conversion unit 120 is a processing unit that converts an original image into a chromaticity space.

  As one embodiment, the conversion unit 120 converts the data format of the original image acquired by the acquisition unit 110 into a data format expressed in a space including luminance and chromaticity. For example, the conversion unit 120 converts the pixel value of each pixel included in the original image from the RGB pixel value expressed in the RGB space to the pixel value expressed by the luminance Y and two color differences. Although the case where the original image is converted to the YUV space is illustrated here, the conversion destination chromaticity space is not limited to the YUV space, and the original image is converted to another color space, for example, an L * a * b space. It doesn't matter as you do.

  The determination unit 130 is a processing unit that refers to the separation condition stored in the separation condition storage unit 140 and determines whether or not the pixel included in the original image is a pixel corresponding to the first subject.

  As one embodiment, the determination unit 130 selects one pixel from the pixels included in the original image converted into the chromaticity space by the conversion unit 120. Subsequently, the determination unit 130 selects from the original image first with reference to the first separation condition stored in the separation condition storage unit 140, that is, the elliptical region on the UV plane corresponding to the first region described above. It is determined whether the UV component of the selected pixel is included in the elliptical region corresponding to the first region on the UV plane.

  At this time, if the UV component of the pixel is not included in the elliptical region corresponding to the first region on the UV plane, the pixel is a pixel that does not form any image of the first subject or the second subject. Can be estimated. In this case, the correction unit 150 selects color correction to be applied to other subjects other than the first subject, for example, color correction by matrix calculation.

  On the other hand, when the UV component of a pixel is included in an elliptical region corresponding to the first region on the UV plane, the pixel is estimated to be a pixel that forms an image of either the first subject or the second subject. Although it is possible, there remains room for pixels forming the second subject. In this case, the determination unit 130 refers to the second separation condition stored in the separation condition storage unit 140, that is, the threshold value Th1 of the luminance Y, and the luminance Y of the pixel previously selected from the original image sets the threshold value Th1. It is further determined whether or not it exceeds.

  Here, when the luminance Y of the pixel exceeds the threshold Th1, it can be estimated that the pixel has a color close to an achromatic color and has a large amount of received infrared light, so that the pixel forming the first subject Is likely to increase. In this case, the correction unit 150 selects color correction to be applied to the first subject, for example, color correction for shifting the color on the UV plane. On the other hand, if the luminance Y of the pixel does not exceed the threshold Th1, the pixel is an achromatic pixel in which the total amount of received light is insufficient to be regarded as the first subject, and is a pixel that forms the second subject. Will increase. In this case, the correction unit 150 selects color correction to be applied to other subjects other than the first subject, for example, color correction by matrix calculation.

  The correction unit 150 is a processing unit that performs color correction with reference to the correction data stored in the correction data storage unit 160.

  As an embodiment, when the luminance Y of the pixel exceeds the threshold Th1, the correction unit 150 performs color correction applied to the first subject, for example, color correction that shifts the color on the UV plane. That is, the correction unit 150 shifts the UV component of the pixel on the UV plane according to the shift amount defined in the first correction data stored in the correction data storage unit 160, and then shifts the UV component on the UV plane. The obtained pixel value is inversely converted to a pixel value in the RGB space. Further, the correction unit 150 does not include the first subject when the UV component of the pixel is not included in the elliptical region corresponding to the first region on the UV plane, or when the luminance Y of the pixel does not exceed the threshold Th1. Color correction to be applied to other subjects, for example, color correction by matrix calculation is executed. That is, the correction unit 150 refers to the second correction data stored in the correction data storage unit 160, that is, the matrix coefficient, and the matrix coefficient defined in the second correction data to the RGB pixel value of the pixel. A matrix operation for multiplying is executed. In addition to the color correction described above, the correction unit 150 can also execute γ correction, saturation correction, and the like on the pixel value on which the color correction has been executed.

  After color correction of each pixel of the original image is performed in this way, the image after color correction can be output to an arbitrary output destination. For example, the color-corrected image can be displayed on a predetermined display, or can be stored in an auxiliary storage device such as a hard disk or an optical disk, or a removable medium such as a memory card or a USB memory. In addition, the image after color correction may be output to an application program that causes a computer to execute various processes using the image as an input, or the image after color correction may be transmitted to an arbitrary external device via a network. it can.

  The functional units such as the acquisition unit 110, the conversion unit 120, the determination unit 130, and the correction unit 150 described above can be implemented as follows. For example, it can be realized by causing a central processing unit, a so-called CPU, or the like to develop and execute a process that exhibits the same function as each of the above processing units on a memory. These processing units are not necessarily executed by the central processing unit, and may be executed by the MPU. Each functional unit described above can also be realized by a hard-wired logic such as ASIC or FPGA.

  In addition, as the separation condition storage unit 140 and the correction data storage unit 160, various semiconductor memory elements such as a RAM and a flash memory can be employed as an example. Further, the separation condition storage unit 140 and the correction data storage unit 160 are not necessarily a main storage device and may be an auxiliary storage device. In this case, an HDD, an optical disk, an SSD, or the like can be employed.

[Process flow]
Next, the process flow of the color correction system according to the present embodiment will be described. Here, after (1) setting processing executed by the setting device 10 is described, (2) color correction processing executed by the color correction device 100 will be described.

(1) Setting Processing FIG. 6 is a flowchart illustrating a setting processing procedure according to the first embodiment. This process is started when a setting instruction to the color correction apparatus 100 is received as an example. As shown in FIG. 6, when the setting process is started, the first sample image is acquired by the first acquisition unit 11 and the second sample image is acquired by the second acquisition unit 12 (step S101). .

  Subsequently, the specifying unit 13 specifies a first region that forms part or all of the first subject from the first sample image acquired in step S101, and the second region from the first sample image. A second region that forms part or all of the subject is identified (step S102).

  The conversion unit 14 expresses the data format of the first sample image acquired by the first acquisition unit 11 and the second sample image acquired by the second acquisition unit 12 in a space including luminance and chromaticity. (Step S103).

  Thereafter, the correction data setting unit 15 determines that the color components of the pixels included in the first region identified as the range for forming the first subject in step S102 among the pixels included in the first sample image are the chromaticity space. The shift amount in the chromaticity space is determined from the position distributed above and the position in which the color component of the pixel on the second sample image corresponding to the pixel included in the first region is distributed in the chromaticity space. 1 is calculated as correction data (step S104).

  In addition, the correction data setting unit 15 calculates the image sensor from the RGB pixel values of the pixels not included in the first region among the pixels included in the two sample images of the first sample image and the second sample image. The matrix coefficient that approximates the spectral sensitivity characteristic of the color matching function to the color matching function is calculated as second correction data (step S105).

  Then, the correction data setting unit 15 registers the first correction data calculated in step S104 and the second correction data calculated in step S105 in the correction data storage unit 160 of the color correction apparatus 100, so that a plurality of correction data are set. Each correction data used for the color correction of the seed is set in the color correction apparatus 100 (step S106).

  Thereafter, the separation condition setting unit 16 is obtained by approximating an elliptical distribution of coordinates in which the color components of each pixel included in the first region on the first sample image are scattered in the chromaticity space. An elliptical area in the chromaticity space is calculated as the first separation condition (step S107).

  The separation condition setting unit 16 also includes a luminance component of each pixel included in the first area on the first sample image, and a luminance component of each pixel included in the second area on the first sample image. Therefore, the threshold value Th1 of the luminance Y that separates the first subject and the second subject is calculated as the second separation condition (step S108).

  Then, the separation condition setting unit 16 registers the first separation condition calculated in step S107 and the second separation condition calculated in step S108 in the separation condition storage unit 140 of the color correction apparatus 100, thereby separating the separation condition setting unit 16. Conditions are set in the color correction apparatus 100 (step S109), and the process ends.

(2) Color Correction Processing FIG. 7 is a flowchart illustrating the procedure of color correction processing according to the first embodiment. As an example, this color correction process is started when an original image to be subjected to color correction is designated. As shown in FIG. 7, when an original image to be subjected to color correction is designated, the acquisition unit 110 acquires the original image (step S301). Subsequently, the conversion unit 120 converts the data format of the original image acquired in step S301 into a data format expressed in a space including luminance and chromaticity (step S302).

  Then, the determination unit 130 selects one of the pixels included in the original image converted into the chromaticity space in step S302 (step S303). Subsequently, the determination unit 130 refers to the first separation condition stored in the separation condition storage unit 140, that is, the elliptical region on the UV plane corresponding to the first region described above, and from the original image in step S303. It is determined whether or not the UV component of the selected pixel is included in an elliptical area corresponding to the first area on the UV plane (step S304).

  At this time, if the UV component of the pixel is included in an elliptical area corresponding to the first area on the UV plane (Yes in step S304), the pixel forms an image of either the first subject or the second subject. Although it can be estimated that it is a pixel, there remains room for a pixel to form the second subject. In this case, the determination unit 130 refers to the second separation condition stored in the separation condition storage unit 140, that is, the threshold value Th1 of the luminance Y, and the luminance Y of the pixel selected from the original image in step S303 is the threshold value Th1. Is further determined (step S305).

  If the luminance Y of the pixel exceeds the threshold Th1 (step S305 Yes), it can be estimated that the pixel has a color close to an achromatic color and has a large amount of received infrared light, so the first subject The likelihood of being a pixel that forms a pixel is increased. In this case, the correction unit 150 shifts the UV component of the pixel on the UV plane according to the shift amount defined in the first correction data stored in the correction data storage unit 160, and then changes the pixel value to the RGB space. Color correction for the first subject to be inversely converted is executed (step S306).

  On the other hand, when the UV component of the pixel is not included in the elliptical region corresponding to the first region on the UV plane, or when the luminance Y of the pixel does not exceed the threshold Th1 (Step S304 No or Step S305 No), the correction unit 150 The following processing is performed. That is, the correction unit 150 refers to the second correction data stored in the correction data storage unit 160, that is, the matrix coefficient, and the matrix coefficient defined in the second correction data to the RGB pixel value of the pixel. Color correction is performed for a subject other than the first subject that executes the matrix operation for multiplying by (step S307).

  Thereafter, the processes from step S303 to step S307 are repeatedly executed until all the pixels included in the original image are selected (No in step S308). Then, when all the pixels included in the original image are selected (Yes in step S308), the correction unit 150 outputs the corrected image to a predetermined output destination (step S309), and the process ends.

[One aspect of effect]
As described above, the color correction system 1 according to the present embodiment has a high spectral reflectance of infrared light included in an image captured including light corresponding to wavelengths of visible light and infrared light. A first subject and a second subject having a color distribution similar to that of the first subject in the color space are separated based on the luminance. Therefore, according to the color correction system 1 according to the present embodiment, it is possible to sufficiently separate a portion where the spectral reflectance of infrared light is high and other portions.

  Although the embodiments related to the disclosed apparatus have been described above, the present invention may be implemented in various different forms other than the above-described embodiments. Therefore, another embodiment included in the present invention will be described below.

[About input images]
In the first embodiment, as an example of a photographing form in which light corresponding to the wavelengths of visible light and infrared light is imaged on the image sensor, an image photographed without an infrared cut filter is input However, the image capturing mode is not limited to this. For example, an image photographed with an optical filter that transmits the visible light region and the bandwidth of the infrared wavelength region to be used is inserted as the first sample image, and the setting process shown in FIG. 6 is applied. It is also possible to use the image as an original image and apply the color correction processing shown in FIG.

[Application Example 1 of Separation Conditions]
In the first embodiment, the case where the threshold value Th1 and the threshold value Th2 of the luminance Y are used as the second separation condition is illustrated, but the threshold value may not necessarily be used as the condition. For example, the setting device 10 can also directly use the distribution shape of the luminance histogram relating to the first region where the spectral reflectance of infrared light is high as the probability distribution. For example, the setting device 10 calculates a luminance histogram from the luminance values of the pixels forming the first region in the first sample image including reflected light with respect to both visible light and infrared light wavelength regions. The luminance histogram is normalized to a value in a predetermined range, for example, a value from 0 to 1. In such a luminance histogram, a portion with a high frequency is assumed to have a high probability of green leaves, and the frequency decreases as it goes to the periphery, so it can be assumed that the probability of green is low. Therefore, the luminance histogram normalized as described above can be used as a probability distribution in which the pixel is the first subject.

  As described above, when the probability distribution is used as the second separation condition, the color correction apparatus 100 determines the above probability distribution when the UV component of the pixel is included in the elliptical region corresponding to the first region on the UV plane. Among them, color shift correction and color correction by matrix calculation can be mixed according to the probability w1 corresponding to the luminance value of the pixel. For example, when the color component of a pixel is included in an elliptical area, the color correction apparatus 100 can blend color shift correction and color correction by matrix calculation using the following equation (1).

  RGB = r’g’b ’× w1 + rgb × (1-w1) (1)

  “R ′”, “g ′”, and “b ′” shown in the above equation (1) are RGB values after color shift correction, and “r”, “g” shown in the above equation (1). "And" b "are RGB values after color correction by matrix calculation. Further, “w1” shown in the above formula (1) indicates the probability that the pixel is the first subject, and for example, a value of 0 ≦ w1 ≦ 1 can be set according to the probability distribution. Thus, when the frequency is high, the color correction can be blended by increasing the color shift correction ratio and increasing the color correction ratio by matrix calculation as the frequency distribution decreases. Here, the case where the blend process is executed in the RGB space is illustrated, but the application range of the blend process is not limited to the image in the RGB space, and the blend process is performed in another color space such as YUV or L * a * b. It is also possible to convert to RGB after executing.

  As described above, when the probability distribution is used as the second separation condition, since the correction is performed in a stepwise manner as compared with the case where the first region is separated from the other subject by using only the threshold value, the specific color correction is unique. As a result of suppressing the change point and smoothing the boundary of the separation condition, the accuracy of separation can be further improved.

  As described above, when the probability distribution is defined by the luminance histogram, the setting device 10 may calculate, for example, an average value and a variance value (or standard deviation) and approximate the probability of the normal distribution. In this case, the average value and the variance value (or standard deviation) are set in the color correction apparatus 100 as the second separation condition. Alternatively, as the asymmetry, the setting device 10 can define a lognormal distribution, a triangular distribution, a gamma distribution, and other probability distributions. In this way, by using the approximation to the probability distribution, it is possible for the human to set the separation condition without setting the threshold value.

[Application example 2 of separation conditions]
Also in the chromaticity space, the idea of the probability described in the application example 1 of the separation condition can be used. FIG. 8A is a diagram illustrating an example of the histogram of the color component U, and FIG. 8B is a diagram illustrating an example of the histogram of the color component V. As shown in FIG. 8A, the setting device 10 generates a histogram of the color component U from the value of the color component U of the pixels forming the first region on the first sample image, and as shown in FIG. 8B. In addition, a histogram of the color component V is generated from the value of the color component V possessed by the pixels forming the first region on the first sample image. Then, as in the case of the luminance, the setting device 10 may set the color correction device 100 as the first separation condition using the probability itself obtained by normalizing the distribution of these histograms, or approximate the probability distribution. Thus, the first separation condition may be set.

[Application example 3 of separation conditions]
In the chromaticity space, the concept of probability can be applied to multi-dimensions, for example, two dimensions. FIG. 9 is a diagram illustrating a calculation example of the probability density regarding the chromaticity space. As shown in FIG. 3, the left side of FIG. 9 shows the color distribution on the UV plane, and the right side of FIG. 9 relates to the pixels forming the first region on the first sample image. The calculation result of probability density is shown.

  In accordance with the definition of the multivariate normal distribution (two-dimensional), the setting device 10 determines that the color component UV of each pixel is the first from the value of the UV component of each pixel forming the first region on the first sample image. The probability density of the subject can be calculated. That is, in the distribution shown on the left side of FIG. 9, if the value of the UV component of each pixel forming the first region on the first sample image is Z as a two-dimensional vector, the average vector of Z is It can be calculated as μ (equivalent to the average of U and the average of V). Furthermore, by calculating the variance-covariance matrix Σ, the degree of distribution variation can be calculated. If the mean and variance are known in this way, the probability density can be calculated as shown on the right side of FIG. That is, since the correlation between U and V can also be calculated, the direction of the main axis (the major axis and the minor axis of the ellipse) can also be calculated. In the probability density calculated in this way, the probability of the area of the average value of the distribution is the highest, and the peripheral probability decreases according to the variance value. The eigenvalue and eigenvector obtained by calculating the average, variance, and UV correlation can be set as the first separation condition.

  In this way, when the probability distribution is set as the first separation condition, the color correction apparatus 100 converts the original image acquired by the acquisition unit 110 into a space including luminance and chromaticity, for example, a YUV space, and the original image The first probability of whether or not each pixel included in the first region is a first region (for example, a leaf region having a high infrared spectral reflectance) is calculated according to the first separation condition. At the same time, the color correction apparatus 100 calculates, for each pixel of the original image, a second probability as to whether or not the region is the first region based on the second separation condition. In addition, the color correction apparatus 100 calculates a total probability of whether or not the pixel of the original image is the first region from the first probability and the second probability, for example, a multiplication value of the first probability and the second probability. The color correction blending process is executed according to the probability.

[Application Example 4 of Separation Conditions]
In the first embodiment, the first separation condition, that is, the separation condition related to chromaticity, and the second separation condition, that is, the separation condition related to luminance, are calculated separately. It can also be calculated as the separation condition. In this case, the setting device 10 may approximate the first region on the first sample image to an ellipsoid in a three-dimensional space including luminance and chromaticity. As a result, as in the case of the first embodiment, it is possible to separate the first subject and the second subject whose color distribution is similar to the first subject in the color space. FIG. 10 is a diagram illustrating an example of the third separation condition. FIG. 10 shows the color distribution in the YUV space. As shown in FIG. 3, it is difficult to separate the first subject and the second subject on the UV plane, but in the three-dimensional space including Y, as shown in FIG. Can be separated from the distribution of pixels included in the pixel, that is, the circle mark shown in the drawing, and the distribution of pixels included in the second region, ie, the triangular mark shown in the drawing. From this, the setting device 10 calculates the average value and variance of the pixels included in the first region on the first sample image in the three-dimensional space including luminance and chromaticity. At this time, the center of the ellipsoid is an average value, and the size in each direction is calculated from the variance value to determine the size of the ellipsoid. In this way, by approximating the distribution relating to the pixels included in the first region on the first sample image to a three-dimensional ellipsoid, the inside of the three-dimensional ellipsoid becomes a green leaf correction region. The outside is another correction. When the center and the radius in each direction regarding such a three-dimensional ellipsoid are set as the third separation condition, the color correction device 100 uses the space including luminance and chromaticity as the original image acquired by the acquisition unit 110, For example, the image is converted into a YUV space, and for each pixel included in the original image, it is determined whether or not the pixel is included in a three-dimensional ellipsoid. If the pixel is included in a three-dimensional ellipsoid, the above color shift correction is performed. When it is not included in the three-dimensional ellipsoid, the color correction by the matrix calculation may be executed.

[Application Example 5 of Separation Conditions]
In the application example 3 of the separation condition described above, the case where the two-dimensional probability density related to the chromaticity UV is calculated is exemplified, but the separation condition can be calculated with the probability density summarized in three dimensions. In this case, in the application example 3 of the above-described separation condition, the two-dimensional (U, V) probability density is calculated, but the probability density of the multivariate normal distribution may be expanded to three dimensions (YUV). As a result, the three directions of the main axis can be calculated as in the case of the application example 3 of the separation condition. The eigenvalue and eigenvector obtained in this way can be set as the third separation condition instead of the first separation condition and the second separation condition. When such a three-dimensional probability density is set as a separation condition, the color correction apparatus 100 uses the third separation condition for each pixel included in the original image according to the third separation condition (for example, the spectral reflectance of infrared light is high). A third probability of whether or not the area is a leaf area) is calculated. Then, the color correction blending process may be executed according to the third probability.

[Distribution and integration]
In addition, each component of each illustrated apparatus does not necessarily have to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. For example, the acquisition unit 110, the conversion unit 120, the determination unit 130, or the correction unit 150 may be connected as an external device of the color correction apparatus 100 via a network. In addition, another device has the acquisition unit 110, the conversion unit 120, the determination unit 130, or the correction unit 150, and the functions of the color correction device 100 described above are realized by cooperation through a network connection. Also good.

[Color correction program]
The various processes described in the above embodiments can be realized by executing a prepared program on a computer such as a personal computer or a workstation. In the following, an example of a computer that executes a color correction program having the same function as that of the above-described embodiment will be described with reference to FIG.

  FIG. 11 is a diagram illustrating a hardware configuration example of a computer that executes a color correction program according to the first and second embodiments. As illustrated in FIG. 11, the computer 1000 includes an operation unit 1100a, a speaker 1100b, a camera 1100c, a display 1200, and a communication unit 1300. Further, the computer 1000 includes a CPU 1500, a ROM 1600, an HDD 1700, and a RAM 1800. These units 1100 to 1800 are connected via a bus 1400.

  As illustrated in FIG. 11, the HDD 1700 stores a color correction program 1700 a that exhibits the same functions as the acquisition unit 110, the conversion unit 120, the determination unit 130, and the correction unit 150 described in the first embodiment. The color correction program 1700a may be integrated or separated as with the components of the acquisition unit 110, the conversion unit 120, the determination unit 130, and the correction unit 150 illustrated in FIG. In other words, the HDD 1700 does not necessarily store all the data shown in the first embodiment, and the HDD 1700 may store data used for processing.

  Under such an environment, the CPU 1500 reads the color correction program 1700a from the HDD 1700 and develops it in the RAM 1800. As a result, the color correction program 1700a functions as a color correction process 1800a as shown in FIG. The color correction process 1800a expands various data read from the HDD 1700 in an area allocated to the color correction process 1800a in the storage area of the RAM 1800, and executes various processes using the expanded data. For example, as an example of processing executed by the color correction process 1800a, the processing shown in FIG. 7 and the processing described in the second embodiment are included. Note that the CPU 1500 does not necessarily operate all the processing units described in the first embodiment, and it is only necessary to virtually realize a processing unit corresponding to a process to be executed.

  Note that the color correction program 1700a is not necessarily stored in the HDD 1700 or the ROM 1600 from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk inserted into the computer 1000, so-called FD, CD-ROM, DVD disk, magneto-optical disk, or IC card. Then, the computer 1000 may acquire and execute each program from these portable physical media. Each program is stored in another computer or server device connected to the computer 1000 via a public line, the Internet, a LAN, a WAN, etc., and the computer 1000 acquires and executes each program from these. It may be.

DESCRIPTION OF SYMBOLS 1 Color correction system 10 Setting apparatus 11 1st acquisition part 12 2nd acquisition part 13 Specification part 14 Conversion part 15 Correction data setting part 16 Separation condition setting part 100 Color correction apparatus 110 Acquisition part 120 Conversion part 130 Determination part 140 Separation condition memory | storage 150 correction unit 160 correction data storage unit

Claims (10)

An acquisition unit for acquiring images;
A conversion unit for converting an image into a component of a color space including luminance and chromaticity;
For each pixel included in the image converted into the color space component, the pixel has a high spectral reflectance of infrared light depending on whether the pixel satisfies the separation condition regarding the chromaticity component and the luminance component. And a determination unit that determines whether the subject corresponds to the subject other than the first subject or the subject other than the first subject. A sample in which the first subject and a second subject whose color distribution is similar to the first subject in the color space include light corresponding to wavelengths of visible light and infrared light A first storage unit that stores a first separation condition in which a first region corresponding to the first subject on the image is defined in a color space;
A second storage unit that stores a second separation condition in which a threshold value of a luminance component that separates the first region corresponding to the first subject and the second region corresponding to the second subject is defined; Further comprising
The determination unit refers to the first separation condition stored in the first storage unit, determines whether or not the color component of the pixel is included in the first region on the color space, When the color component of the pixel is included in the first area on the color space, the luminance component of the pixel has the threshold value with reference to the second separation condition stored in the second storage unit. The color correction apparatus according to claim 1, wherein it is determined whether the pixel corresponds to the first subject or the other subject by determining whether or not the pixel exceeds.   The second separation condition includes a luminance histogram generated from luminance values of pixels included in the first area, and a luminance histogram generated from luminance values of pixels included in the second area. The color correction apparatus according to claim 2, wherein a threshold value of a luminance component for separating the color is set.   When the determination unit determines that the luminance component of the pixel exceeds the threshold, color correction for the first subject is performed, and the color component of the pixel is the first region on the color space. And a correction unit that performs color correction different from the color correction for the first subject when it is determined that the luminance component of the pixel does not exceed the threshold value. The color correction apparatus according to claim 2 or 3.   The determination unit determines a luminance value of a first region corresponding to the first subject on the sample image captured in a state where the first subject includes light corresponding to wavelengths of visible light and infrared light. Calculating a probability that a luminance component of the pixel corresponds to the first subject with reference to a probability distribution defined as a probability of whether or not a generated luminance histogram corresponds to the first subject. The color correction apparatus according to claim 1, wherein the color correction apparatus is a color correction apparatus.   The ratio of mixing the color correction for the first subject and the color correction different from the color correction for the first subject is changed according to the probability that the luminance component of the pixel corresponds to the first subject. The color correction apparatus according to claim 5, further comprising a correction unit that performs color correction. A three-dimensional space in which the first region corresponding to the first subject on the sample image captured in a state where the first subject includes light corresponding to wavelengths of visible light and infrared light includes luminance and chromaticity A storage unit for storing the ellipsoid approximated above;
The determination unit refers to the storage unit, and the pixel corresponds to either the first subject or the other subject depending on whether the pixel is included in the ellipsoid in the three-dimensional space. The color correction apparatus according to claim 1, wherein the color correction apparatus determines whether or not.   In the determination unit, pixels in the first region corresponding to the first subject on the sample image captured in a state where the first subject includes light corresponding to wavelengths of visible light and infrared light are luminance The probability that the pixel corresponds to the first subject is calculated with reference to a probability density distribution calculated from an average and a variance of a distribution scattered in a three-dimensional space including chromaticity. The color correction apparatus according to claim 1. Computer
Get an image,
Convert the image into color space components including brightness and chromaticity,
For each pixel included in the image converted into the color space component, the pixel has a high spectral reflectance of infrared light depending on whether the pixel satisfies the separation condition regarding the chromaticity component and the luminance component. A process for determining whether to correspond to a subject other than the first subject or a subject other than the first subject is performed. On the computer,
Get an image,
Convert the image into color space components including brightness and chromaticity,
For each pixel included in the image converted into the color space component, the pixel has a high spectral reflectance of infrared light depending on whether the pixel satisfies the separation condition regarding the chromaticity component and the luminance component. A color correction program for executing a process of determining which one of the subject and the other subject other than the first subject corresponds.

Images