JP2017017564A - Color correction program, color correction method, and color correction apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve a color reproducibility.SOLUTION: A color correction apparatus 100 performs a processing of acquiring an image; a processing of converting the image into a component of a color space including luminance and chromaticity; a processing of determining whether or not a pixel included in the image converted into the component of color space is the pixel corresponding to a subject of which a spectral reflectance of an infrared light is higher than that of a visible light; and a processing of shifting a color component of the pixel corresponding to the subject on the color space according to correction data associated with a ratio of a distance relating to a color distribution of the pixel corresponding to the subject and a degree of spread between a first sample image of which the subject is imaged at the state of receiving a light corresponding to a wavelength of the visible light and infrared light and a second sample image of which the subject is imaged at the state of receiving the light corresponding to the wavelength of visible light.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a color correction program, a color correction method, and a color correction apparatus.

  Cameras are used in a wide variety of applications. For example, the camera may be implemented as a surveillance camera or an in-vehicle camera, or may be implemented as an image capturing device used for various types of biometric authentication or gaze detection using corneal reflection.

  For example, in the case of a surveillance camera or an in-vehicle camera, it may be used both day and night. In this case, by installing an infrared cut filter that removes infrared light in the daytime, visible light that can be recognized by the human eye is imaged on the light receiving element, while the infrared cut filter is removed at night and in the dark. As a result, infrared light such as near infrared may be used as illumination. Further, in the case of an information processing terminal equipped with functions such as biometric authentication and line-of-sight detection, an image used for biometric authentication and line-of-sight detection and an image used for other purposes are captured by one camera, thereby There are also trends aimed at reducing scale and costs.

  When the imaging device receives visible light and infrared light in this way, the image captured by the camera is different from the color that a person recognizes with eyes under visible light. From this, by correcting the color of an image captured by an image sensor that receives visible light and infrared light, visible light is received, in other words, the same color as when infrared light is not received. Techniques for reproducing the taste have been proposed.

  As an example, there is a method of correcting an input image by matrix calculation. In this method, matrix coefficients used for color correction are calculated from two images taken by different image input systems of a plurality of colors, for example, a color chart including 24 colors. That is, from two images of an image taken with an infrared cut filter incorporated on the optical path and an image taken with no infrared cut filter incorporated on the optical path, A matrix coefficient that approximates the spectral sensitivity characteristic to the color matching function is obtained by the method of least squares. The color correction of the input image is performed by the calculation using the matrix coefficient.

  However, when color correction is performed by the above matrix calculation, color reproducibility can be sufficiently exerted on a specific subject having a high spectral reflectance of infrared light, such as a plant leaf or a chemical fiber cloth. There are cases where it is not possible. For example, in the case of plant leaves, the spectral reflectance increases at a wavelength corresponding to near-infrared light exceeding the upper limit of visible light due to the influence of chlorophyll contained in the leaves. For this reason, the influence of the color that infrared light included in environmental light such as natural light or illumination has on an image is greater in the leaves of plants than in other subjects than the leaves of plants. Therefore, even if color correction is performed by the above matrix calculation, the leaf color of the plant cannot be sufficiently reproduced as compared with other subjects. As a result, the color of the leaf, which is recognized as green under visible light, is recognized as a color different from the green color of the color chart that is also recognized as green with visible light.

  In order to suppress such a decrease in color reproducibility, the following correction data generation apparatus has also been proposed. In the correction data generation device, the first sample image in which the specific subject is imaged without the infrared cut filter and the first sample image in which the specific subject is imaged with the infrared cut filter attached. Correction data for correcting the color of the specific subject is generated using the two sample images.

  In such a correction data generation device, the knowledge that colors included in a specific subject are concentrated in a local area close to an achromatic color in the color space is used. Based on such knowledge, the correction data generation device specifies an elliptical region in which the color included in the specific subject is distributed in the color space by designating a range in which the specific subject appears on the first sample image. Set. Then, the correction data generation device calculates, as a correction amount, a shift amount that shifts the elliptical region set using the first sample image to the elliptical region in which the color of a specific subject included in the second sample image is distributed. To do. In addition, a portion having a high spectral reflectance of infrared light depending on whether or not the color of the pixel included in the input image is included in an elliptical region on the color space set using the first sample image And other parts. That is, correction for shifting the color using the above correction data is performed on the portion where the spectral reflectance of infrared light is high, while color correction is performed on the other portions by the above matrix calculation. .

JP2013-150269A JP 2014-200045 A JP 2005-328845 A JP2013-128259A JP 2006-094112 A

Yoshitaka Toyoda, Tetsuya Kuno, Shinzo Ishida, Hiroaki Sugiura “Study on Near-Infrared Imaging Device Capable of Color Imaging”, Journal of the Institute of Image Information and Television Engineers, Vol. 64, no. 1, pp. 101-110 2010

  However, the above technique has its own limitations in color reproducibility, as will be described below.

  That is, the correction data generation apparatus uses the knowledge that colors included in a specific subject are concentrated in a local region close to an achromatic color in the color space of the first sample image. However, even in the above-described second sample image, the colors included in the specific subject are not concentrated in the local area, but actually, they are distributed over a wider range than the distribution on the first sample image. Nevertheless, since the correction data generation apparatus performs color correction by uniformly applying the shift amount, the color of a specific subject is concentrated in a local area even after color correction. Become. As a result, even if the color of a specific subject is corrected to a color similar to the color recognized under visible light, it is corrected to a color that does not have a spread like the real thing, so color reproducibility is improved. There is a limit.

  In one aspect, an object of the present invention is to provide a color correction program, a color correction method, and a color correction apparatus that can improve color reproducibility.

  In one embodiment, the computer includes processing for acquiring an image, processing for converting the image into a component of a color space including luminance and chromaticity, and pixels included in the image converted into the component of the color space are infrared light. A process for determining whether or not the pixel corresponds to a subject whose spectral reflectance is higher than that of visible light, and the subject receives light corresponding to wavelengths of visible light and infrared light. The distribution of chromaticity of pixels corresponding to the subject between the first sample image taken and the second sample image taken in a state where the subject receives light corresponding to the wavelength of visible light According to the correction data in which the ratio between the distance and the spread degree is associated, a process of executing color correction for shifting the color component of the pixel corresponding to the subject on the color space is executed.

  Color reproducibility can be improved.

FIG. 1 is a block diagram illustrating a functional configuration of each device included in the color correction system according to the first embodiment. FIG. 2A is a diagram illustrating an example of a sample image. FIG. 2B is a diagram illustrating an example of a sample image. FIG. 3A is a diagram illustrating an example of luminance values of RGB components. FIG. 3B is a diagram illustrating an example of luminance values of RGB components. FIG. 4A is a diagram illustrating an example of a histogram. FIG. 4B is a diagram illustrating an example of a histogram. FIG. 5 is a diagram illustrating an example of a color distribution on the UV plane. FIG. 6 is a flowchart illustrating the procedure of the setting process according to the first embodiment. FIG. 7 is a flowchart of the first correction data calculation process according to the first embodiment. FIG. 8 is a flowchart illustrating the procedure of the color correction process according to the first embodiment. FIG. 9 is a diagram illustrating an example of a color distribution in the YUV space. FIG. 10A is a diagram illustrating an example of a color distribution on the UV plane. FIG. 10B is a diagram illustrating an example of a color distribution on the UV plane. FIG. 11 is a diagram illustrating an example of the first correction data. FIG. 12 is a flowchart illustrating a procedure of color correction processing according to the second embodiment. FIG. 13 is a diagram illustrating a hardware configuration example of a computer that executes a color correction program according to the first and second embodiments.

  Hereinafter, a color correction program, a color correction method, and a color correction apparatus according to the present application will be described with reference to the accompanying drawings. Note that this embodiment does not limit the disclosed technology. Each embodiment can be appropriately combined within a range in which processing contents are not contradictory.

[Configuration of color correction system]
FIG. 1 is a block diagram illustrating a functional configuration of each device included in the color correction system according to the first embodiment. A color correction system 1 shown in FIG. 1 realizes a color correction process for correcting the color of an image captured including light corresponding to wavelengths of visible light and infrared light in accordance with human color vision characteristics. . Here, as an example, a case where the image is taken without a filter that removes infrared rays, that is, a so-called infrared cut filter, is exemplified. However, as described later, one of the wavelengths corresponding to infrared light is used. Needless to say, the scope of application includes a case where an image photographed in a state where an optical filter that transmits light in a certain wavelength region is attached is input.

  Here, as part of the above color correction processing, color components of pixels corresponding to a specific subject having a high spectral reflectance of infrared light among images picked up including light having wavelengths of visible light and infrared light. Are shifted in a wider range than the range in which the color of the subject is distributed in the color space. In this way, when color correction is performed on a specific subject such as a plant leaf or a chemical fiber cloth, the color of the specific subject is prevented from being concentrated in a local area on the color space. As a result, the color of a specific subject is corrected to a color that is similar to the color recognized under visible light, and even if it is classified as the same color as the real thing, it is corrected to a color in which each other has a spread. Is done. By such color correction processing, improvement of color reproducibility is realized as one aspect.

  As shown in FIG. 1, the color correction system 1 includes a setting device 10 and a color correction device 100. The setting device 10 and the color correction device 100 are connected to be communicable via a predetermined network. As an example, any type of communication network such as the Internet (Internet), LAN (Local Area Network), and VPN (Virtual Private Network) can be adopted as such a network, regardless of whether it is wired or wireless.

  The setting device 10 is a device that sets various parameters relating to color correction.

  As one embodiment, the setting device 10 sets correction data related to color correction to the color correction device 100, a specific subject whose infrared light spectral reflectance is higher than other wavelength regions, and other than that. Separation conditions for separating the subject from each other are set. Although FIG. 1 shows an example in which various parameters relating to color correction are set through the setting device 10, they may be set in advance in the color correction device 100 before the color correction device 100 is shipped. Further, the setting device 10 and the color correction device 100 do not necessarily need to be constructed separately, and the color correction device 100 can also have the function of the setting device 10.

  The color correction apparatus 100 is a computer that realizes the color correction process.

  As one embodiment, the color correction apparatus 100 can be implemented by installing a color correction program for executing the above color correction processing as package software or online software on a desired computer. For example, the information processing apparatus can function as the color correction apparatus 100 by causing the information processing apparatus to execute the color correction program. The information processing apparatus mentioned here includes a desktop or notebook personal computer, a mobile communication terminal such as a smartphone, a mobile phone or a personal handyphone system (PHS), or a slate such as a PDA (Personal Digital Assistants). Terminals are included in the category. Further, the terminal device used by the user can be used as a client, and the client can be implemented as a server device that provides the client with services related to the color correction processing. For example, the color correction apparatus 100 is implemented as a server apparatus that provides a color correction service that receives an image or identification information associated with the image and outputs the execution result of the color correction processing for the image. . In this case, the color correction apparatus 100 may be implemented as a Web server, or may be implemented as a cloud that provides a service related to the color correction processing described above by outsourcing.

[Configuration of Setting Device 10]
Subsequently, a functional configuration of the setting apparatus 10 according to the present embodiment will be described. As illustrated in FIG. 1, the setting device 10 includes a first acquisition unit 11, a second acquisition unit 12, a specification unit 13, a conversion unit 14, a correction data setting unit 15, and a separation condition setting unit 16. Have. Note that the setting device 10 may include various functional units included in known computers, for example, functional units such as various input devices and audio output devices, in addition to the functional units illustrated in FIG.

  Among these, the first acquisition unit 11 and the second acquisition unit 12 are both processing units that acquire sample images as materials for generating various parameters relating to color correction. Hereinafter, the sample image acquired by the first acquisition unit 11 is referred to as a “first sample image”, and the sample image acquired by the second acquisition unit 12 is referred to as a “second sample image”. May be described.

  As one embodiment, the first acquisition unit 11 and the second acquisition unit 12 acquire a sample image captured by a camera equipped with an image sensor such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). Can do. The first acquisition unit 11 and the second acquisition unit 12 acquire the sample image by reading a sample image stored in an auxiliary storage device such as a hard disk or an optical disk or a removable medium such as a memory card or a USB (Universal Serial Bus) memory. You can also In addition, the 1st acquisition part 11 and the 2nd acquisition part 12 can also acquire a sample image by receiving via a network from an external device.

  2A and 2B are diagrams illustrating examples of sample images. Here, as an example, the following description will be made on the assumption that an image having three RGB light receiving elements is used to capture an image in which each pixel has RGB luminance values. Among these, the red component R receives the red wavelength range and the infrared wavelength range among the visible light wavelengths, and the green component G includes the green wavelength range and the infrared light among the visible light wavelengths. The blue component B receives the blue wavelength range and the infrared wavelength range among the visible light wavelengths.

  The first sample image 20A shown in FIG. 2A and the second sample image 20B shown in FIG. 2B are common in that the same target is photographed at the same position and the same angle of view by the same camera. However, a part of the image input system in which images of each other are taken is different. That is, the first sample image 20A shown in FIG. 2A is an image taken without an infrared cut filter, while the second sample image 20B shown in FIG. 2B has an infrared cut filter attached. The difference is that the image is taken in a state. Therefore, the first sample image 20A includes the influence of light having the wavelength of infrared light in addition to light having the wavelength of visible light, while the second sample image 20B includes infrared light. This does not include the influence of light having a wavelength of. In addition, although the case where it image | photographed in the state which is not mounting | wearing with an infrared cut filter was illustrated as an example of the imaging | photography form of a 1st sample image here, the bandwidth of the visible light region and the infrared wavelength region to be used, for example, An image photographed with an optical filter that transmits a part of the wavelength of near-infrared light being inserted may be used as the first sample image.

  When the first sample image 20A and the second sample image 20B are photographed, for example, the photographing is performed in a state where the specific subject is included in the imaging range. An example of such a specific subject is a plant containing a chemical substance such as chlorophyll in a leaf. When the plant is photographed in this way, the shape of the plant image 21A and the plant image 21B is reflected in the same manner between the first sample image 20A and the second sample image 20B. Is different. That is, in the case of a plant, the spectral reflectance of infrared light becomes larger than other subjects due to the influence of chlorophyll contained in the leaves of the plant. Therefore, the plant image 21B appears green like the color recognized by the human eye, while the plant image 21A is grayish celadon or grayish pale green due to the influence of reflected light of infrared light. Reflected in.

  Furthermore, shooting is performed in a state where other subjects other than the specific subject are included in the imaging range. An example of such another subject is a color chart in which 24 different colors are arranged in 4 rows and 6 columns, as shown in FIGS. 2A and 2B. By including such a color chart in the imaging range, a correction amount used when color correction is performed on a subject other than a specific subject, for example, the matrix coefficient described above in the background section, is set in a subsequent processing unit. It becomes possible.

  Here, a plant is illustrated as an example of the specific subject described above, but other than this, any object other than a plant may be used as long as the spectral reflectance increases at a predetermined wavelength, for example, about 700 nm. It does not matter as a subject. That is, any object, for example, a cloth of a chemical fiber, is used as long as the spectral reflectance is higher than the spectral reflectance in the wavelength range corresponding to visible light near the upper limit of visible light or in the near infrared wavelength range. It doesn't matter.

  The specifying unit 13 forms a first region that forms part or all of a specific subject on each sample image, and a second region that forms part or all of a subject other than the specific subject on each sample image. It is a processing unit for specifying an area.

  As one embodiment, the specifying unit 13 displays at least one of the first sample image and the second sample image on the display unit of the setting device 10 or the display unit of the external device connected to the setting device 10. Is displayed. Then, the specifying unit 13 causes a range including one or a plurality of pixels forming a plant leaf to be designated on the sample image displayed on the display unit through an input unit (not shown). For example, the specifying unit 13 can specify the first region corresponding to the leaf of the plant on the sample image by receiving the drag and drop range specification via the pointing device. The specifying unit 13 specifies a second region that forms part or all of the color chart on the sample image in the same manner as in the case of specifying the first region. In addition, although the case where the first area and the second area are specified by an operation input is illustrated here, it does not necessarily have to be specified by an operation input. For example, by applying image recognition such as template matching to the sample image, a region similar in shape to a plant or a plant leaf among the regions labeled on the sample image is specified as the first region. A region similar in shape to the color chart may be specified as the second region.

  The conversion unit 14 is a processing unit that converts the data format of the sample image.

  As an embodiment, the conversion unit 14 converts the data format of the first sample image acquired by the first acquisition unit 11 and the second sample image acquired by the second acquisition unit 12 in a space including luminance and color difference. Convert to a data format that can be represented. For example, the conversion unit 14 includes pixels represented by luminance Y and two color differences from the RGB pixel values represented in the RGB space for the pixel values of the pixels included in the first sample image and the second sample image. Convert to value. Here, the case where each sample image is converted to the YUV space is illustrated, but the color space of the conversion destination is not limited to the YUV space, and the sample image is converted to another color space, for example, L * a * b space. It doesn't matter as you do.

  The correction data setting unit 15 is a processing unit that sets correction data related to color correction in the color correction apparatus 100.

  As one embodiment, the correction data setting unit 15 includes first correction data used for color correction related to the specific subject, second correction data used for color correction related to a subject other than the specific subject, Is calculated.

(1) First Correction Data FIGS. 3A and 3B are diagrams illustrating examples of luminance values of RGB components. In FIG. 3A and FIG. 3B, the luminance value of the pixel located at the same coordinate between the first sample image and the second sample image is shown for each RGB component. That is, in the left part of FIG. 3A and FIG. 3B, the R component and the G component possessed by the pixel 22B or the pixel 23B which is one of the pixels included in the plant image 21B on the second sample image shown in FIG. 2B. 3A and 3B, the right side of FIGS. 3A and 3B shows a pixel 22A located at the same coordinates as the pixel 22B or the pixel 23B on the first sample image shown in FIG. 2A. Or, the luminance values of the R component, G component, and B component of the pixel 23A are shown. Further, FIGS. 3A and 3B show the breakdown of the amount of visible light and infrared light in the luminance value for each of the R component, the G component, and the B component. That is, in FIG. 3A and FIG. 3B, the luminance value measured by receiving visible light is shown as a black background, while the luminance value measured by receiving infrared light is shown as a white background. It is shown.

  As shown in FIGS. 3A and 3B, the luminance value of the G component is the highest among the R component, the G component, and the B component in any of the pixel 22B and the pixel 23B on the second sample image, and compared with the G component. The luminance value of the R component and the luminance value of the B component are small. For this reason, both the pixel 22B and the pixel 23B are classified as green, but this does not mean that the classification is the same and the color is the same. That is, the same green color also varies, and the pixel 22B is visually recognized as darker green than the pixel 23B.

  On the other hand, the pixels 22A and 23A on the first sample image receive not only light corresponding to the wavelength of visible light but also light corresponding to the wavelength of infrared light. In this case, the ratio of the luminance value among the R component, G component, and B component is the pixel 22B and the pixel on the second sample image by the amount of light corresponding to the wavelength of infrared light received by each RGB component. It becomes relatively smaller than 23B. As a result, the color in which the pixel 22A and the pixel 23A are visually recognized is less likely to have a color variation as the saturation is reduced by approaching an achromatic color.

  4A and 4B are diagrams illustrating examples of histograms. In FIGS. 4A and 4B, a histogram relating to the color difference of the pixels included in the first region on the first sample image is indicated by a white background and included in the first region on the second sample image. A histogram relating to the color difference of each pixel is shown with a black background. 4A and 4B, the vertical axis indicates the frequency, and the horizontal axis of the graph indicates the color difference U or the color difference V, which is a class value.

  As described with reference to FIGS. 3A and 3B, the histograms shown in FIGS. 4A and 4B are in the first region when only visible light is received and when visible light and infrared light are received. There is a difference in the extent of the distribution in which the colors of the contained pixels are scattered in the color space, and the latter has a smaller extent of distribution than the former, in other words, the former has a greater extent of distribution than the latter I support that.

  Focusing on the difference in the spread degree, the correction data setting unit 15, as an example, has a distribution in which the color components of pixels included in the first region of the first sample image are scattered in the color space, and the second From the distribution in which the color components of the pixels included in the first region of the sample image are scattered in the color space, in addition to the distance related to the mutual distribution, the ratio of the extent of the mutual distribution is calculated as the first correction data To do.

  In this way, in addition to the distance between the two distributions, by using the ratio of the extents of the two distributions as the first correction data, color correction for shifting the color component of the pixel corresponding to the specific subject is performed. In addition, the shifted color components are prevented from being concentrated in a local area on the color space.

  More specifically, the correction data setting unit 15 includes each pixel included in the first region specified by the specifying unit 13 among the first sample images converted from the RGB space to the YUV space by the conversion unit 14. The average value of the color difference is calculated for each U component and V component. Similarly, the correction data setting unit 15 among the pixels included in the first region specified by the specifying unit 13 among the second sample images converted from the RGB space to the YUV space by the conversion unit 14. The average value of the color difference is calculated for each U component and V component. Then, the correction data setting unit 15 determines the distance between the average color difference coordinate calculated from the first sample image and the average color difference coordinate calculated from the second sample image as the U component and Calculated for each V component.

  The distance of the color difference between the first areas calculated from the first sample image and the second sample image in this way is the shift amount for shifting the color component of the pixel corresponding to the specific subject on the color space. Used as Δu and Δv. Here, as an example, the case where the distance is obtained from the average value of the color differences is illustrated, but the distance may not necessarily be the average value, and the distance may be obtained from the median value or the mode value.

  Further, the correction data setting unit 15 distributes the color difference of the pixels included in the first region specified by the specifying unit 13 among the first sample images converted from the RGB space to the YUV space by the conversion unit 14. Is calculated for each U component and V component. As an example of a statistic that evaluates the degree of spread of the distribution, the standard deviation and variance of the color difference can be calculated using the average value of the color differences used for calculating the shift amounts Δu and Δv. Similarly, the correction data setting unit 15 determines the color difference of the pixels included in the first region specified by the specifying unit 13 among the second sample images converted from the RGB space to the YUV space by the conversion unit 14. The extent of distribution is calculated for each U component and V component. In addition, the correction data setting unit 15 sets the ratio of the spread degree between the spread degree calculated from the first sample image and the spread degree calculated from the second sample image for each of the U component and the V component. To calculate.

  The ratios σu_hi and σv_hi of the spread degree between the first regions calculated from the first sample image and the second sample image in this way are the color components of the pixels corresponding to the specific subject, and the shift amounts Δu and It is used as a correction amount that defines the range of the color space to be shifted according to Δv.

  The shift amounts Δu and Δv and the spread degree ratios σu_hi and σv_hi are registered as first correction data in the correction data storage unit 160 of the color correction device 100, whereby the first correction data is stored in the color correction device 100. Is set. In addition, although the case where the ratio of the shift amount and the spread degree on the UV plane is calculated is illustrated here, the ratio of the shift amount and the spread degree on the ab plane in the L * a * b space may be calculated. It doesn't matter.

(2) Second Correction Data The second correction data is calculated according to the method disclosed in Non-Patent Document 1 “Study on Near-Infrared Imaging Device Capable of Color Imaging” as an example. can do. In other words, the correction data setting unit 15 determines the image pickup element from the RGB pixel values of the pixels included in the second region among the pixels included in the two sample images of the first sample image and the second sample image. A matrix coefficient that approximates the spectral sensitivity characteristic to the color matching function can be obtained by the method of least squares.

  The matrix coefficient calculated in this way is registered as the second correction data in the correction data storage unit 160 of the color correction apparatus 100, whereby the second correction data is set in the color correction apparatus 100.

  The separation condition setting unit 16 is a processing unit that sets, in the color correction apparatus 100, a separation condition for separating the specific subject from other subjects.

  As one embodiment, the separation condition setting unit 16 scatters the colors of the pixels included in the first region among the pixels included in the first sample image as an example of the above-described separation condition in the color space. An area in which the shape and size of the coordinate distribution is defined can be set in the color correction apparatus 100. For example, the separation condition setting unit 16 determines the distribution of coordinates where the UV components of each pixel included in the first region on the first sample image are scattered on the UV plane as the ellipse center point, the ellipse axis length, and the ellipse rotation. By defining with parameters such as an angle, it can be approximated as an elliptical region on the UV plane. At this time, the separation condition setting unit 16 defines an elliptical region by narrowing down to a location where pixels forming the first region are concentrated and distributed on the UV plane of the color space in the first sample image. It is also possible to include pixels whose distance between pixels is within a predetermined value and exclude pixels that exceed the predetermined value from the inclusion target. In this manner, an elliptical area representing a distribution in which the UV components of the pixels forming the first subject are scattered on the UV plane on the first sample image is calculated. In addition, the separation condition setting unit 16 sets the separation condition in the color correction apparatus 100 by registering the above elliptical area as the separation condition in the separation condition storage unit 140 of the color correction apparatus 100. Here, the case where the distribution is approximated to an elliptical area is illustrated, but it may be approximated to another shape, for example, a polygon.

  The processing units such as the first acquisition unit 11, the second acquisition unit 12, the identification unit 13, the conversion unit 14, the correction data setting unit 15, and the separation condition setting unit 16 can be implemented as follows. For example, in a central processing unit, a so-called CPU (Central Processing Unit), a process that performs the same function as each processing unit described above is developed in various semiconductor memory devices, such as RAM (Random Access Memory) and flash memory. It can be realized by executing. These processing units are not necessarily executed by the central processing unit, but may be executed by an MPU (Micro Processing Unit) or a DSP (Digital Signal Processor). Each functional unit described above can also be realized by a hard wired logic such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[Configuration of Color Correction Apparatus 100]
Next, a functional configuration of the color correction apparatus 100 according to the present embodiment will be described. As illustrated in FIG. 1, the color correction apparatus 100 includes an acquisition unit 110, a conversion unit 120, a determination unit 130, a separation condition storage unit 140, a correction unit 150, and a correction data storage unit 160.

  The acquisition unit 110 is a processing unit that acquires an image. The “image” here refers to an image to which the above color correction processing is applied, unlike the sample image acquired by the first acquisition unit 11 and the second acquisition unit 12 of the setting device 10. In the following description, an “original image” may be described from one aspect of an image from which color correction is performed. Here, as an example of the form of the original image, it is assumed that the image is captured without an infrared cut filter, but the bandwidth of the visible light region and the infrared wavelength region to be used, for example, near infrared An image captured with an optical filter that transmits a part of the wavelength of light being inserted may be used as an original image.

  As one embodiment, the acquisition unit 110 can acquire an original image captured by a camera equipped with an image sensor such as a CCD or a CMOS. The acquisition unit 110 can also acquire an original image stored in an auxiliary storage device such as a hard disk or an optical disk or a removable medium such as a memory card or a USB (Universal Serial Bus) memory. In addition, the acquisition unit 110 can also acquire an original image by receiving it from an external device via a network.

  The conversion unit 120 is a processing unit that converts the data format of the original image.

  As one embodiment, the conversion unit 120 converts the data format of the original image acquired by the acquisition unit 110 into a data format expressed in a space including luminance and chromaticity. For example, the conversion unit 120 converts the pixel value of each pixel included in the original image from the RGB pixel value expressed in the RGB space to the pixel value expressed by the luminance Y and two color differences. Here, the case where the original image is converted to the YUV space is exemplified, but the color space of the conversion destination is not limited to the YUV space, and the original image is converted to another color space, for example, the L * a * b space. It doesn't matter.

  The determination unit 130 is a processing unit that refers to the separation condition stored in the separation condition storage unit 140 and determines whether a pixel included in the original image is a pixel corresponding to a specific subject.

  As one embodiment, the determination unit 130 selects one pixel from the pixels included in the original image converted into the color space by the conversion unit 120. Subsequently, the determination unit 130 refers to the separation condition stored in the separation condition storage unit 140, that is, the pixel previously selected from the original image with reference to the elliptical region on the UV plane corresponding to the first region described above. Is determined to be included in an elliptical area corresponding to the first area on the UV plane. At this time, if the UV component of the pixel is not included in the elliptical region corresponding to the first region on the UV plane, it can be estimated that the pixel is a pixel that does not form a specific subject. In this case, the correction unit 150 selects color correction to be applied to a subject other than the specific subject, for example, color correction by matrix calculation. On the other hand, when the UV component of a pixel is included in an elliptical region corresponding to the first region on the UV plane, it can be estimated that the pixel is a pixel that forms a specific subject. In this case, the correction unit 150 selects color correction to be applied to the first subject, for example, color correction that shifts the color on the UV plane according to the shift amounts Δu and Δv and the spread degree ratios σu_hi and σv_hi. Note that here, as an example only, a pixel corresponding to a specific subject is separated from a pixel that is not based on whether or not the UV component of the pixel is included in an elliptical region corresponding to the first region on the UV plane. Although the case is illustrated, it goes without saying that other separation conditions can be used.

  The correction unit 150 is a processing unit that performs color correction with reference to the correction data stored in the correction data storage unit 160.

  As one embodiment, the correction unit 150 performs color correction to be applied to a specific subject or other than a specific subject depending on whether or not the UV component of the pixel is included in an elliptical region corresponding to the first region on the UV plane. Switch and execute color correction to be applied to other subjects.

  For example, when the UV component of the pixel is not included in the elliptical area corresponding to the first area on the UV plane, the correction unit 150 performs color correction applied to another subject, for example, color correction by matrix calculation. That is, the correction unit 150 refers to the second correction data stored in the correction data storage unit 160, that is, the matrix coefficient, and the matrix coefficient defined in the second correction data to the RGB pixel value of the pixel. A matrix operation for multiplying is executed. In addition to the color correction described above, the correction unit 150 can also execute γ correction, saturation correction, and the like on the pixel value on which the color correction has been executed.

  On the other hand, when the UV component of the pixel is included in an elliptical area corresponding to the first area on the UV plane, the correction unit 150 performs color correction to be applied to a specific subject, for example, the ratio between the shift amounts Δu and Δv and the spread degree. Color correction is performed to shift the color on the UV plane according to σu_hi and σv_hi.

  More specifically, the correction unit 150 substitutes the value of the U component of the previously selected pixel into the following formula (1), and sets the value of the V component of the pixel to the following formula ( By substituting in 2), the color component of the pixel is shifted.

u_o = u_in + Δu + (u_in-u_in average) * σu_hi (1)
v_o = v_in + Δv + (v_in-v_in average) * σv_hi (2)

  Here, “Δu” in the above equation (1) indicates the shift amount of the U component included in the first correction data, and “σu_hi” is the spread degree of the U component included in the first correction data. Refers to the ratio. Furthermore, “u_in” in the above equation (1) indicates the value of the U component of the pixel of the original image, “u_o” indicates the value of the U component after the shift, and “u_in average” indicates the original image This means the average value of U components of each pixel. In the above equation (2), “Δv” indicates the shift amount of the V component included in the first correction data, and “σv_hi” indicates the ratio of the spread degree of the V component included in the first correction data. Point to. Furthermore, “v_in” in the above equation (2) indicates the value of the V component of the pixel of the original image, “v_o” indicates the value of the V component after the shift, and “v_in average” indicates the original image Means the average value of the V component of each pixel.

  In this way, color correction is performed using the spread degree ratios σu_hi and σv_hi in addition to the shift amounts Δu and Δv, thereby suppressing the color components after the shift from being concentrated in a local region in the color space. As a result, the color of the specific subject is corrected to a color similar to the color recognized under visible light, and is also corrected to a color having a spread like a real object.

  FIG. 5 is a diagram illustrating an example of a color distribution on the UV plane. The vertical axis of the graph shown in FIG. 5 indicates the color difference V, and the horizontal axis indicates the color difference U. In FIG. 5, the distribution of the color components of the pixels included in the first region on the first sample image is plotted with a circle mark on the UV plane, while the first on the second sample image is plotted. The distribution of the color components of the pixels included in the region is plotted with rhombus marks on the UV plane. Further, FIG. 5 shows an ellipse area 51 in which the distribution of coordinates in which the color of each pixel included in the first area among the pixels included in the first sample image is scattered in the color space is approximated to an ellipse. In addition, an ellipse region 52 in which the distribution of coordinates in which the color of each pixel included in the first region among the pixels included in the second sample image is scattered in the color space is approximated to an ellipse is shown.

  As shown in FIG. 5, when color correction is performed using the spread ratios σu_hi and σv_hi in addition to the shift amounts Δu and Δv, the color of the pixel included in the elliptical region 51 on the UV plane among the pixels included in the original image The component is shifted to an elliptical area 52 that is wider than the elliptical area 51. Therefore, the color of a specific subject is not only corrected to a color similar to the color recognized under visible light, but even if it is classified as the same color as the real thing, it is a color in which each other has a spread. Can be corrected.

  Thus, the value of the color component of the pixel shifted on the UV plane according to the shift amounts Δu and Δv and the spread degree ratios σu_hi and σv_hi, together with the luminance Y value of the pixel, the pixel value in the RGB space, that is, RGB Is converted back to the luminance value of each component.

  After color correction of each pixel of the original image is performed in this way, the image after color correction can be output to an arbitrary output destination. For example, the color-corrected image can be displayed on a predetermined display, or can be stored in an auxiliary storage device such as a hard disk or an optical disk, or a removable medium such as a memory card or a USB memory. In addition, the image after color correction may be output to an application program that causes a computer to execute various processes using the image as an input, or the image after color correction may be transmitted to an arbitrary external device via a network. it can.

  The functional units such as the acquisition unit 110, the conversion unit 120, the determination unit 130, and the correction unit 150 described above can be implemented as follows. For example, it can be realized by causing a central processing unit, a so-called CPU, or the like to develop and execute a process that exhibits the same function as each of the above processing units on a memory. These processing units do not necessarily have to be executed by the central processing unit, and may be executed by an MPU or DSP. Each functional unit described above can also be realized by a hard-wired logic such as ASIC or FPGA.

  In addition, as the separation condition storage unit 140 and the correction data storage unit 160, various semiconductor memory elements such as a RAM and a flash memory can be employed as an example. Further, the separation condition storage unit 140 and the correction data storage unit 160 are not necessarily a main storage device and may be an auxiliary storage device. In this case, an HDD, an optical disk, an SSD, or the like can be employed.

[Process flow]
Next, the process flow of the color correction system according to the present embodiment will be described. Here, (1) the setting process executed by the setting apparatus 10 and (2) the first correction data calculation process executed as one procedure of the setting process are described, and then executed by the color correction apparatus 100. (3) The color correction process will be described.

(1) Setting Processing FIG. 6 is a flowchart illustrating a setting processing procedure according to the first embodiment. This process is started when a setting instruction to the color correction apparatus 100 is received as an example. As shown in FIG. 6, when the setting process is started, the first sample image is acquired by the first acquisition unit 11 and the second sample image is acquired by the second acquisition unit 12 (step S101). .

  Subsequently, the specifying unit 13 specifies a first region that forms part or all of the specific subject from the first sample image acquired in step S101, and other than the specific subject from the first sample image. A second region that forms part or all of the other subject is identified (step S102).

  The conversion unit 14 expresses the data format of the first sample image acquired by the first acquisition unit 11 and the second sample image acquired by the second acquisition unit 12 in a space including luminance and chromaticity. (Step S103).

  Thereafter, the correction data setting unit 15 includes the distribution in which the color components of the pixels included in the first region of the first sample image are scattered in the color space and the first region of the second sample image. From the distribution in which the color components of the pixels are scattered in the color space, in addition to the distance related to the mutual distribution, the ratio of the extent of the mutual distribution is calculated as first correction data (step S104).

  In addition, the correction data setting unit 15 calculates the pixel value of the image sensor from the RGB pixel values of the pixels included in the second region among the pixels included in the two sample images of the first sample image and the second sample image. A matrix coefficient that approximates the spectral sensitivity characteristic to the color matching function is calculated as second correction data (step S105).

  Then, the correction data setting unit 15 registers the first correction data calculated in step S104 and the second correction data calculated in step S105 in the correction data storage unit 160 of the color correction apparatus 100, so that a plurality of correction data are set. Each correction data used for the color correction of the seed is set in the color correction apparatus 100 (step S106).

  Thereafter, the separation condition setting unit 16 obtains a color obtained by approximating an ellipse with a coordinate distribution in which the color components of each pixel included in the first region on the first sample image are scattered in the color space. An elliptical area in the space is calculated as a separation condition (step S107).

  After that, the separation condition setting unit 16 registers the separation condition calculated in step S107 in the separation condition storage unit 140 of the color correction device 100, thereby setting the separation condition in the color correction device 100 (step S108). The process is terminated.

  In the flowchart shown in FIG. 6, the case where the process of step S105 is executed after the process of step S104 is illustrated, but these processes can be executed in the reverse order or in parallel. Can also be executed. Further, in the flowchart shown in FIG. 6, the case where the processing of Step S107 to Step S108 is executed after the processing of Step S104 to Step S106 is illustrated, but these processing are also executed in the reverse order. You can also run in parallel.

(2) First Correction Data Calculation Processing FIG. 7 is a flowchart illustrating a procedure of first correction data calculation processing according to the first embodiment. This process is a process corresponding to the process in step S104 shown in FIG. 6, and is started after the process in step S103 is executed.

  As illustrated in FIG. 7, the correction data setting unit 15 includes the first sample image converted from the RGB space to the YUV space in step S103, for each pixel included in the first region specified in step S102. The average value of the color difference is calculated for each U component and V component (step S201).

  Similarly, the correction data setting unit 15 of the second sample image converted from the RGB space to the YUV space in step S103 determines the color difference between the pixels included in the first region specified in step S102. The average value is calculated for each U component and V component (step S202).

  Then, the correction data setting unit 15 determines the average color difference coordinates calculated from the first sample image in step S201 and the average color difference coordinates calculated from the second sample image in step S202. Is calculated for each U component and V component (step S203).

  Further, the correction data setting unit 15 spreads the distribution of color differences of the pixels included in the first region specified in step S102 out of the first sample image converted from the RGB space to the YUV space in step S103. The degree is calculated for each U component and V component (step S204).

  Similarly, the correction data setting unit 15 of the second sample image converted from the RGB space to the YUV space in step S103 has a color difference distribution of the pixels included in the first area specified in step S102. The degree of spread is calculated for each U component and V component (step S205).

  In addition, the correction data setting unit 15 sets a ratio of the spread degree between the spread degree calculated from the first sample image in step S204 and the spread degree calculated from the second sample image in step S205. Calculation is performed for each of the U component and the V component (step S206), and the process ends.

  Through these series of procedures, the shift amounts Δu and Δv and the spread degree ratios σu_hi and σv_hi are calculated as the first correction data.

  In the flowchart shown in FIG. 7, the case where the process of step S202 is executed after the process of step S201 is illustrated, but these processes can be executed in the reverse order or in parallel. Can also be executed. In the flowchart shown in FIG. 7, the case where the process of step S <b> 205 is executed after the process of step S <b> 204 is illustrated, but these processes can be executed in the reverse order or in parallel. Can also be executed. Furthermore, in the flowchart shown in FIG. 7, the case where the processing of step S204 to step S206 is executed after the processing of step S201 to step S203 is illustrated, but these processing are also executed in the reverse order. You can also run in parallel.

(3) Color Correction Processing FIG. 8 is a flowchart illustrating the procedure of color correction processing according to the first embodiment. As an example, this color correction process is started when an original image to be subjected to color correction is designated. As shown in FIG. 8, when an original image to be subjected to color correction is designated, the acquisition unit 110 acquires the original image (step S301). Subsequently, the conversion unit 120 converts the data format of the original image acquired in step S301 into a data format expressed in a space including luminance and chromaticity (step S302).

  Then, the determination unit 130 selects one of the pixels included in the original image converted into the color space in step S302 (step S303). Subsequently, the determination unit 130 is selected from the original image in step S303 with reference to the separation condition stored in the separation condition storage unit 140, that is, the elliptical region on the UV plane corresponding to the first region described above. It is determined whether or not the UV component of the pixel is included in an elliptical area corresponding to the first area on the UV plane (step S304).

  Here, when the UV component of the pixel is included in an elliptical region corresponding to the first region on the UV plane (step S304 Yes), the pixel has a color close to an achromatic color and has a large amount of infrared light received. Since it can be estimated that it is a pixel, the probability that it is a pixel that forms the specific subject is increased. In this case, the correction unit 150 converts the UV component of the pixel into the UV plane according to the shift amounts Δu and Δv defined in the first correction data stored in the correction data storage unit 160 and the spread degree ratios σu_hi and σv_hi. After the above shift, color correction is performed to reversely convert the pixel values into the RGB space (step S305).

  On the other hand, when the UV component of the pixel is not included in the elliptical area corresponding to the first area on the UV plane (step S304 No), the correction unit 150 performs the following process. That is, the correction unit 150 refers to the second correction data stored in the correction data storage unit 160, that is, the matrix coefficient, and the matrix coefficient defined in the second correction data to the RGB pixel value of the pixel. The color correction for another subject that executes the matrix operation for multiplying by is performed (step S306).

  Thereafter, the processes from step S303 to step S306 are repeatedly executed until all the pixels included in the original image are selected (No in step S307). Then, when all the pixels included in the original image are selected (Yes in step S307), the correction unit 150 outputs the corrected image to a predetermined output destination (step S308), and ends the process.

  In the flowchart of FIG. 8, the case where color correction is performed in the order in which the pixels included in the original image are selected in step S303 is illustrated, but it goes without saying that the color correction for each pixel can be performed in parallel.

[One aspect of effect]
As described above, the color correction system 1 according to the present embodiment corresponds to a specific subject having a high spectral reflectance of infrared light among images captured including light having wavelengths of visible light and infrared light. The color component of the pixel to be shifted is shifted over a wider range than the range in which the color of the subject is distributed in the color space. In this way, when color correction is performed on a specific subject such as a plant leaf or a chemical fiber cloth, the color of the specific subject is prevented from being concentrated in a local area on the color space. As a result, the color of a specific subject is corrected to a color that is similar to the color recognized under visible light, and even if it is classified as the same color as the real thing, it is corrected to a color in which each other has a spread. Is done. Therefore, it becomes possible to improve color reproducibility.

  Although the embodiments related to the disclosed apparatus have been described above, the present invention may be implemented in various different forms other than the above-described embodiments. Therefore, another embodiment included in the present invention will be described below.

[Input image]
In the first embodiment, as an example of a photographing form in which light corresponding to the wavelengths of visible light and infrared light is imaged on the image sensor, an image photographed without an infrared cut filter is input However, the image capturing mode is not limited to this. For example, an image photographed with an optical filter that transmits the visible light region and the bandwidth of the infrared wavelength region to be used is inserted as the first sample image, and the setting process shown in FIG. 6 is applied. It is also possible to use the image as an original image and apply the color correction processing shown in FIG.

[Color Correction Blend]
In the first embodiment, the case where the two types of color correction are performed exclusively depending on whether the UV component of the pixel is included in the elliptical region corresponding to the first region on the UV plane is exemplified. Two types of color correction can also be mixed. For example, when the color component of a pixel is included in an elliptical area, the color correction apparatus 100 can blend color shift correction and color correction by matrix calculation using the following equation (3).

  RGB = r′g’b ′ × w1 + rgb × (1-w1) (3)

  “R ′”, “g ′”, and “b ′” shown in the above equation (3) are RGB values after color shift correction, and “r”, “g” shown in the above equation (3). "And" b "are RGB values after color correction by matrix calculation. Also, “w1” shown in the above equation (3) indicates the probability that the pixel is a specific subject, and for example, an arbitrary value of 0 ≦ w1 ≦ 1 can be set. Thus, for example, the rate of color shift correction is increased as the probability that a pixel is a specific subject increases, and the rate of color correction by matrix calculation is increased as the probability that a pixel is a specific subject is decreased. You can increase the color correction blend. Here, the case where the blend process is executed in the RGB space is illustrated, but the application range of the blend process is not limited to the image in the RGB space, and the blend process is performed in another color space such as YUV or L * a * b. It is also possible to convert to RGB after executing.

[Extension to 3D]
In the first embodiment, the case where the color difference U and the color difference V are shifted on the two-dimensional UV plane is exemplified, but the application range is not limited to the shift on the two-dimensional plane. For example, in the color correction system 1, the colors of the pixels included in the original image can be shifted in a three-dimensional color space of luminance Y, color difference U, and color difference V.

  Specifically, the setting device 10 further calculates the average value of the Y component together with the average value of the U component and the average value of the V component in the process of step S201 and the process of step S202 illustrated in FIG. After that, the setting device 10 calculates the coordinates of the average value of the luminance Y calculated from the first sample image in the process of step S203 and the coordinates of the average value of the luminance Y calculated from the second sample image. By calculating the distance, the shift amount Δy related to the luminance Y is further calculated. Further, the setting device 10 further calculates the spread degree of the Y component together with the spread degree of the U component and the spread degree of the V component in the process of step S204 and the process of step S205. In addition, the setting device 10 determines that the luminance Y is between the spread degree of the luminance Y calculated from the first sample image and the spread degree of the luminance Y calculated from the second sample image in the process of step S206. The ratio σy_hi of the extent of spread is calculated. Through these series of procedures, the shift amounts Δu, Δv and Δy and the spread degree ratios σu_hi, σv_hi and σy_hi are set in the color correction apparatus 100 as the first correction data.

  Under such settings, the color correction apparatus 100 substitutes the value of the U component of the pixel selected in step S303 shown in FIG. 8 into the above equation (1) and the value of the V component of the pixel. Is substituted into the above equation (2), and further, the value of the Y component of the pixel is substituted into the following equation (4), thereby shifting the color component of the pixel in the three-dimensional space.

  y_o = y_in + Δy + (y_in-y_in average) * σy_hi (4)

  FIG. 9 is a diagram illustrating an example of a color distribution in the YUV space. In the graph shown in FIG. 5, three axes of luminance Y, color difference V, and color difference U are shown. In FIG. 9, the distribution of the color components of the pixels included in the first region on the first sample image is plotted with a circle mark on the YUV space, while the first on the second sample image is plotted. The distribution of the color components of the pixels included in the area is plotted with rhombus marks on the YUV space. Further, FIG. 9 shows an elliptic sphere 91 in which the distribution of coordinates in which the color of each pixel included in the first region among the pixels included in the first sample image is scattered in the color space is approximated to an elliptic sphere. And an elliptical sphere 92 in which the distribution of coordinates in which the colors of the pixels included in the first region among the pixels included in the second sample image are scattered in the color space is approximated to an elliptical sphere. Yes.

  As shown in FIG. 5, when color correction is further performed using the shift amount Δy and the spread degree ratio σy_hi, the color components of the pixels included in the elliptical region 51 of the UV plane among the pixels included in the original image are the elliptical spheres. Shifted to an elliptical sphere 92 wider than 91. At this time, color shift correction can be performed in consideration of luminance that increases by receiving light corresponding to the wavelength of infrared light in addition to light corresponding to the wavelength of visible light. Therefore, the color reproducibility can be further improved.

[Spreading direction]
In the first embodiment, the ratio of the spread degree is used when performing the color correction. However, the spread direction difference and the difference in the spread direction are further set as the first correction data and used for the color correction. it can.

  That is, among the pixels included in the first sample image, an elliptical region in which the distribution of coordinates in which the color of each pixel included in the first region is scattered in the color space is approximated to an ellipse, and the second sample image Stipulated by the mutual short axis and long axis between the ellipse area in which the distribution of coordinates in which the color of each pixel included in the first area is scattered in the color space is approximated to an ellipse. The directions are not necessarily the same.

  From this, the setting device 10 has an elliptical area defined from the first sample image and an elliptical area defined from the second sample image in the direction defined by the short axis and the long axis. Find the relative angle. In addition, the setting device 10 can add the above relative angle as the difference Δθ in the spreading direction to the first correction data. Under such setting of the first correction data, the color correction apparatus 100 moves the color component of the pixel according to the shift amounts Δu and Δv and moves the color component of the pixel according to the spread degree ratios σu_hi and σv_hi. In addition, the color component of the pixel is rotated on the UV plane according to the difference Δθ in the spreading direction. When the difference in the spread direction is taken into account in this way, the color reproducibility can be further improved as a result of the color component after the shift approaching the real thing.

[Movement transformation matrix]
In this embodiment, the case where the color shift correction is performed by setting three correction amounts of the shift amounts Δu and Δv, the spread degree ratios σu_hi and σv_hi, and the spread direction difference Δθ is illustrated. It is also possible to obtain a movement conversion matrix including one correction amount and set the movement conversion matrix as the first correction data.

  For example, when the number of pixels included in the first region on the first sample image is n, the U component of n pixels included in the first region on the first sample image and The V component can be expressed as two-dimensional vector data A as shown in the following formula (5). In this case, since the number of pixels included in the first region on the second sample image is also n, the U component and V included in the n pixels included in the first region on the second sample image. The component can be expressed as two-dimensional vector data B as shown in the following equation (6).

Among these, the average value for each of the U component and the V component regarding n pixels included in the two-dimensional vector data A can be expressed by the following equation (7). covariance matrix sigma _a vector data a dimension can be expressed by the following equation (9). On the other hand, the average value for each of the U component and the V component regarding n pixels included in the two-dimensional vector data B can be expressed by the following equation (8). The covariance matrix Σ _b of the vector data B can be expressed by the following equation (10).

In this case, the eigenvalue D _a and eigenvector V _a of the two-dimensional vector data A are expressed by the following equation (11) using a function eig used for calculating eigenvalues and eigenvectors by numerical analysis software or the like. It can be calculated. Similarly, the eigenvalues _{D b} and eigenvectors _{V b} of the two-dimensional vector data B, using function eig, can be calculated according to Equation (12) below. Although the case where the function eig is used for the calculation of the eigenvalue and the eigenvector is illustrated here, other functions may be used.

After the eigenvalue D _a and the eigenvector V _a of the two-dimensional vector data A are calculated in this way, the movement transformation matrix A _out of the two-dimensional vector data A is calculated as the following equation (13). Further, the movement transformation matrix B _out of the two-dimensional vector data B is calculated as in the following equation (14). Thereafter, the correction matrix out is obtained from the ratio of the movement conversion matrix A _out and the movement conversion matrix B _{out according} to the following equation (15).

out = B _out / A _out (15)

  When such a correction matrix out is set as the first correction data, the color correction apparatus 100 inputs the two-dimensional vector data in related to the pixels included in the elliptical area corresponding to the first area in the original image. By performing vector calculation with the correction matrix out, it is possible to obtain a color reproducibility equivalent to the color correction using the spread direction described in the above “spread direction” section.

  10A and 10B are diagrams illustrating an example of a color distribution on the UV plane. 10A and 10B, the vertical axis indicates the color difference V, and the horizontal axis indicates the color difference U. In FIG. 10A, the distribution of the two-dimensional vector data A is plotted with a circle mark, while the distribution of the two-dimensional vector data B is plotted with a triangular mark “Δ” on the UV plane. Further, in FIG. 10B, while the distribution of the two-dimensional vector data A is plotted with a circle mark, the calculation result of the two-dimensional vector data A and the correction matrix out, that is, the corrected two-dimensional vector The distribution of data is plotted with an inverted triangle mark “▽” on the UV plane.

  As shown in FIGS. 10A and 10B, it can be said that the distribution of the target two-dimensional vector data B and the corrected distribution of the two-dimensional vector data are close to each other. For this reason, even when color correction of the two-dimensional vector data in is performed using the correction matrix out, color reproducibility equivalent to the color correction using the spread direction described in the above-mentioned “spread direction” is obtained. Can be estimated. Note that, here, the case of using two-dimensional vector data is illustrated, but the luminance Y may be further expanded as three-dimensional vector data to set the correction matrix out and perform color correction using the correction matrix out. it can.

[Environmental condition]
In the first embodiment, the shift amounts Δu and Δv and the spread degree ratios σu_hi and σv_hi are exemplified as the first correction data, but the shift amounts Δu, Correction amounts such as a shift amount Δv, a spread degree ratio σu_hi, and a spread degree ratio σv_hi can also be set.

  FIG. 11 is a diagram illustrating an example of the first correction data. FIG. 11 shows, as an example of the environmental condition, first correction data in which a correction amount is associated with each illuminance measured in the imaging range of the camera. In the example of FIG. 11, when the illuminance measured in the imaging range of the camera that captures the original image is included in the illuminance range corresponding to illuminance 1, the shift amount 1 and the spread degree ratio 1 are used for color correction. It is done. When the illuminance measured in the imaging range of the camera that captures the original image is included in the illuminance range corresponding to the illuminance 2, the shift amount 2 and the spread degree ratio 2 are used for color correction. When the illuminance measured in the imaging range of the camera that captures the original image is included in the illuminance range corresponding to the illuminance 3, the shift amount 3 and the spread degree ratio 3 are used for color correction. Accordingly, the correction amount used for color correction can be changed according to the amount of illuminance measured in the imaging range of the camera. In FIG. 11, the illuminance is illustrated as an example of the environmental condition. However, the environmental condition is not limited to this, and the environment such as the time zone when the original image is captured, the type of weather that can be acquired by sensing or browsing, and the like. It can also be a condition.

  When the correction amount is set for each environmental condition in this way, a part of the processing procedure shown in FIG. 8 may be changed, and the color correction processing may be executed according to the procedure shown in FIG. FIG. 12 is a flowchart illustrating a procedure of color correction processing according to the second embodiment. In FIG. 12, the same step number is assigned to the same procedure as the process shown in FIG. As shown in FIG. 12, after the process of step S301 shown in FIG. 8 is executed, the color correction apparatus 100 detects an environmental condition (step S401). For example, when the environmental condition is illuminance, the illuminance may be acquired from an illuminance sensor provided around the camera. Subsequently, the color correction apparatus 100 acquires a correction amount corresponding to the environmental condition detected in step S401 among the correction amounts stored as the first correction data in the correction data storage unit 160 (step S402). After the correction amount used for the color correction in step S305 is acquired by the processing in step S402, the processing in steps S302 to S308 shown in FIG. 8 may be executed.

[Distribution and integration]
In addition, each component of each illustrated apparatus does not necessarily have to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. For example, the acquisition unit 110, the conversion unit 120, the determination unit 130, or the correction unit 150 may be connected as an external device of the color correction apparatus 100 via a network. In addition, another device has the acquisition unit 110, the conversion unit 120, the determination unit 130, or the correction unit 150, and the functions of the color correction device 100 described above are realized by cooperation through a network connection. Also good.

[Color correction program]
The various processes described in the above embodiments can be realized by executing a prepared program on a computer such as a personal computer or a workstation. In the following, an example of a computer that executes a color correction program having the same function as that of the above embodiment will be described with reference to FIG.

  FIG. 13 is a diagram illustrating a hardware configuration example of a computer that executes a color correction program according to the first and second embodiments. As illustrated in FIG. 13, the computer 1000 includes an operation unit 1100 a, a speaker 1100 b, a camera 1100 c, a display 1200, and a communication unit 1300. Further, the computer 1000 includes a CPU 1500, a ROM 1600, an HDD 1700, and a RAM 1800. These units 1100 to 1800 are connected via a bus 1400.

  As illustrated in FIG. 13, the HDD 1700 stores a color correction program 1700 a that exhibits the same functions as those of the acquisition unit 110, the conversion unit 120, the determination unit 130, and the correction unit 150 described in the first embodiment. The color correction program 1700a may be integrated or separated as with the components of the acquisition unit 110, the conversion unit 120, the determination unit 130, and the correction unit 150 illustrated in FIG. In other words, the HDD 1700 does not necessarily store all the data shown in the first embodiment, and the HDD 1700 may store data used for processing.

  Under such an environment, the CPU 1500 reads the color correction program 1700a from the HDD 1700 and develops it in the RAM 1800. As a result, the color correction program 1700a functions as a color correction process 1800a as shown in FIG. The color correction process 1800a expands various data read from the HDD 1700 in an area allocated to the color correction process 1800a in the storage area of the RAM 1800, and executes various processes using the expanded data. For example, as an example of processing executed by the color correction process 1800a, the processing shown in FIGS. 8 and 12, the processing described in the second embodiment, and the like are included. Note that the CPU 1500 does not necessarily operate all the processing units described in the first embodiment, and it is only necessary to virtually realize a processing unit corresponding to a process to be executed.

  Note that the color correction program 1700a is not necessarily stored in the HDD 1700 or the ROM 1600 from the beginning. For example, the color correction program 1700a is stored in a “portable physical medium” such as a flexible disk inserted into the computer 1000, so-called FD, CD-ROM, DVD disk, magneto-optical disk, IC card or the like. The computer 1000 may acquire and execute the color correction program 1700a from these portable physical media. In addition, the color correction program 1700a is stored in another computer or server device connected to the computer 1000 via a public line, the Internet, a LAN, a WAN, etc., and the computer 1000 acquires the color correction program 1700a from these. May be executed.

DESCRIPTION OF SYMBOLS 1 Color correction system 10 Setting apparatus 11 1st acquisition part 12 2nd acquisition part 13 Specification part 14 Conversion part 15 Correction data setting part 16 Separation condition setting part 100 Color correction apparatus 110 Acquisition part 120 Conversion part 130 Determination part 140 Separation condition memory | storage 150 correction unit 160 correction data storage unit

Claims (5)

On the computer,
Processing to acquire images,
A process of converting an image into a component of a color space including luminance and chromaticity;
A process of determining whether or not a pixel included in the image converted into the color space component is a pixel corresponding to a subject having a spectral reflectance of infrared light higher than that of visible light;
A first sample image captured in a state where the subject receives light corresponding to wavelengths of visible light and infrared light, and a first sample image captured in a state where the subject receives light corresponding to the wavelength of visible light. The color component of the pixel corresponding to the subject is represented on the color space according to the correction data in which the ratio of the distance and spread degree relating to the chromaticity distribution of the pixel corresponding to the subject is correlated with the two sample images. A color correction program that executes a process of executing color correction for shifting. The correction data is further associated with a ratio of a distance and a spread degree regarding a luminance distribution of pixels corresponding to the subject between the first sample image and the second sample image,
The process for executing the color correction performs color correction for shifting a luminance component and a chromaticity component of a pixel corresponding to the subject in a color space according to the correction data. Color correction program. The correction data is further associated with a difference in spreading direction regarding the distribution of chromaticity of pixels corresponding to the subject between the first sample image and the second sample image,
The process for executing the color correction further executes color correction for rotating a color component of a pixel corresponding to the subject in a color space in accordance with a difference in spread direction included in the correction data. Item 3. A color correction program according to item 1 or 2. Computer
Processing to acquire images,
A process of converting an image into a component of a color space including luminance and chromaticity;
A process of determining whether or not a pixel included in the image converted into the color space component is a pixel corresponding to a subject having a spectral reflectance of infrared light higher than that of visible light;
A first sample image captured in a state where the subject receives light corresponding to wavelengths of visible light and infrared light, and a first sample image captured in a state where the subject receives light corresponding to the wavelength of visible light. The color component of the pixel corresponding to the subject is represented on the color space according to the correction data in which the ratio of the distance and spread degree relating to the chromaticity distribution of the pixel corresponding to the subject is correlated with the two sample images. A color correction method comprising: performing a process of performing color correction for shifting. A first sample image captured in a state in which a subject having a spectral reflectance of infrared light higher than that of visible light receives light corresponding to wavelengths of visible light and infrared light, and the subject is visible Correction data in which a ratio of a distance and a spread degree relating to a chromaticity distribution of a pixel corresponding to the subject is associated with a second sample image captured in a state of receiving light corresponding to a wavelength of light. A storage unit for storing;
An acquisition unit for acquiring images;
A conversion unit for converting an image into a component of a color space including luminance and chromaticity;
A determination unit that determines whether a pixel included in the image converted into the color space component is a pixel corresponding to a subject whose spectral reflectance of infrared light is higher than that of visible light;
A color correction apparatus comprising: a correction unit that performs color correction for shifting a color component of a pixel corresponding to the subject in a color space according to the correction data.