JP6531522B2 - Color correction program, color correction method and color correction apparatus - Google Patents

Description

  The present invention relates to a color correction program, a color correction method, and a color correction apparatus.

  Cameras are used in a wide variety of applications. For example, the camera is implemented as a surveillance camera or an on-vehicle camera, or implemented as an imaging device of an image used for various biometrics authentication, gaze detection using corneal reflection, and the like.

  For example, in the case of a surveillance camera or a car-mounted camera, it may be used day and night. In this case, during the daytime, by mounting an infrared cut filter for removing infrared light, visible light that can be recognized by the human eye is formed on the light receiving element, while at night or in the dark, the infrared cut filter is removed In some cases, infrared light, such as near-infrared light, may be used as illumination. In addition, in the case of an information processing terminal equipped with functions such as biometric authentication and gaze detection, it is possible to capture an image used for biometric authentication and gaze detection and an image used for other purposes by a single camera. There are also trends that aim to reduce the scale and cost.

  As described above, when the imaging element receives visible light and infrared light, the image captured by the camera is different from the color perceived by the human eye under the visible light. From this, when visible light is received by correcting the color of the image captured by the imaging device that receives visible light and infrared light, in other words, a color equivalent to the case where infrared light is not received Techniques for reproducing taste have been proposed.

  As an example, there is a method of correcting an input image by matrix operation. In this method, matrix coefficients used for color correction are calculated from two images captured by different image input systems, for example, a color chart including a plurality of colors, for example, 24 colors. That is, the image pickup device has two elements, an image captured with the infrared cut filter incorporated in the light path and an image captured with the infrared cut filter not incorporated in the light path. Matrix coefficients that approximate spectral sensitivity characteristics to color matching functions are obtained by the method of least squares. Color correction of the input image is performed by an operation using such matrix coefficients.

  However, when color correction is performed by the above matrix operation, the color reproducibility is sufficiently exhibited for a specific object having a high spectral reflectance of infrared light, such as a plant leaf or chemical fiber cloth. It may not be possible. For example, in the case of plant leaves, due to the effect of chlorophyll contained in the leaves, the spectral reflectance increases at a wavelength corresponding to near infrared light exceeding the upper limit of visible light. For this reason, the influence of the tint that the infrared light contained in the ambient light such as natural light and illumination gives to the image is larger in the leaves of the plant than in subjects other than the leaves of the plant. Therefore, even if the color correction is performed by the above matrix operation, the color of the leaves of the plant can not be sufficiently reproduced as compared with other objects. As a result, the color of the leaf recognized as green under visible light is recognized as a color different from the green of the color chart also recognized as green in visible light.

  In order to suppress such deterioration in color reproducibility, the following correction data generation apparatus has also been proposed. In this correction data generation device, the first sample image in which the specific subject is imaged in a state in which the infrared cut filter is not attached, and the in which the specific subject is imaged in a state in which the infrared cut filter is attached Correction data for correcting the color of the specific subject is generated using the two sample images.

  In such a correction data generation device, the knowledge that colors included in a specific subject are concentrated on a local area close to an achromatic color in the color space is used. Under such knowledge, the correction data generation apparatus designates an area in which the specific subject appears on the first sample image, whereby an elliptical area in which colors included in the specific subject are distributed in the color space is determined. Set Then, the correction data generation apparatus calculates, as a correction amount, a shift amount for shifting the elliptical area set using the first sample image to an elliptical area in which the color of a specific subject included in the second sample image is distributed. Do. Furthermore, depending on whether the color of the pixel included in the input image is included in the elliptical area on the color space set by using the first sample image, the portion having a high spectral reflectance of infrared light And separated into other parts. That is, while the correction for shifting the color is performed using the above-mentioned correction data in the part where the spectral reflectance of infrared light is high, the color correction is performed in the other parts by the above matrix operation. .

JP, 2013-150269, A JP, 2014-200045, A JP 2005-328845 A JP, 2013-128259, A JP, 2006-094112, A

Toyotaka Yoshitaka, Kuno Tetsuya, Ishida Shozo, Sugiura Hiroaki "Study of color imaging capable near infrared imaging device" Journal of the Institute of Image Information and Television Engineers Vol. 64, no. 1, pp. 101-110 2010

  However, in the above-described techniques, color reproducibility is naturally limited, as described below.

  That is, the above-described correction data generation device utilizes the knowledge that the colors contained in a specific subject are concentrated in a local region close to an achromatic color on the color space of the first sample image. However, even on the second sample image described above, the colors contained in a specific subject do not necessarily concentrate on a local region, but in fact, they are distributed over a wider range than the distribution on the first sample image. Nevertheless, in the above correction data generation apparatus, color correction is performed by uniformly applying the above shift amount, so that the color of a specific subject is concentrated on a local region even after color correction. Become. As a result, even if the color of a specific subject is corrected to a color similar to the color recognized under visible light, it is corrected to a color that does not have spread as real, so color reproducibility is improved. There is a limit.

  In one aspect, the present invention aims to provide a color correction program, a color correction method, and a color correction device capable of enhancing color reproducibility.

  In one aspect, the computer includes a process of acquiring an image, a process of converting the image into components of a color space including luminance and chromaticity, and pixels included in the image converted into the components of the color space are infrared light A process of determining whether or not the pixel corresponds to a subject whose spectral reflectance is higher than that of visible light, and in a state in which the subject receives light corresponding to wavelengths of visible light and infrared light Regarding the distribution of the chromaticity of the pixel corresponding to the subject between the captured first sample image and the second sample image captured in a state where the subject receives light corresponding to the wavelength of visible light According to the correction data associated with the ratio of the distance and the expansion degree, processing of executing color correction for shifting the color component of the pixel corresponding to the subject on the color space is executed.

  The color reproducibility can be enhanced.

FIG. 1 is a block diagram showing a functional configuration of each device included in the color correction system according to the first embodiment. FIG. 2A is a diagram showing an example of a sample image. FIG. 2B is a diagram showing an example of a sample image. FIG. 3A is a diagram showing an example of luminance values possessed by each component of RGB. FIG. 3B is a diagram showing an example of luminance values possessed by each component of RGB. FIG. 4A is a diagram showing an example of a histogram. FIG. 4B is a diagram showing an example of a histogram. FIG. 5 is a view showing an example of the color distribution on the UV plane. FIG. 6 is a flowchart of the setting process according to the first embodiment. FIG. 7 is a flowchart of the first correction data calculation process according to the first embodiment. FIG. 8 is a flowchart of the color correction process according to the first embodiment. FIG. 9 is a diagram showing an example of color distribution in YUV space. FIG. 10A is a diagram showing an example of color distribution on the UV plane. FIG. 10B is a diagram showing an example of color distribution on the UV plane. FIG. 11 is a diagram showing an example of the first correction data. FIG. 12 is a flowchart of the color correction process according to the second embodiment. FIG. 13 is a diagram illustrating an example of a hardware configuration of a computer that executes a color correction program according to the first embodiment and the second embodiment.

  Hereinafter, a color correction program, a color correction method, and a color correction apparatus according to the present application will be described with reference to the accompanying drawings. Note that this embodiment does not limit the disclosed technology. And each Example can be suitably combined in the range which does not make processing contents contradictory.

[Configuration of color correction system]
FIG. 1 is a block diagram showing a functional configuration of each device included in the color correction system according to the first embodiment. The color correction system 1 shown in FIG. 1 implements a color correction process that corrects the color of an image captured including light corresponding to the wavelengths of visible light and infrared light in accordance with human color vision characteristics. . Here, as an example, a case where a filter for removing infrared light, so-called infrared (infrared) cut filter is not attached is illustrated, but as described later, one of the wavelengths corresponding to infrared light It goes without saying that the application range also includes the case where an image captured in a state where the optical filter that transmits the light of the wavelength range of the part is attached is input.

  Here, as part of the above color correction process, a color component of a pixel corresponding to a specific subject having a high spectral reflectance of infrared light in an image captured including light of wavelengths of visible light and infrared light Is shifted to a wider range than the range in which the color of the subject is distributed on the color space. As described above, when color correction is performed on a specific subject such as a plant leaf or chemical fiber cloth, the color of the specific subject is prevented from being concentrated on a local area in the color space. As a result, the color of a specific subject is corrected to a color similar to the color recognized under visible light, and even if it is a classification of the same color as in the real thing, it is corrected to a color having spread among them. Be done. As one aspect, such color correction processing realizes improvement in color reproducibility.

  As shown in FIG. 1, the color correction system 1 includes a setting device 10 and a color correction device 100. The setting device 10 and the color correction device 100 are communicably connected via a predetermined network. As such a network, any type of communication network such as LAN (Local Area Network) or VPN (Virtual Private Network) can be adopted, such as the Internet regardless of wired or wireless.

  The setting device 10 is a device that sets various parameters related to color correction.

  In one embodiment, the setting device 10 sets correction data related to color correction to the color correction device 100, and a specific subject whose spectral reflectance of infrared light is higher than other wavelength ranges and others. Set separation conditions to separate from the subject. Although FIG. 1 shows an example in which various parameters relating to color correction are set through the setting device 10, they may be preset in the color correction device 100 before shipping of the color correction device 100 or the like. The setting device 10 and the color correction device 100 may not necessarily be constructed separately, and the color correction device 100 may also have the function of the setting device 10.

  The color correction apparatus 100 is a computer that implements the above-described color correction processing.

  In one embodiment, the color correction apparatus 100 can be implemented by installing a color correction program that executes the above-described color correction processing as packaged software or online software in a desired computer. For example, the information processing apparatus can function as the color correction apparatus 100 by causing the information processing apparatus to execute the color correction program. The information processing apparatus referred to here includes not only desktop type or laptop type personal computers, but also mobile communication terminals such as smart phones, mobile phones and PHS (Personal Handyphone System), and slates such as PDA (Personal Digital Assistants). Terminals are included in that category. In addition, the terminal device used by the user may be a client, and may be implemented as a server device that provides the client with a service related to the color correction process. For example, the color correction apparatus 100 is implemented as a server apparatus that provides a color correction service that receives an image or identification information associated with the image and outputs the execution result of the color correction process on the image. . In this case, the color correction apparatus 100 may be implemented as a Web server, or may be implemented as a cloud that provides services related to the above-described color correction process by outsourcing.

[Configuration of setting device 10]
Subsequently, a functional configuration of the setting device 10 according to the present embodiment will be described. As shown in FIG. 1, the setting device 10 includes a first acquisition unit 11, a second acquisition unit 12, a specification unit 13, a conversion unit 14, a correction data setting unit 15, and a separation condition setting unit 16. Have. The setting apparatus 10 may have various functional units of a known computer, for example, various functional units such as various input devices and audio output devices, in addition to the functional units shown in FIG.

  Among these, each of the first acquisition unit 11 and the second acquisition unit 12 is a processing unit that acquires a sample image that is a material for generating various parameters related to color correction. Hereinafter, the sample image acquired by the first acquisition unit 11 will be referred to as “first sample image”, and the sample image acquired by the second acquisition unit 12 will be referred to as “second sample image”. It may be written as

  In one embodiment, the first acquisition unit 11 and the second acquisition unit 12 acquire sample images captured by a camera equipped with an imaging device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). Can. The first acquisition unit 11 and the second acquisition unit 12 acquire the sample image stored in an auxiliary storage device such as a hard disk or an optical disk, or a removable medium such as a memory card or a USB (Universal Serial Bus) memory. It can also be done. Besides, the first acquisition unit 11 and the second acquisition unit 12 can also acquire a sample image by receiving from an external device via a network.

  2A and 2B are diagrams showing an example of a sample image. Here, as an example, the following description will be made on the assumption that an image in which each pixel has a luminance value of RGB is captured by an imaging device having three light receiving elements of RGB. Among these, the red component R receives the red wavelength range and the infrared wavelength range of visible light wavelengths, and the green component G has the green wavelength range and infrared light of visible light wavelengths. The blue component B receives the wavelength range of blue and the wavelength range of infrared light among the wavelengths of visible light.

  The first sample image 20A shown in FIG. 2A and the second sample image 20B shown in FIG. 2B are common in that they are images obtained by photographing the same target at the same position and the same angle of view by the same camera. However, parts of the image input system from which each other's images are captured are different. That is, while the first sample image 20A shown in FIG. 2A is an image taken without the infrared cut filter attached, the second sample image 20B shown in FIG. 2B has the infrared cut filter attached. The difference is that the image is taken in the state. Therefore, the first sample image 20A includes the influence of light having the wavelength of infrared light in addition to light having the wavelength of visible light, while the second sample image 20B includes infrared light. The influence of light with a wavelength of Here, as an example of the imaging mode of the first sample image, the case where imaging is performed without the infrared cut filter is illustrated, but the bandwidth of the visible light region and the infrared wavelength region to be used, for example An image captured in a state in which an optical filter that transmits a part of the near infrared light wavelength is inserted may be used as the first sample image.

  When the first sample image 20A and the second sample image 20B are photographed, the photographing is performed, for example, in a state in which the specific subject described above is included in the imaging range. An example of such a specific subject is a plant containing chemical substances such as chlorophyll in leaves. When a plant is photographed in this manner, the shapes of the plant image 21A and the plant image 21B are similarly reflected between the first sample image 20A and the second sample image 20B, while the colors are reflected. Is different. That is, in the case of a plant, the spectral reflectance of infrared light becomes larger than that of other objects due to the influence of chlorophyll contained in the leaves of the plant. Therefore, while the plant image 21B looks green as well as the color recognized by the human eye, the plant image 21A has a greyish blue porcelain or greyish pale green color due to the influence of the reflected infrared light. It looks like

  Furthermore, shooting is performed in a state where the subject other than the above-described specific subject is included in the imaging range. An example of such another subject is a color chart in which 24 different colors are arranged in 4 rows and 6 columns as shown in FIGS. 2A and 2B. By including such a color chart in the imaging range, a correction amount used when performing color correction on an object other than a specific object, for example, the matrix coefficient described above in the background section is set in the processing unit in the subsequent stage It becomes possible.

  In addition, although the plant was illustrated as an example of said specific subject here, if it is an object from which a spectral reflectance rises with a predetermined wavelength, for example, about 700 nm, besides this, things other than a plant will be mentioned. It does not matter if it is a subject. That is, as long as the spectral reflectance is higher than the spectral reflectance of the wavelength range corresponding to visible light near the upper limit of visible light or in the wavelength region of near infrared light, any object such as cloth of chemical fiber It does not matter.

  The identifying unit 13 generates a first area forming a part or all of a specific subject on each sample image and a second area forming a part or all of another subject other than the specific subject on each sample image. It is a processing unit that specifies an area.

  In one embodiment, the identification unit 13 may display at least one of the first sample image and the second sample image on the display unit of the setting device 10 or the display unit of the external device connected to the setting device 10. Is displayed. Then, the specification unit 13 causes a range including one or more pixels forming a plant leaf to be specified on the sample image displayed on the display unit through an input unit (not shown). For example, the specification unit 13 can specify the first region corresponding to the leaf of the plant on the sample image by receiving the range specification of the drag and drop via the pointing device. Further, the specifying unit 13 specifies a second area on which a part or all of the color chart is to be formed on the sample image in the same manner as in the case of specifying the first area described above. Here, although the case where the first area and the second area are specified by the operation input is illustrated, the first area and the second area may not necessarily be specified by the operation input. For example, by applying image recognition such as template matching to a sample image, a region having a shape similar to that of a plant or a plant leaf is specified as a first region among regions labeled on the sample image. An area having a similar shape to the color chart may be specified as the second area.

  The conversion unit 14 is a processing unit that converts the data format of the sample image.

  In one embodiment, the conversion unit 14 converts the data format of the first sample image acquired by the first acquisition unit 11 and the second sample image acquired by the second acquisition unit 12 in a space including luminance and color difference. Convert to the data format to be represented. For example, the conversion unit 14 is a pixel represented by luminance Y and two color differences from RGB pixel values in which the pixel value of each pixel included in the first sample image and the second sample image is represented in RGB space. Convert to a value Although the case of converting each sample image to YUV space is exemplified here, the color space of conversion destination is not limited to YUV space, and the sample image is converted to another color space, for example, L * a * b space. It does not matter as well.

  The correction data setting unit 15 is a processing unit that sets correction data regarding color correction in the color correction apparatus 100.

  As one embodiment, the correction data setting unit 15 includes first correction data used for color correction regarding the specific subject described above, and second correction data used for color correction regarding another subject other than the specific subject described above. Calculate

(1) First Correction Data FIGS. 3A and 3B are diagrams showing an example of luminance values possessed by each component of RGB. In FIG. 3A and FIG. 3B, the luminance values of the pixels located at the same coordinates between the first sample image and the second sample image are shown separately for the RGB components. That is, in the left part of FIGS. 3A and 3B, the R component and the G component of the pixel 22B or the pixel 23B which is one of the pixels included in the image 21B of the plant on the second sample image shown in FIG. And the B component luminance values are shown, and in the right part of FIGS. 3A and 3B, a pixel 22A located at the same coordinates as the pixel 22B or the pixel 23B on the first sample image shown in FIG. 2A. Or, the luminance values of the R component, the G component and the B component of the pixel 23A are shown. Further, FIG. 3A and FIG. 3B show the breakdown of the amounts of visible light and infrared light among the luminance values for each of the R component, the G component and the B component. That is, in FIG. 3A and FIG. 3B, while the luminance value measured by receiving visible light is shown by filling in a black background, the luminance value measured by receiving infrared light is filling in a white background. It is shown.

  As shown in FIGS. 3A and 3B, in any of the pixel 22B and the pixel 23B on the second sample image, the luminance value of the G component among the R component, the G component, and the B component is the highest, and is higher than the G component. The luminance value of the R component and the luminance value of the B component are small. Therefore, although the pixel 22B and the pixel 23B are both classified as green, this does not mean that the classification is the same and the color is the same. That is, the same green color also has variations, and the pixel 22B is visually recognized as a dark green color than the pixel 23B.

  On the other hand, in the pixels 22A and 23A on the first sample image, not only the light corresponding to the wavelength of visible light but also the light corresponding to the wavelength of infrared light is received. In this case, since each component of RGB receives the light corresponding to the wavelength of infrared light, the ratio of the luminance value among the R component, the G component and the B component is the pixel 22B and the pixel on the second sample image Relatively smaller than 23B. As a result, the color in which the pixel 22A and the pixel 23A are visually recognized becomes less achromatic as the saturation decreases by approaching an achromatic color.

  4A and 4B are diagrams showing an example of a histogram. In FIGS. 4A and 4B, a histogram relating to color differences possessed by pixels included in the first region on the first sample image is shown by solid white fill and included in the first region on the second sample image. The histogram relating to the color difference possessed by the selected pixel is shown by a solid black background. The vertical axes of the graphs shown in FIGS. 4A and 4B indicate frequencies, and the horizontal axes of the graphs indicate the color difference U or color difference V, which is a class value.

  The histograms shown in FIGS. 4A and 4B are, as described with reference to FIGS. 3A and 3B, in the first region when only visible light is received and when visible light and infrared light are received. There is a difference in the degree of spread of the distribution in which the colors of the contained pixels are scattered on the color space, and the latter has a smaller degree of spread than the former, in other words the former has a larger degree of spread than the latter. I support that.

  Focusing on the difference in the degree of spread, the correction data setting unit 15 takes, as an example, a distribution in which color components of pixels included in the first region of the first sample image are scattered, and the second From the distribution in which the color components of the pixels included in the first region of the sample image are scattered in the color space, in addition to the distance relating to the mutual distribution, the ratio of the spread degree of the mutual distribution is calculated as the first correction data Do.

  When color correction is performed to shift the color component of a pixel corresponding to a specific subject by using the ratio of the spread degree of the two distributions as the first correction data in addition to the distance between the two distributions as described above. In addition, it is possible to prevent the color components after shifting from being concentrated on a local region on the color space.

  More specifically, the correction data setting unit 15 causes each pixel included in the first area specified by the specifying unit 13 in the first sample image converted from RGB space to YUV space by the converting unit 14 The average value of the color difference is calculated for each of the U component and the V component. Similarly, among the second sample images converted from the RGB space to the YUV space by the conversion unit 14, the correction data setting unit 15 sets each of the pixels included in the first area specified by the specification unit 13 to one another. The average value of the color difference is calculated for each of the U component and the V component. Then, the correction data setting unit 15 sets the distance between the coordinates of the average value of the color difference calculated from the first sample image and the coordinate of the average value of the color difference calculated from the second sample image to the U component and Calculated for each V component.

  Thus, the distance of the color difference between the first region calculated from the first sample image and the second sample image is a shift amount for shifting the color component of the pixel corresponding to the specific subject on the color space It is used as Δu and Δv. Here, as an example, the case of obtaining the distance from the average value of the color difference has been exemplified, but it may not necessarily be the average value, and the distance may be obtained from the median or the mode.

  Furthermore, the correction data setting unit 15 further includes a distribution of color differences of pixels included in the first area specified by the specifying unit 13 in the first sample image converted from RGB space to YUV space by the converting unit 14. The degree of spread of is calculated for each of the U component and the V component. The standard deviation and the variance of the color difference can be calculated using the average value of the color difference used to calculate the shift amounts Δu and Δv as an example of the statistic for evaluating the degree of spread of the distribution. Similarly, in the second sample image converted from RGB space to YUV space by the conversion unit 14, the correction data setting unit 15 sets the color difference of the pixels included in the first area specified by the specification unit 13. The spread degree of the distribution is calculated for each of the U component and the V component. Then, the correction data setting unit 15 sets the ratio of the degree of spread between the degree of spread calculated from the first sample image and the degree of spread calculated from the second sample image for each of the U component and the V component. Calculate to

  Thus, the ratio σu_hi and σv_hi of the spread degree between the first regions calculated from the first sample image and the second sample image is a shift amount Δu of the color component of the pixel corresponding to the specific subject. It is used as a correction amount that defines the range of the color space to be shifted according to Δv.

  The first correction data is transmitted to the color correction device 100 by registering these shift amounts Δu and Δv and the ratio of spread degree σu_hi and σv_hi as the first correction data in the correction data storage unit 160 of the color correction device 100. It is set. Here, the case of calculating the ratio of the shift amount and the spread degree on the UV plane has been exemplified, but it is also possible to calculate the ratio of the shift amount and the spread degree on the ab plane on the L * a * b space. I do not mind.

(2) Second correction data Further, the above second correction data is calculated according to the method disclosed in the above-mentioned Non-Patent Document 1 "Study of a near-infrared imaging device capable of color imaging" as an example only. can do. That is, the correction data setting unit 15 detects the RGB pixel value of the pixel included in the second area among the pixels included in the two sample images of the first sample image and the second sample image. Matrix coefficients that approximate spectral sensitivity characteristics to color matching functions can be obtained by the method of least squares.

  The second correction data is set in the color correction apparatus 100 by registering the matrix coefficients calculated in this manner as the second correction data in the correction data storage unit 160 of the color correction apparatus 100.

  The separation condition setting unit 16 is a processing unit that sets, in the color correction apparatus 100, separation conditions for separating the above-described specific subject and the other subjects.

  In one embodiment, as an example of the above-described separation condition, the color of the pixels included in the first area among the pixels included in the first sample image is scattered in the color space as the separation condition setting unit 16. An area in which the shape and size of the distribution of coordinates are defined can be set in the color correction apparatus 100. For example, the separation condition setting unit 16 determines the distribution of the coordinates at which the UV component of each pixel included in the first region on the first sample image is scattered on the UV plane, the elliptical center point, the elliptical axis length, and the elliptical rotation By defining with parameters such as angle, it can be approximated as an elliptical area on the UV plane. At this time, the separation condition setting unit 16 narrows down to locations where the pixels forming the first region in the first sample image are concentrated and distributed on the UV plane of the color space to define an elliptical region, Pixels having a distance between the pixels within a predetermined value may be targets of inclusion, and pixels exceeding the predetermined value may be excluded from the targets of inclusion. As described above, an elliptical area is calculated which represents a distribution in which the UV components of the pixels forming the first subject on the first sample image are scattered on the UV plane. Then, the separation condition setting unit 16 sets the separation condition in the color correction device 100 by registering the above-described elliptical area as the separation condition in the separation condition storage unit 140 of the color correction device 100. Although the case of approximating the distribution to an elliptical area is exemplified here, it may be approximated to another shape, for example, a polygon.

  The processing units such as the first acquisition unit 11, the second acquisition unit 12, the identification unit 13, the conversion unit 14, the correction data setting unit 15, and the separation condition setting unit 16 can be mounted as follows. For example, in a central processing unit, so-called CPU (Central Processing Unit), etc., a process that exhibits the same function as each processing unit described above is developed in various semiconductor memory devices, such as RAM (Random Access Memory), flash memory, etc. It can be realized by executing it. These processing units may not necessarily be executed by the central processing unit, and may be executed by an MPU (Micro Processing Unit) or a DSP (Digital Signal Processor). Further, each functional unit described above can also be realized by hard wired logic such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[Configuration of color correction apparatus 100]
Next, the functional configuration of the color correction apparatus 100 according to the present embodiment will be described. As shown in FIG. 1, the color correction apparatus 100 includes an acquisition unit 110, a conversion unit 120, a determination unit 130, a separation condition storage unit 140, a correction unit 150, and a correction data storage unit 160.

  The acquisition unit 110 is a processing unit that acquires an image. Unlike the sample image acquired by the first acquisition unit 11 and the second acquisition unit 12 of the setting device 10, the “image” referred to here indicates an image to which the above-described color correction process is applied. Below, it may be described as an "original image" from one side called the image which is the origin from which color correction is performed. Here, as an example of the imaging mode of the original image, it is assumed that imaging is performed without an infrared cut filter, but the bandwidth of the visible light region and the infrared wavelength region to be used, for example, near infrared light An image captured in a state in which an optical filter that transmits a part of the light wavelength is inserted may be used as an original image.

  In one embodiment, the acquisition unit 110 can acquire an original image captured by a camera equipped with an imaging device such as a CCD or a CMOS. The acquisition unit 110 can also acquire the original image stored in an auxiliary storage device such as a hard disk or an optical disk or a removable medium such as a memory card or a USB (Universal Serial Bus) memory. In addition, the acquisition unit 110 can also acquire an original image by receiving from an external device via a network.

  The conversion unit 120 is a processing unit that converts the data format of the original image.

  In one embodiment, the conversion unit 120 converts the data format of the original image acquired by the acquisition unit 110 into a data format represented by a space including luminance and chromaticity. For example, the conversion unit 120 converts the pixel value of each pixel included in the original image from the pixel value of RGB expressed in RGB space to a pixel value expressed by luminance Y and two color differences. Here, although the case of converting the original image to the YUV space is illustrated, the color space of the conversion destination is not limited to the YUV space, and the original image is converted to another color space, for example, L * a * b space It does not matter.

  The determination unit 130 is a processing unit that refers to the separation condition stored in the separation condition storage unit 140 to determine whether a pixel included in the original image is a pixel corresponding to a specific subject.

  In one embodiment, the determination unit 130 selects one pixel among the pixels included in the original image converted to the color space by the conversion unit 120. Subsequently, the determination unit 130 refers to the separation condition stored in the separation condition storage unit 140, that is, the elliptical area on the UV plane corresponding to the above-described first area, and selects the pixel previously selected from the original image. It is determined whether or not the UV component of is included in the elliptical area corresponding to the first area on the UV plane. At this time, when the UV component of the pixel is not included in the elliptical area corresponding to the first area on the UV plane, it can be estimated that the pixel is a pixel that does not form a specific subject. In this case, the correction unit 150 selects color correction to be applied to objects other than the specific object, for example, color correction by matrix operation. On the other hand, when the UV component of the pixel is included in the elliptical area corresponding to the first area on the UV plane, it can be estimated that the pixel is a pixel forming a specific subject. In this case, the correction unit 150 selects the color correction to be applied to the first object, for example, the color correction to shift the color on the UV plane according to the shift amounts Δu and Δv and the ratio σu_hi and σv_hi of the spread degree. Here, as an example only, the pixel corresponding to the specific subject and the pixel not corresponding to each other are separated depending on whether the UV component of the pixel is included in the elliptical region corresponding to the first region on the UV plane. Although the case has been illustrated, it goes without saying that other separation conditions can be used.

  The correction unit 150 is a processing unit that executes color correction with reference to the correction data stored in the correction data storage unit 160.

  In one embodiment, the correction unit 150 applies color correction or a specific subject other than the specific subject depending on whether the UV component of the pixel is included in an elliptical area corresponding to the first area on the UV plane. Switch color correction to be applied to other objects.

  For example, when the UV component of the pixel is not included in the elliptical area corresponding to the first area on the UV plane, the correction unit 150 executes color correction to be applied to another object, for example, color correction by matrix operation. That is, the correction unit 150 refers to the second correction data stored in the correction data storage unit 160, that is, the matrix coefficient, and defines the matrix coefficient defined in the second correction data as the RGB pixel value of the pixel. Perform a matrix operation that multiplies In addition to the above-described color correction, the correction unit 150 can also execute γ correction, saturation correction, and the like together with the pixel value for which the color correction has been performed.

  On the other hand, when the UV component of the pixel is included in the elliptical area corresponding to the first area on the UV plane, the correction unit 150 performs color correction to be applied to a specific subject, for example, the ratio of shift amounts Δu and Δv to the spread degree Perform color correction to shift the color on the UV plane according to σu_hi and σv_hi.

  Specifically, the correction unit 150 substitutes the value of the U component of the previously selected pixel into the following equation (1), and changes the value of the V component of the pixel into the following equation (1) Substituting for 2) shifts the color component of the pixel.

u_o = u_in + Δu + (u_in-u_in average) * σu_hi formula (1)
v_o = v_in + Δv + (v_in-v_in average) * σv_hi equation (2)

  Here, “Δu” in the above equation (1) indicates the shift amount of the U component included in the first correction data, and “σu_hi” is the spread degree of the U component included in the first correction data. Point to ratio. Furthermore, "u_in" in the above equation (1) refers to the value of the U component of the pixels of the original image, "u_o" refers to the value of the U component after shifting, and "u_in average" refers to the original image Points to the average value of the U component of each pixel. Further, “Δv” in the above equation (2) refers to the shift amount of the V component contained in the first correction data, and “σv_hi” is a ratio of the spread degree of the V component contained in the first correction data Point to Furthermore, “v_in” in the above equation (2) indicates the value of the V component of the pixels of the original image, “v_o” indicates the value of the V component after shifting, and “v_in average” indicates the original image Points to the average value of the V component of each pixel.

  As described above, by performing color correction using the ratio of spread degrees .sigma.u_hi and .sigma.v_hi in addition to the shift amounts .DELTA.u and .DELTA.v, it is possible to suppress concentration of shifted color components in a local region on the color space. As a result, the color of a specific object is corrected to a color similar to the color recognized under visible light, and to a color having a spread like a real thing.

  FIG. 5 is a view showing an example of the color distribution on the UV plane. The vertical axis of the graph shown in FIG. 5 indicates the color difference V, and the horizontal axis indicates the color difference U. In FIG. 5, while the distribution of color components possessed by the pixels included in the first region on the first sample image is plotted as a circle mark on the UV plane, the first distribution on the second sample image is plotted. The distribution of color components possessed by the pixels contained in the region of is plotted with diamond marks on the UV plane. Furthermore, in FIG. 5, among the pixels included in the first sample image, an elliptical region 51 in which the distribution of coordinates in which the color of each pixel included in the first region is scattered in the color space is approximated to an ellipse. Among the pixels included in the second sample image, an elliptical area 52 is shown, in which the distribution of coordinates in which the color of each pixel included in the first area is scattered in the color space is approximated to an ellipse.

  As shown in FIG. 5, when color correction is performed using the ratio of spread degrees σu_hi and σv_hi in addition to the shift amounts Δu and Δv, colors of pixels included in the elliptical area 51 of the UV plane among the pixels included in the original image The component is shifted to an elliptical area 52 wider than the elliptical area 51. Therefore, the color of a specific subject is not only corrected to a color similar to the color recognized under visible light, but colors with spread among each other even in the same color classification as real Can be corrected.

  Thus, the values of the color components of the pixels shifted on the UV plane according to the shift amounts Δu and Δv and the ratio of the spread degrees σu_hi and σv_hi, together with the values of the luminance Y of the pixels, are the pixel values of the RGB space, ie RGB Is converted back to the luminance value of each component of.

  After color correction of each pixel of the original image is performed in this manner, the image after color correction can be output to any output destination. For example, the image after color correction may be displayed on a predetermined display, or may be stored in an auxiliary storage device such as a hard disk or an optical disk, or in a removable medium such as a memory card or USB memory. In addition, the image after color correction may be output to an application program that causes the computer to execute various processes using an image as input, or the image after color correction may be transmitted to any external device via a network. it can.

  The functional units such as the acquisition unit 110, the conversion unit 120, the determination unit 130, and the correction unit 150 described above can be mounted as follows. For example, it can be realized by causing a central processing unit, so-called CPU or the like, to expand and execute a process exhibiting the same function as each processing unit described above on a memory. These processing units may not necessarily be executed by the central processing unit, and may be executed by the MPU or DSP. Further, each functional unit described above can also be realized by hard wired logic such as ASIC or FPGA.

  In addition, various semiconductor memory devices such as a RAM and a flash memory can be adopted as an example of the separation condition storage unit 140 and the correction data storage unit 160 described above. Further, the separation condition storage unit 140 and the correction data storage unit 160 may not necessarily be the main storage device, and may be an auxiliary storage device. In this case, an HDD, an optical disk, an SSD, or the like can be employed.

[Flow of processing]
Subsequently, the flow of processing of the color correction system according to the present embodiment will be described. Here, the color correction apparatus 100 executes the first correction data calculation process after (1) setting process executed by the setting device 10 and (2) first correction data calculation process executed as one procedure of the setting process. (3) The color correction process will be described.

(1) Setting Process FIG. 6 is a flowchart showing the procedure of the setting process according to the first embodiment. This process is activated, for example, when a setting instruction to the color correction apparatus 100 is received. As shown in FIG. 6, when the setting process is activated, the first acquisition unit 11 acquires a first sample image and the second acquisition unit 12 acquires a second sample image (step S101). .

  Subsequently, the specifying unit 13 specifies a first area forming a part or all of a specific subject from the first sample image acquired in step S101, and the specific area other than the specific subject from the first sample image A second area forming part or all of the other objects of the subject is identified (step S102).

  Then, the conversion unit 14 expresses the data format of the first sample image acquired by the first acquisition unit 11 and the second sample image acquired by the second acquisition unit 12 in a space including luminance and chromaticity. Converted to a data format (step S103).

  Thereafter, the correction data setting unit 15 causes the color components of the pixels included in the first region of the first sample image to be included in the distribution in which the color components are scattered in the color space and the first region of the second sample image. From the distribution in which the color components of the pixels are scattered in the color space, in addition to the distance relating to each other, the ratio of the spread degree of each other is calculated as first correction data (step S104).

  In addition, the correction data setting unit 15 detects the RGB pixel value of the pixel included in the second area among the pixels included in the two sample images of the first sample image and the second sample image. Matrix coefficients that approximate the spectral sensitivity characteristics to the color matching function are calculated as second correction data (step S105).

  Then, the correction data setting unit 15 registers the plurality of first correction data calculated in step S104 and the second correction data calculated in step S105 in the correction data storage unit 160 of the color correction apparatus 100. The respective correction data used for the color correction of the kind are set in the color correction apparatus 100 (step S106).

  Thereafter, the separation condition setting unit 16 obtains the color obtained by approximating the distribution of the coordinates in which the color component of each pixel included in the first region on the first sample image is scattered on the color space into an ellipse. An elliptical area in space is calculated as the separation condition (step S107).

  Then, the separation condition setting unit 16 sets the separation condition in the color correction device 100 by registering the separation condition calculated in step S107 in the separation condition storage unit 140 of the color correction device 100 (step S108). , End the process.

  Although the flowchart of FIG. 6 illustrates the case where the process of step S105 is performed after the process of step S104 is performed, these processes may be performed in the reverse order, or in parallel. Can also be implemented. Further, in the flowchart shown in FIG. 6, the case where the processing of step S107 to step S108 is performed after the processing of step S104 to step S106 is exemplified, but these processing may also be performed in the reverse order. It can also be run in parallel.

(2) Calculation Processing of First Correction Data FIG. 7 is a flowchart showing a procedure of calculation processing of first correction data according to the first embodiment. This process is a process corresponding to the process of step S104 shown in FIG. 6, and is activated after execution of the process of step S103.

  As shown in FIG. 7, the correction data setting unit 15 sets each pixel included in the first area specified in step S102 out of the first sample image converted from RGB space to YUV space in step S103. The average value of the color difference is calculated for each of the U component and the V component between the two (step S201).

  Similarly, in the second sample image converted from RGB space to YUV space in step S103, the correction data setting unit 15 performs color difference between pixels included in the first area specified in step S102. An average value is calculated for each of the U component and the V component (step S202).

  Then, the correction data setting unit 15 calculates the coordinates of the average value of color differences calculated from the first sample image in step S201, and the coordinates of the average value of color differences calculated from the second sample image in step S202. The shift amount is calculated for each of the U component and the V component by calculating the distance of (step S203).

  Further, the correction data setting unit 15 spreads the distribution of the color difference possessed by the pixels included in the first area specified in step S102 among the first sample images converted from RGB space to YUV space in step S103. The degree is calculated for each of the U component and the V component (step S204).

  Similarly, in the second sample image converted from RGB space to YUV space in step S103, the correction data setting unit 15 performs distribution of color differences of pixels included in the first region specified in step S102. The spread degree is calculated for each of the U component and the V component (step S205).

  Then, the correction data setting unit 15 sets the ratio of the degree of spread between the degree of spread calculated from the first sample image in step S204 and the degree of spread calculated from the second sample image in step S205. Calculation is performed for each of the U component and the V component (step S206), and the process is ended.

  By these series of procedures, the shift amounts Δu and Δv and the ratios σu_hi and σv_hi of the spread degree are calculated as first correction data.

  Although the flowchart of FIG. 7 illustrates the case where the process of step S202 is performed after the process of step S201 is performed, these processes may be performed in the reverse order, or in parallel. Can also be implemented. Further, in the flowchart shown in FIG. 7, the case where the process of step S205 is performed after the process of step S204 is performed is illustrated, but these processes can also be performed in the reverse order. Can also be implemented. Furthermore, although the case where the process of step S204-step S206 is performed after the process of step S201-step S203 was performed was illustrated in the flowchart shown in FIG. 7, these processes are also performed in reverse order. It can also be run in parallel.

(3) Color Correction Process FIG. 8 is a flowchart showing the procedure of the color correction process according to the first embodiment. The color correction process is activated, for example, when an original image to be subjected to color correction is designated. As shown in FIG. 8, when an original image to be subjected to color correction is designated, the acquisition unit 110 acquires the original image (step S301). Subsequently, the conversion unit 120 converts the data format of the original image acquired in step S301 into a data format represented by a space including luminance and chromaticity (step S302).

  Then, the determination unit 130 selects one of the pixels included in the original image converted to the color space in step S302 (step S303). Subsequently, the determination unit 130 refers to the separation condition stored in the separation condition storage unit 140, that is, the elliptical area on the UV plane corresponding to the above-described first area, and is selected from the original image in step S303. It is determined whether the UV component of the pixel is included in the elliptical area corresponding to the first area on the UV plane (step S304).

  Here, when the UV component of the pixel is included in the elliptical region corresponding to the first region on the UV plane (Yes in step S304), the pixel has a color close to achromatic color, and the amount of received infrared light is large. Since it can be estimated that it is a pixel, it is more likely that it will be the pixel that forms the above-mentioned specific subject. In this case, the correction unit 150 sets the UV component of the pixel according to the shift amounts .DELTA.u and .DELTA.v defined in the first correction data stored in the correction data storage unit 160 and the ratio .sigma.u_hi and .sigma.v_hi of the spread degree to the UV plane. After shifting up, color correction is performed to convert back to pixel values in RGB space (step S305).

  On the other hand, when the UV component of the pixel is not included in the elliptical area corresponding to the first area on the UV plane (No in step S304), the correction unit 150 performs the following process. That is, the correction unit 150 refers to the second correction data stored in the correction data storage unit 160, that is, the matrix coefficient, and defines the matrix coefficient defined in the second correction data as the RGB pixel value of the pixel. The color correction for the other subject is executed to execute the matrix operation of multiplying by (step S306).

  Thereafter, the process from step S303 to step S306 is repeatedly executed until all the pixels included in the original image are selected (No in step S307). Then, when all the pixels included in the original image are selected (Yes at step S307), the correction unit 150 outputs the image after correction to a predetermined output destination (step S308), and the process ends.

  In the flowchart of FIG. 8, the color correction is performed in the order in which the pixels included in the original image are selected in step S303. However, it goes without saying that the color correction can be performed in parallel for each pixel.

[One side of effect]
As described above, the color correction system 1 according to the present embodiment is compatible with a specific subject having a high spectral reflectance of infrared light in an image captured including light of wavelengths of visible light and infrared light. The color components of the pixels are shifted to a wider range than the range in which the color of the subject is distributed on the color space. As described above, when color correction is performed on a specific subject such as a plant leaf or chemical fiber cloth, the color of the specific subject is prevented from being concentrated on a local area in the color space. As a result, the color of a specific subject is corrected to a color similar to the color recognized under visible light, and even if it is a classification of the same color as in the real thing, it is corrected to a color having spread among them. Be done. Therefore, it is possible to improve color reproducibility.

  Although the embodiments of the disclosed apparatus have been described above, the present invention may be implemented in various different forms other than the above-described embodiments. Therefore, another embodiment included in the present invention will be described below.

[Input image]
In Example 1 described above, as an example of the imaging mode in which light corresponding to the wavelengths of visible light and infrared light is imaged on the imaging device, an image captured without the infrared cut filter attached is input However, the form of taking an image is not limited to this. For example, an image captured in a state in which an optical filter transmitting a visible light region and a bandwidth of an infrared wavelength region to be used is inserted is used as a first sample image, and the setting process shown in FIG. 6 is applied. Alternatively, the image can be used as an original image and the color correction process shown in FIG. 7 can be applied.

Blend of color correction
In the first embodiment described above, the case is exemplified in which two types of color correction are exclusively performed depending on whether the UV component of the pixel is included in the elliptical area corresponding to the first area on the UV plane. Two color corrections can also be mixed. For example, when the color component possessed by the pixel is included in the elliptical area, the color correction apparatus 100 can blend color shift correction and color correction by matrix operation using the following equation (3).

  RGB = r'g'b '× w1 + rgb × (1-w1) (3)

  “R ′”, “g ′” and “b ′” shown in the above equation (3) are RGB values after color shift correction, and “r” and “g” shown in the above equation (3) And “b” are RGB values after color correction by matrix operation. Further, “w1” shown in the above equation (3) indicates the probability that the pixel is a specific subject, and, for example, an arbitrary value of 0 ≦ w1 ≦ 1 can be set. Thus, for example, the color shift correction rate is increased as the probability that the pixel is a specific subject increases, and the color correction rate by matrix operation is decreased as the probability that the pixel is a specific subject decreases. You can blend it high for color correction. Here, although the case where the blending process is performed in the RGB space is illustrated, the application range of the blending process is not limited to the image in the RGB space, and the blending process is performed in another color space such as YUV or L * a * b. You may convert to RGB after executing.

[Extension to 3 dimensions]
Although the case of shifting the two-dimensional UV plane of the color difference U and the color difference V is exemplified in the above-described first embodiment, the application range is not limited to the shift on the two-dimensional plane. For example, in the color correction system 1, the colors of the pixels included in the original image can be shifted on the three-dimensional color space of the luminance Y, the color difference U, and the color difference V.

  Specifically, the setting device 10 further calculates the average value of the Y component together with the average value of the U component and the average value of the V component in the process of step S201 and the process of step S202 shown in FIG. 7. Then, the setting device 10 calculates the coordinates of the average of the luminance Y calculated from the first sample image in the process of step S203 and the coordinates of the average of the luminance Y calculated from the second sample image. By calculating the distance, the shift amount Δy related to the luminance Y is further calculated. Further, the setting device 10 further calculates the spread degree of the Y component together with the spread degree of the U component and the spread degree of the V component in the processing of step S204 and the processing of step S205. Then, the setting apparatus 10 sets the brightness Y between the spread degree of the brightness Y calculated from the first sample image in the process of step S206 and the spread degree of the brightness Y calculated from the second sample image. The ratio σy_hi of the degree of spread of is calculated. By these series of procedures, the shift amounts Δu, Δv and Δy and the ratio of the spread degree σu_hi, σv_hi and σy_hi are set as the first correction data to the color correction apparatus 100.

  Under such settings, the color correction apparatus 100 substitutes the value of the U component of the pixel selected in step S303 shown in FIG. 8 into the above equation (1) and the value of the V component of the same pixel. Is substituted into the above equation (2), and the value of the Y component of the pixel is substituted into the following equation (4) to shift the color component of the pixel in a three-dimensional space.

  y_o = y_in + Δy + (y_in-y_in average) * σy_hi equation (4)

  FIG. 9 is a diagram showing an example of color distribution in YUV space. Three axes of luminance Y, color difference V and color difference U are shown in the graph shown in FIG. In FIG. 9, while the distribution of color components possessed by the pixels included in the first region on the first sample image is plotted as a circle mark on the YUV space, the first distribution on the second sample image is plotted. The distribution of color components possessed by the pixels included in the region of is plotted with diamond marks on the YUV space. Furthermore, in FIG. 9, an elliptical sphere 91 in which the distribution of the coordinates in which the color of each pixel included in the first region is scattered among the pixels included in the first sample image is approximated to an elliptical sphere. And an elliptical sphere 92 in which the distribution of the coordinates in which the color of each pixel included in the first region is scattered among the pixels included in the second sample image is approximated to an elliptical sphere, There is.

  As shown in FIG. 5, when color correction is further performed using the shift amount Δy and the ratio σy_hi of the expansion degree, among the pixels contained in the original image, the color components of the pixels contained in the elliptical area 51 of the UV plane are the oval spheres. The elliptical sphere 92 is shifted to a wider range than 91. At this time, the color shift correction can be performed in consideration of the luminance which is increased by receiving the light corresponding to the wavelength of the infrared light in addition to the light corresponding to the wavelength of the visible light. Therefore, it is possible to further improve the color reproducibility.

[Spreading direction]
In the first embodiment described above, the case of using the ratio of the degree of spread when performing color correction has been exemplified, but the difference in the direction of spread is also set as the first correction data together with the ratio of the degree of spread. it can.

  That is, among the pixels included in the first sample image, an elliptical region in which the color distribution of each pixel included in the first region is scattered in the color space approximates an ellipse, and the second sample image Of the pixels contained in the first region, and the distribution of the coordinates scattered in the color space is defined by the minor axis and the major axis of each other with the elliptical region in which the distribution of the coordinates is approximated to an ellipse There is no guarantee that the directions will be the same.

  From this, the setting device 10 determines between the elliptical area defined by the first sample image and the elliptical area defined by the second sample image in the directions defined by the minor axis and the major axis of each other. Find the relative angle. Then, the setting device 10 can add the above relative angle to the first correction data as the spread direction difference Δθ. Under the setting of the first correction data, the color correction apparatus 100 moves the color components of the pixel according to the shift amounts Δu and Δv and moves the color components of the pixel according to the ratio of spread degrees σu_hi and σv_hi. The color component of the pixel is rotated on the UV plane according to the spread Δθ. When the difference in the spreading direction is taken into consideration in this manner, the color components after the shift approach the real thing, and it is possible to further improve the color reproducibility.

[Movement transformation matrix]
In this embodiment, the case of performing color shift correction by setting three correction amounts of the shift amounts Δu and Δv, the ratio of spread degrees σu_hi and σv_hi, and the spread direction difference Δθ has been exemplified. It is also possible to obtain a movement transformation matrix in which one correction amount is added, and to set the movement transformation matrix as the first correction data.

  For example, assuming that the number of pixels included in the first region on the first sample image is n, the U component of n pixels included in the first region on the first sample image and The V component can be represented as two-dimensional vector data A, as shown in the following equation (5). In this case, since the number of pixels included in the first region on the second sample image is also n, the U component and V of n pixels included in the first region on the second sample image are also included. The component can be represented as two-dimensional vector data B, as shown in the following equation (6).

Among them, the average value for each of the U component and the V component for n pixels included in the two-dimensional vector data A can be expressed by the following equation (7), and using the equation (7), 2 The covariance matrix _{a a} of the dimensional vector data A can be expressed by the following equation (9). On the other hand, the average value for each of the U component and the V component for n pixels included in the two-dimensional vector data B can be expressed by the following equation (8), and using the equation (8), the two-dimensional The covariance matrix _{b b} of the vector data B of can be expressed by the following equation (10).

In this case, the eigenvalue D _a and the eigenvector V _a of the two-dimensional vector data A are expressed by the following equation (11) using a function eig used to calculate the eigenvalue and the eigenvector with numerical analysis software etc. It can be calculated. Similarly, the eigenvalues D _b and the eigenvectors V _b of the two-dimensional vector data B can be calculated using the function eig according to the following equation (12). In addition, although the case where function eig is used for calculation of an eigen value and an eigenvector was illustrated here, it is good also as using another function.

After the eigenvalues D _a and the eigenvectors V _a of the two-dimensional vector data A are calculated as described above, the movement transformation matrix A _out of the two-dimensional vector data A is calculated according to the following equation (13). Further, the movement transformation matrix B _out of the two-dimensional vector data B is calculated as the following equation (14). After that, the correction matrix out is obtained from the ratio of the movement transformation matrix A _out and the movement transformation matrix B _out as in the following equation (15).

out = B _out / A _out formula (15)

  When such a correction matrix out is set as the first correction data, the color correction apparatus 100 inputs two-dimensional vector data in regarding pixels included in an elliptical area corresponding to the first area in the original image. By performing a vector operation with the above correction matrix out, it is possible to obtain the color reproducibility equivalent to the color correction using the spread direction described in the above “spreading direction”.

  10A and 10B show an example of the distribution of color on the UV plane. The vertical axes of the graphs shown in FIGS. 10A and 10B indicate the color difference V, and the horizontal axes indicate the color difference U. In FIG. 10A, the distribution of two-dimensional vector data A is plotted by a circle mark, while the distribution of two-dimensional vector data B is plotted by a triangular mark “Δ” on the UV plane. Furthermore, in FIG. 10B, while the distribution of the two-dimensional vector data A is plotted by a circle mark, the calculation result of the two-dimensional vector data A and the correction matrix out, that is, the two-dimensional vector after correction. The distribution of the data is plotted on the UV plane with the inverse triangle mark "▽".

  As shown in FIGS. 10A and 10B, it can be said that the distribution of the target two-dimensional vector data B and the distribution of the two-dimensional vector data after correction approximate each other. Therefore, even when color correction of two-dimensional vector data in is performed using the correction matrix out, the same color reproducibility as the color correction using the spread direction described in the above "spread direction" can be obtained. It can be estimated that Although the case of using two-dimensional vector data has been exemplified here, the luminance Y may be further extended to be extended as three-dimensional vector data, and setting of the correction matrix out and color correction using the correction matrix out may be performed. it can.

[Environmental condition]
Although the shift amounts Δu and Δv and the ratios σu_hi and σv_hi of the spread degrees are exemplified as the first correction data in the above-described first embodiment, the shift amounts Δu and Δu, which are related to the environment in which the image is captured Correction amounts such as the shift amount Δv, the ratio of spread degree σu_hi, and the ratio of spread degree σv_hi can also be set.

  FIG. 11 is a diagram showing an example of the first correction data. FIG. 11 illustrates, as an example of the environmental conditions, first correction data in which a correction amount is associated with each illuminance measured in the imaging range of the camera. In the example of FIG. 11, when the illuminance measured in the imaging range of the camera that captures the original image is included in the illuminance range corresponding to the illuminance 1, the shift amount 1 and the ratio 1 of the expansion degree are used for color correction Be Further, when the illuminance measured in the imaging range of the camera that captures the original image is included in the range of the illuminance corresponding to the illuminance 2, the shift amount 2 and the ratio 2 of the expansion degree are used for the color correction. Further, when the illuminance measured in the imaging range of the camera that captures the original image is included in the range of the illuminance corresponding to the illuminance 3, the shift amount 3 and the ratio 3 of the expansion degree are used for the color correction. This makes it possible to change the correction amount used for color correction in accordance with the degree of illuminance measured in the imaging range of the camera. Although illuminance is illustrated as an example of environmental conditions in FIG. 11, the environmental conditions are not limited to this, and the time zone in which the original image is captured, the type of weather that can be acquired by sensing or browsing, etc. It can also be a condition.

  When the correction amount is set for each environmental condition as described above, a part of the procedure of the process shown in FIG. 8 may be changed, and the color correction process may be performed according to the procedure shown in FIG. FIG. 12 is a flowchart of the color correction process according to the second embodiment. In FIG. 12, the same step numbers are assigned to the same procedures as the process shown in FIG. As shown in FIG. 12, after the process of step S301 shown in FIG. 8 is performed, the color correction apparatus 100 detects an environmental condition (step S401). For example, when the environmental condition is illuminance, the illuminance may be acquired from an illuminance sensor provided around the camera. Subsequently, the color correction apparatus 100 acquires a correction amount corresponding to the environmental condition detected in step S401 among the correction amounts stored in the correction data storage unit 160 as the first correction data (step S402). After the correction amount used for the color correction in step S305 is acquired by the process in step S402, the processes in steps S302 to S308 illustrated in FIG. 8 may be performed.

Distributed and integrated
Further, each component of each illustrated device may not necessarily be physically configured as illustrated. That is, the specific form of the distribution and integration of each device is not limited to the illustrated one, and all or a part thereof may be functionally or physically dispersed in any unit depending on various loads, usage conditions, etc. It can be integrated and configured. For example, the acquisition unit 110, the conversion unit 120, the determination unit 130, or the correction unit 150 may be connected as an external device of the color correction apparatus 100 via a network. In addition, another device has an acquisition unit 110, a conversion unit 120, a determination unit 130, or a correction unit 150, and the functions of the color correction apparatus 100 are realized by network connection and cooperation. It is also good.

[Color correction program]
The various processes described in the above embodiments can be realized by executing a prepared program on a computer such as a personal computer or a workstation. Therefore, an example of a computer that executes a color correction program having the same function as that of the above embodiment will be described below with reference to FIG.

  FIG. 13 is a diagram illustrating an example of a hardware configuration of a computer that executes a color correction program according to the first embodiment and the second embodiment. As shown in FIG. 13, the computer 1000 includes an operation unit 1100 a, a speaker 1100 b, a camera 1100 c, a display 1200, and a communication unit 1300. The computer 1000 further includes a CPU 1500, a ROM 1600, an HDD 1700, and a RAM 1800. The components 1100 to 1800 are connected via a bus 1400.

  As shown in FIG. 13, the HDD 1700 stores a color correction program 1700 a that exhibits the same functions as the acquisition unit 110, the conversion unit 120, the determination unit 130, and the correction unit 150 described in the first embodiment. The color correction program 1700a may be integrated or separated, as with each component of the acquisition unit 110, the conversion unit 120, the determination unit 130, and the correction unit 150 illustrated in FIG. That is, the HDD 1700 may not necessarily store all the data described in the first embodiment, and data used for processing may be stored in the HDD 1700.

  Under such an environment, the CPU 1500 reads out the color correction program 1700 a from the HDD 1700 and develops it in the RAM 1800. As a result, the color correction program 1700a functions as a color correction process 1800a as shown in FIG. The color correction process 1800a expands various data read from the HDD 1700 in the area allocated to the color correction process 1800a in the storage area of the RAM 1800, and executes various processes using the expanded various data. For example, the process illustrated in FIG. 8 or 12 or the process described in the second embodiment may be included as an example of the process performed by the color correction process 1800a. In the CPU 1500, all the processing units described in the first embodiment may not necessarily operate, and processing units corresponding to processing to be executed may be virtually realized.

  The color correction program 1700 a may not necessarily be stored in the HDD 1700 or the ROM 1600 from the beginning. For example, the color correction program 1700 a is stored in a “portable physical medium” such as a flexible disk inserted in the computer 1000, a so-called FD, a CD-ROM, a DVD disk, a magnetooptical disk, or an IC card. Then, the computer 1000 may acquire the color correction program 1700a from these portable physical media and execute it. In addition, the color correction program 1700a is stored in another computer or server device connected to the computer 1000 via a public line, the Internet, LAN, WAN, etc., and the computer 1000 acquires the color correction program 1700a from these. It may be carried out.

DESCRIPTION OF SYMBOLS 1 color correction system 10 setting device 11 first acquisition unit 12 second acquisition unit 13 identification unit 14 conversion unit 15 correction data setting unit 16 separation condition setting unit 100 color correction device 110 acquisition unit 120 conversion unit 130 determination unit 140 separation condition storage Unit 150 Correction unit 160 Correction data storage unit

Claims (5)

On the computer
The process of acquiring an image,
Converting the image into color space components including luminance and chromaticity;
A process of determining whether a pixel included in the image converted to the component of the color space is a pixel corresponding to an object whose spectral reflectance of infrared light is higher than the spectral reflectance of visible light;
A first sample image captured in a state in which the subject receives light corresponding to wavelengths of visible light and infrared light, and a image captured in a state in which the subject receives light corresponding to a wavelength of visible light Pixels included in the image converted to the component of the color space according to correction data in which the ratio of the distance and the degree of spread regarding the chromaticity distribution of the pixel corresponding to the subject to the two sample images is associated Performing color correction to shift, on the color space, a color component of a pixel determined to be a pixel corresponding to a subject whose spectral reflectance of infrared light is higher than the spectral reflectance of visible light among them. Color correction program characterized by. The correction data further corresponds to a ratio of a distance and a spread degree regarding the distribution of the luminance of the pixel corresponding to the subject between the first sample image and the second sample image.
2. The color correction process according to claim 1, wherein the color correction is performed according to the correction data to shift a luminance component and a chromaticity component of a pixel corresponding to the subject on a color space. Color correction program. The correction data further corresponds to a difference in the spread direction regarding the distribution of the chromaticity of the pixel corresponding to the subject between the first sample image and the second sample image,
The process of performing the color correction is further characterized by performing color correction in which color components of pixels corresponding to the subject are rotated on a color space according to the spread direction difference included in the correction data. The color correction program according to Item 1 or 2. The computer is
The process of acquiring an image,
Converting the image into color space components including luminance and chromaticity;
A process of determining whether a pixel included in the image converted to the component of the color space is a pixel corresponding to an object whose spectral reflectance of infrared light is higher than the spectral reflectance of visible light;
A first sample image captured in a state in which the subject receives light corresponding to wavelengths of visible light and infrared light, and a image captured in a state in which the subject receives light corresponding to a wavelength of visible light Pixels included in the image converted to the component of the color space according to correction data in which the ratio of the distance and the degree of spread regarding the chromaticity distribution of the pixel corresponding to the subject to the two sample images is associated Performing color correction to shift, on the color space, a color component of a pixel determined to be a pixel corresponding to an object whose spectral reflectance of infrared light is higher than the spectral reflectance of visible light, A color correction method characterized by A subject with a spectral reflectance of infrared light higher than that of visible light has a first sample image captured in a state in which light corresponding to wavelengths of visible light and infrared light is received, and the subject is visible Correction data in which the ratio of the distance and the degree of spread related to the distribution of the chromaticity of the pixel corresponding to the subject with the second sample image imaged in the state of receiving the light corresponding to the wavelength of light A storage unit to store;
An acquisition unit for acquiring an image;
A converter for converting an image into color space components including luminance and chromaticity;
A determination unit that determines whether a pixel included in the image converted to the component of the color space is a pixel corresponding to an object whose spectral reflectance of infrared light is higher than the spectral reflectance of visible light;
Among the pixels included in the image converted to the component of the color space according to the correction data , the pixels of the pixels determined to correspond to the subject whose spectral reflectance of infrared light is higher than the spectral reflectance of visible light And a correction unit that performs color correction to shift color components on a color space.