JP5187225B2 - Bias component estimation method, bias component estimation apparatus, and bias component estimation program for captured image - Google Patents

Description

  The present invention relates to a bias component estimation method for accurately grasping bias components such as dark current and flare included in a sensor response value recorded as image data in an image photographed using an imaging device such as a digital camera, and the like. The present invention relates to a bias component estimation device and a bias component estimation program.

  In order to correctly acquire color information of an object with an imaging device such as a digital camera, it is necessary to correctly grasp the relationship between the amount of incident light and a sensor response value from an image obtained by photographing a color chart or a gray step chart.

  In general, when photographing with a digital camera, a bias component is included in the sensor response value due to the influence of dark current and flare. The magnitude of this bias component depends on the performance of the digital camera, the temperature at the time of shooting, and the like, and depending on the shooting conditions, it may be included in several percent with respect to the sensor response value for the white subject.

  Therefore, in order to correctly grasp the relationship between the amount of incident light and the sensor response value, it is necessary to correctly grasp the bias component. The bias component corresponds to a sensor response value obtained when a subject having a spectral reflectance of 0 is photographed. However, in actual shooting, even the darkest subject does not have a spectral reflectance of zero. For this reason, the bias component cannot be directly acquired from the captured image.

  Here, as disclosed in Patent Document 1, there is a method of grasping the dark current, which is one of the bias components, by capturing a light-shielded image under the same conditions as when exposure was performed for each capturing. In addition, as shown in Patent Document 2, flare which is another bias component is calculated from the number of pixels in a specified area of a captured image and the number of pixels in which the sensor response value becomes a certain value or more. There is a way. In this way, there is a method for grasping the bias component for each factor. However, in order to grasp the bias component for each factor, each process is necessary, and thus a lot of work is required.

  Therefore, in order to easily obtain the bias component, it is assumed that the spectral reflectances of the gray step chart are similar to each other, and from the sensor response value and the spectral reflectance obtained when the gray step chart is photographed, A method for estimating a bias component by regression analysis is widely used. In this method, it is assumed that the spectral reflectances of the gray step chart are similar to each other. However, this condition is not always satisfied in the achromatic (gray) patch of the color chart generally used and the gray step chart. Absent.

  FIG. 1 shows the spectral reflectance of an achromatic (gray) patch of a commercially available color chart (Macbeth ColorChecker manufactured by X-Rite), that is, a gray step chart. FIG. 2 shows a graph obtained by normalizing the graph of FIG. 1 at a wavelength of 550 nm (reference wavelength). From the figure, it can be seen that even if the spectral reflectance is multiplied by a factor, the graphs do not overlap (not similar). As described above, when the spectral reflectance is not similar for each achromatic color patch, the method using multiple regression analysis cannot estimate the bias component with sufficient accuracy.

JP-A-10-98651 Japanese Patent Laid-Open No. 11-355637

  In the present invention, an image obtained by photographing a general gray step chart in which spectral reflectances are not similar for each achromatic (gray) patch is used, and bias components such as dark current and flare included in the sensor response value of the photographed image are obtained. The purpose is to grasp easily and accurately.

  In order to solve the above-mentioned problem, a first invention is a bias component estimation method for estimating a bias component corresponding to a sensor response value when a subject having a spectral reflectance of 0 is photographed by an imaging device. A step of calculating a basis vector group of spectral reflectance of the gray step chart and a coefficient for the basis vector by principal component analysis of the spectral reflectance group of the gray step chart included in the color chart used in the calibration of the gray step chart; Capturing each gray patch with the imaging device to obtain captured image data, obtaining a sensor response value for each gray patch from the captured image data, weights of the sensor response value group and the base vector It has a step to calculate the bias component from the coefficient group by multiple regression analysis A bias component estimation method of.

  In order to solve the above-described problem, the second invention is a bias component estimation device that estimates a bias component corresponding to a sensor response value when a subject having a spectral reflectance of 0 is captured by an imaging device, Means for calculating a basis vector group of spectral reflectance of the gray step chart and a coefficient for the basis vector by principal component analysis of the spectral reflectance group of the gray step chart included in the color chart used for calibration of the imaging apparatus; Means for capturing each gray patch of the step chart with the imaging device to obtain photographed image data, means for obtaining a sensor response value for each gray patch from the photographed image data, the sensor response value group, and the basis vector Having a means for calculating a bias component from multiple weighting coefficient groups by multiple regression analysis. Scan is a component estimation device.

  In order to solve the above problem, a third invention is a bias component estimation program for estimating a bias component corresponding to a sensor response value when a subject having a spectral reflectance of 0 is photographed by an imaging device, Processing for calculating a basis vector group of spectral reflectance of the gray step chart and a coefficient for the base vector by principal component analysis of the spectral reflectance group of the gray step chart included in the color chart used for calibration of the imaging apparatus; A process of capturing each gray patch of the step chart with the imaging device to obtain captured image data, a process of acquiring a sensor response value for each gray patch from the captured image data, the sensor response value group, and the basis vector It is characterized by having processing to calculate bias components from multiple weighting coefficient groups by multiple regression analysis. A bias component estimation program.

  As described above, according to the present invention, an image obtained by photographing a general gray step chart whose spectral reflectances are not similar to each other is used, and dark current, flare, and the like included in the sensor response value of the photographed image are used. In addition, it is possible to easily and accurately grasp a bias component corresponding to a sensor response value when a subject having a spectral reflectance of 0 is photographed by the imaging device.

It is a graph of the spectral reflectance of the achromatic color chart of a commercially available chart. It is a graph of the spectral reflectance of the achromatic color chart of the commercially available chart normalized at 550 nm. It is a flowchart showing the flow of the bias component estimation method by one Embodiment of this invention.

式１ <Bias component estimation principle>
First, a bias component estimation method according to the present invention will be described. Under ideal conditions without a bias component such as dark current or flare, which corresponds to a sensor response value when a subject having a spectral reflectance of 0 is photographed by an imaging device, red, green, Consider the case of imaging with a three-band imaging device that outputs blue RGB image data. In this case, the sensor response values S _R , S _G , and S _{B of} each channel and the spectral reflectance r of the subject (r is a column vector made up of discrete values of spectral reflectance sampled at equal intervals in the visible region). In the meantime, the linear transformation shown in Formula 1 is established. Where F is a matrix of the number of samplings consisting of the spectral sensitivity characteristics of the imaging device × the number of channels of the imaging device, E is a diagonal matrix having the spectral energy of the photographic illumination light as a diagonal component, and T is transposition.

Formula 1

式２ In an actual photographing apparatus, the bias values β _R , β _G , β _B due to the influence of dark current, flare, etc. are included in the sensor response values S _R , S _G , S _B of each channel formulated in Expression 1. If the sensor response values of the channels actually obtained are d _R , d _G , and d _B , d _R , d _G , and d _B are normalized to S _R , S _G , and S _B , and β _R , β _G , β _B Formula 2 can be formulated. k _R , k _G , and k _B are normalization coefficients.

Formula 2

式３ On the other hand, it is widely known that the spectral reflectance can be approximated by a low-dimensional linear sum of a small number of basis vectors. Here, assuming that the n-dimensional basis vector is b ₁ , b ₂ ... B _n (b _i is the i-th orthonormal vector obtained by principal component analysis from the spectral reflectance of the gray step chart), the spectral reflection of the object. The rate can be approximated by Equation 3. Where r is an approximate value of spectral reflectance, w ₁ , w ₂ ... W _n are basis vectors b ₁ , b _2, b _i, b _n for each achromatic patch of the gray step chart. This represents the weighting coefficient of the base vector that is added to obtain the spectral reflectance. The number (dimensions) n of the basis vectors b _i is determined for each reflector type. Then, to calculate the components of each wavelength basis vectors b _i by principal component analysis.

Formula 3

式４ Here, in many achromatic color charts of gray step charts on the market, the vector r of the approximate value of the spectral reflectance of Equation 3 is approximately approximated by the linear sum of the _two base vectors b ₁ and b _2. Since it is known that n = 2, the spectral reflectance can be expressed by Equation 4.

Formula 4

式５

式６ By substituting this approximate spectral reflectance vector r into Equation 1, Equation 5 is obtained. However, M _{3 × 2} is a 3 × 2 constant matrix. Further, Expression 6 is obtained by substituting Expression 5 into Expression 2.

Formula 5

Equation 6

_{Here, a 11, a 21, a} 31, a 12, a 22, a 32 are constants.
From this, a set of actual sensor response values d _Ri , d _Gi , d _Bi for each i-th patch and basis vector weight coefficients w ₁ , w ₂ giving the spectral reflectance of the i-th patch is gray-stepped. By obtaining for all patches in the chart, bias components β _R , β _G , and β _B are estimated by multiple regression analysis.

<First Embodiment>
Hereinafter, a first embodiment of a bias component estimation method of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 3 is a flowchart showing the flow of the bias component estimation method according to the first embodiment. Hereinafter, the process in each step will be described in detail along the flowchart of FIG. In the present embodiment, an imaging device that outputs 3-channel RGB image data is used as the imaging device.

In step 21, the basis vector calculation means reads spectral reflectance data of P achromatic (gray) patches of the gray step chart of the color chart from the storage means. Next, the number n of the basis vectors b _i is set to 2, and components for each wavelength of the basis vectors b _i are calculated by principal component analysis. Thereby, the vector components of the _two basis vectors b ₁ and b ₂ in Equation 4 are calculated, and the weight coefficient w of the basis vector that gives the spectral reflectance in Equation 4 for each i-th achromatic (gray) patch. _1i and _w2i are calculated. Step 21 stores each parameter obtained by principal component analysis in the storage means.

In step 22, the color chart imaging means captures P achromatic (gray) patches of the gray step chart and reads sensor response values (d _R , d _G , d _B ) for each pixel of the imaging device. Image data is stored in the storage means.

In step 23, the sensor response value calculation means reads out the photographed image data from the storage means, and for all P patches of the gray step chart, for each i-th achromatic (gray) patch, Values d _Ri , d _Gi , and d _Bi that are obtained by averaging the sensor response values of the pixels in the region near the center of the i-th patch are calculated and stored in the storage unit in correspondence with the patch number i.

式７

式８

式９ In the following step 24, the multiple regression analysis means reads the weight coefficients w _1i and w _2i of the basis vectors obtained in step 21 from the storage means for every i-th patch of P achromatic (gray) patches, The sensor response values d _Ri , d _Gi and d _Bi obtained in step 23 are read out, and bias components β _R , β _G and β _B are calculated by multiple regression analysis from both data in all P patches. In this calculation process, an array X of Expression 7 is formed by a group of weight coefficients w _1i , w _{2i of} basis vectors and value 1 of P achromatic (gray) patches of the gray step chart, and a sensor response value d The group Y of _Ri , d _Gi , and d _Bi forms the array Y of Expression 8. The bias components β _R , β _G , and β _B are calculated by Expression 9 using the array X and the array Y and stored in the storage means.

Equation 7

Equation 8

Equation 9

More bias component beta _R by _treatment, beta _G, beta _B by calculating the graph of the spectral reflectance of each patch despite use relative to no gray patches scale is similar, bias component beta _R, beta There is an effect that _{G 1} and β _B can be accurately calculated.

<Modification>
As described above, the embodiment of the spectral reflectance derivation method according to the present invention has been described with reference to an example. However, for example, the following modifications may be considered.

(A) Although a three-band imaging device is used, the present invention is not limited to this, and a single-band or multi-band imaging device may be used.
(B) The basis vector calculation means for calculating the basis vector b _i of the spectral reflectance of the gray step chart and the weighting coefficient of the basis vector is a component for each wavelength of the basis vector b _i by principal component analysis in the first embodiment. However, the present invention is not limited to this, and the basis vector b _i may be calculated using other methods.
(C) In the first embodiment, the number of basis vectors n is 2, but this is not restrictive, and the number of basis vectors may be 2 or more.
(D) In the first embodiment, the chart used for bias component estimation is a gray step chart. However, the present invention is not limited to this, and another color chart may be used.
(E) In the first embodiment, multiple regression analysis is used for bias component estimation. However, the present invention is not limited to this, and other nonlinear optimization methods may be used.

Claims (3)

  A bias component estimation method for estimating a bias component corresponding to a sensor response value when a subject having a spectral reflectance of 0 is photographed by an imaging device, which is a gray step chart included in a color chart used for calibration of the imaging device. A step of calculating a basis vector group of a spectral reflectance of a gray step chart and a coefficient with respect to the basis vector by principal component analysis of the spectral reflectance group, and taking each gray patch of the gray step chart with the imaging device and taking a photographed image A bias component is calculated by multiple regression analysis from a step of obtaining data, a step of obtaining a sensor response value for each gray patch from the photographed image data, and a group of weight coefficients of the sensor response value and the base vector A bias component estimation method comprising the steps of:   A bias component estimation device that estimates a bias component corresponding to a sensor response value when a subject having a spectral reflectance of 0 is photographed by an imaging device, and is a gray step chart included in a color chart used for calibration of the imaging device. By means of principal component analysis of the spectral reflectivity group, means for calculating the base vector group of the spectral reflectivity of the gray step chart and the coefficient for the base vector, and each gray patch of the gray step chart is taken by the imaging device and taken. A bias component is calculated by multiple regression analysis from means for obtaining data, means for obtaining a sensor response value for each gray patch from the captured image data, and a group of weight coefficients for the sensor response value and the base vector. A bias component estimation apparatus characterized by comprising means for performing   A bias component estimation program for estimating a bias component corresponding to a sensor response value when a subject having a spectral reflectance of 0 is photographed by an imaging device, which is a gray step chart included in a color chart used for calibration of the imaging device. A process of calculating the basis vector group of the spectral reflectance of the gray step chart and the coefficient for the base vector by principal component analysis of the spectral reflectance group, and taking each gray patch of the gray step chart with the imaging device A bias component is calculated by multiple regression analysis from a process for obtaining data, a process for obtaining a sensor response value for each gray patch from the captured image data, and a weight coefficient group for the sensor response value group and the base vector. The bias component estimation program characterized by having the process to perform.

Images