JP2010199849A - Image processor, image processing method and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology of executing demosaic processing capable of flexibly coping with a pixel array to image signals obtained in an imaging apparatus provided with a color filter for transmitting light for which a plurality of primary color components are combined. <P>SOLUTION: An image processing part 200 is provided with a parameter calculation part 201 for calculating the parameter of the approximation function of input image signals and a primary color signal calculation part 202 for complementing color information that respective pixels do not have in the input image signals by using the approximation function calculated by the parameter calculation part 201. Also, the approximation function indicates a function for approximating the signal values of respective color components with the position in the image signals of the respective pixels as a variable. A derivative is common for the approximation functions of the color components of primary colors mutually. For the approximation function of the W pixel, a constant term is defined independent of the approximation function of the color components of the primary colors. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and an imaging apparatus that performs the image processing.

  In a conventional small camera, a Bayer array is used to obtain a color image. In the Bayer array, a color filter that passes light of the primary color components red (R), blue (B), and green (G) is arranged in front of the image sensor, and a RAW having a signal of one color for each pixel. Get the data. Thereafter, a demosaic process for obtaining a color image by calculating from the surroundings the primary color signals of G and B at the position of the missing color component, for example, the pixel where the R color filter is arranged, has been performed.

  However, if a color filter is arranged in front of the pixel, the amount of light incident on the image sensor is reduced, so that the S / N is lowered. To reduce the S / N, there is a method of increasing the amount of incident light by arranging a part of a color filter that transmits light combining a plurality of primary color components. One of the color filters that transmits light combining a plurality of primary color components is a pixel (W pixel) having sensitivity to all RGB light. A proposal has been made to suppress false colors by giving a positive correlation between the RGB signal of the output image and the signal from the W pixel (for example, Patent Document 1). However, the method of Patent Document 1 can only deal with one arrangement of color filters, and cannot cope with color filter changes, digital zoom, and the like.

  On the other hand, in the demosaic process, a function that approximates a signal value for each color component of the primary color is estimated from the imaging data of the pixel at the target position and the imaging data of a plurality of pixels around the target position, and the target pixel A method for estimating the signal amount of a missing color component at a position has been proposed (for example, Patent Document 2). According to the method of Patent Document 2, it is possible to suppress color misregistration and cope with changes in pixel arrangement, color filters, and digital zoom.

  However, since a large amount of light is incident on a pixel having sensitivity to a plurality of primary color components such as a W pixel, it is more likely to be saturated than an RGB pixel having sensitivity only to the primary color component. The correlation between the saturated signal and the non-saturated signal is lost. Since the method of Patent Document 2 assumes that all signals have a strong correlation, when applied to RAW data whose correlation has been lost due to signal saturation, there is an unnatural correlation between signals that originally do not have a correlation. Will disturb the image.

JP 2002-369212 A JP 2009-010847 A

  The present invention provides a technique for performing demosaic processing that can flexibly correspond to a pixel arrangement on an image signal obtained by an imaging device having a color filter that transmits light in which a plurality of primary color components are combined.

  In order to solve the above problem, an input image signal having a signal value of one color component of a plurality of primary color components or a color component expressed by a combination of the primary color components in accordance with a predetermined arrangement rule for each pixel. An estimation unit that estimates the approximate function for each color component, which approximates the signal value of each color component and satisfies the following (a) to (c), with the position of each pixel as a variable: ) Derivatives of the approximate functions of the primary color components are equal to each other; (b) The derivative of the approximate function of the color components expressed by the combination of the primary color components is the approximate function obtained for the primary color component corresponding to the combination (C) the constant term of the approximate function of the color component expressed by the combination of the primary color components is defined independently of the approximate function of the plurality of primary color components; Multiple raw A calculation unit that obtains a signal value of the output image signal having a set of signal values of a plurality of primary color components for each pixel by substituting the position of each pixel in the output image signal for the approximate function of the component. An image processing apparatus characterized by the above and a method for image processing by the image processing apparatus are provided.

  An imaging device having sensitivity to light of a color component of one primary color component among a plurality of primary color components or light of a color component expressed by a combination of the primary color components is arranged for each pixel according to a predetermined arrangement rule. An imaging unit having an image sensor, and the position of each pixel in the input image signal captured by the imaging unit as a variable, the signal values of the respective color components are approximated, and the following (a) to (c) An estimation unit that estimates an approximate function for each color component, and (a) derivatives of the approximate functions of the primary color components are equal; (b) approximation of color components expressed by a combination of the primary color components The derivative of the function is equal to the sum of the derivatives of the approximate functions obtained for the primary color components corresponding to the combination; (c) the constant term of the approximate function of the color components expressed by the combination of the primary color components is: Above A plurality of primary color components for each pixel by substituting the position of each pixel in the output image signal for the approximate function of the obtained primary color components. And a calculation unit for obtaining a signal value of the output image signal having a set of signal values.

  Further, according to a predetermined arrangement rule for each pixel, each pixel position is obtained from an input image signal having a signal value of one color component among a plurality of primary color components or a color component signal value expressed by a combination of the primary color components. An approximation unit that approximates the signal values of the respective color components at and satisfies the following (a) to (c) for each color component, and (a) an approximation function of the primary color components (B) The approximate function of the color component expressed by the combination of the primary color components is expressed by a function obtained by adding the approximate functions obtained for the primary color components corresponding to the combination. (C) the constant terms of the approximate function for each color component are defined independently; from the obtained approximate functions of the plurality of primary color components, a plurality of primary color components for each pixel Of signal value Providing a calculation unit for determining the signal value of the output image signal having a dot and an image processing apparatus characterized by comprising a.

  ADVANTAGE OF THE INVENTION According to this invention, the demosaic process which can respond flexibly to a pixel arrangement | sequence is attained with respect to the image signal obtained with the imaging device which has the color filter which lets the light which combined the several primary color component.

1 is a diagram illustrating a configuration of an imaging apparatus according to a first embodiment. The figure explaining a color filter. The flowchart which shows operation | movement of 1st Embodiment. FIG. 3 is a diagram of an image sensor used in the first embodiment. The block diagram which shows the structure of the imaging device of 2nd Embodiment. The flowchart which shows operation | movement of 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described. Here, the same code | symbol is attached | subjected to a mutually similar structure, and the overlapping description is abbreviate | omitted.

RAW data (hereinafter referred to as input image signal) captured by an image sensor such as a single-plate color CCD / CMOS image sensor used in many digital cameras has a signal value of a single color component in each pixel. ing. Therefore, for each pixel, it is necessary to perform demosaic processing that complements the color component by deriving information on the color component that the pixel does not have from the surrounding pixels and generates a full-color image signal. In each of the following embodiments, a method for obtaining an output image signal having pixel values of three primary color components by complementing missing color information for each pixel of a captured input image signal will be exemplified. In each of the following embodiments, a case will be described in which a color component expressed by a combination of primary color components is a white (W) pixel (luminance component) that is the sum of RGB. However, the color component expressed by the combination of the primary color components is not necessarily white (W), and may be a color such as magenta (Mg), cyan (Cy), and yellow (Ye). For example, Mg may be a combination (sum) of primary color components R and B.

[First Embodiment]
FIG. 1 is a diagram illustrating an imaging apparatus 1 according to the present embodiment. The imaging device 1 performs demosaic processing on the imaging unit 100 and an input image signal captured by the imaging unit 100, and outputs an output image signal that is a color image in which each pixel has all of the R, G, and B components. And a processing unit 200.

  The imaging unit 100 includes a lens 101 that collects light from a subject, and an image sensor 102 that photoelectrically converts the light collected for each pixel to generate an electric signal that is a signal value of an input image signal. The image sensor 102 includes a color filter in which a single color component is assigned to each pixel position according to a predetermined arrangement rule. The signal value obtained from each pixel of the image sensor 102 is an electrical signal corresponding to the color filter corresponding to the pixel. For example, a red component signal value is obtained from a pixel having a red (R) filter, but other blue (B) and green (G) component signal values are not obtained. Note that a control unit such as a diaphragm (not shown) that adjusts the amount of light incident on the lens 101 may be provided.

  The image processing unit 200 performs demosaic processing for interpolating and generating signal values of color components that cannot be obtained from the image sensor 102 using signal values of peripheral pixels. Specifically, the parameter calculation unit 201 that calculates the parameters of the approximate function of the input image signal and the approximate function calculated by the parameter calculation unit 201 are used to complement the color information that each pixel did not have in the input image signal. And a primary color signal calculation unit 202. The approximate function refers to a function that approximates the signal value of each color component using the position of each pixel in the image signal as a variable. Details will be described later.

  Next, the relationship between the color filter and the color component of the input image signal will be described.

  FIG. 2 is a diagram illustrating an example of the arrangement rule of the color components of the color filter used in the present embodiment. (x, y) is a two-dimensional coordinate system indicating the pixel position on the sensor, and the origin is the center of the target pixel, that is, the target pixel 425. The W pixel is displayed as white, the B pixel is displayed as a stripe to the left, the R pixel is displayed as a dot, and the G pixel is displayed as a vertical stripe. Since the B color filter is arranged at the B pixel position, the R color filter of the R pixel, the G color filter at the G pixel position, and the colorless filter at the W pixel position, 3 A signal value obtained by adding the light of all the primary colors is obtained. The signal value of the B color component is obtained from the image sensor of the image sensor 102 at a position corresponding to the B pixel. Similarly, the signal value of the R color component from the image sensor at the position corresponding to the R pixel, the signal value of the G color component from the image sensor at the position corresponding to the G pixel, and the signal value of the position corresponding to the W pixel. A signal value of the W color component is obtained from the image sensor. The W pixel does not have a color filter and has sensitivity to all three primary colors (RGB: red, green, and blue) of light.

The image processing unit 200 generates an output image signal that has been demosaiced so that all pixels have all R, G, and B components based on input image signal values having different color components for each pixel. For example, a pixel captured at the pixel position of the R component illustrated in FIG. In that case, the magnitudes of the signal values of the G component, the B component, and the W component at the pixel position (x ₄₂₅ , y ₄₂₅ ) = () of the pixel of interest 425 are estimated from the signal values of pixels around the pixel of interest 425. Then, an output image signal having the three primary colors (R, B, G) including the estimated B, G signal values as one set is output.

  Next, the operation of the imaging apparatus 1 will be described.

  FIG. 3 is a flowchart showing the operation of the imaging apparatus 1. An example in which an image is captured with the arrangement of the color components of the color filter shown in FIG. 2 will be described.

  First, an input image signal obtained by imaging by the image sensor 102 is sent to the image processing unit 200 (step 1).

Next, the parameter calculation unit 201 sets one pixel of the input image signal as a target pixel (step 2). Hereinafter, for example, the pixel position of the pixel 401 is (x ₄₀₁ , y ₄₀₁ ), the color filter of the pixel 401 is c ₄₀₁ , and the signal value obtained from the pixel 401 is labeled using subscripts such as s ₄₀₁ .

Next, the parameter calculation unit 201 uses the signal value of a block (hereinafter referred to as a processing block) of 7 × 7 pixels (hereinafter, the unit is omitted) around the pixel of interest as the center, and the color of each color filter. A parameter of the approximate function corresponding to the component is derived (step 3). The approximate function is an expression that approximates the signal value at the pixel position (x, y). Equation (1) shows an approximate function for each primary color when a polynomial is used.

The three approximate functions f _R (x, y), f _G (x, y), and f _B (x, y) in Equation (1) have the same derivatives. In other words, depending on the variable position x or y, the partial derivatives of the first derivative of f _R (x, y), f _G (x, y), f _B (x, y) are the same function. Become. That is, the shape of the function excluding the constant term is the same for the approximate functions of the three primary color components. This condition matches the property of the image that the hue hardly changes when viewed locally. In addition, the parameters β = (β _R , β _G, β _B , β _1, β ₂ , β _3, β ₄ , β ₅ ) ^t of the approximate function of Equation (1) are all real numbers and are estimated from the input image signal. To do.

An approximation function derived from the signal values obtained from all the WRGB pixels and approximating the signal value of the W pixel is represented by Expression (2).

w _R , w _G , and w _B are positive real numbers that have larger values as the spectral sensitivity of the W pixel with respect to the wavelength of light corresponding to the primary color signal values R, G, and B increases. As in Expression (3), the constant term of f _W (x, y) is defined independently of f _R (x, y), f _G (x, y), and f _B (x, y). This value can be determined from design data of the image sensor 102. With this formulation, the derivative of f _W (x, y) becomes the weighted sum of the derivatives of f _R (x, y), f _G (x, y), and f _B (x, y). That is, since the color component of W is represented by a combination of R, G, and B, the derivative of the approximate function of the color component of W is defined as the sum of the derivatives of the primary color components representing the combination. Also, a vector having an approximate function parameter is expressed as in equation (3).

The estimated value β ′ of this parameter vector is

Is a parameter that minimizes the sum of squares of the difference between the signal value of the input image signal at each pixel position and the value estimated by the approximation function. That is, a value that satisfies Equation (5) is obtained. Incidentally, (x _i, y _i) represents the position of the pixels included in the processing block, c _i represents the color of a color filter assigned to each pixel, s _i is the pixel location (x _i, y _i) The signal value obtained from is shown.

w _i is a weight given to each pixel, and is a positive real number. A specific value may be w _i = 1.

In Expression (6), the weight of a pixel that is far from the target pixel may be relatively small. σ is a constant and may be determined while evaluating the image quality at the design stage. Note that C in Expression (6) may be switched to a matrix as in Expression (7) according to the presence or absence of an edge in the processing block and its direction. Note that an edge refers to a region where a change in signal value occurs linearly in an image, and corresponds to, for example, a contour line of a subject.

By using C ₁ instead of C for processing blocks whose vertical direction is along the edge, and C ₂ instead of C for processing blocks whose horizontal direction is along the edge, a sense of resolution is obtained. S / N can be improved while maintaining. As an algorithm for solving the minimization problem in the form of Equation (6), a number of methods such as an iterative algorithm such as the steepest descent method and the conjugate gradient method, and a generalized inverse matrix are disclosed.

Next, the primary color signal calculation unit 103 substitutes the coordinates of the pixel of interest into the obtained approximate functions f _R (x, y), f _G (x, y), f _B (x, y), and obtains each point of the image. The primary color components are calculated (step 4). That is, the values f _R (x ₄₂₅ , y ₄₂₅ ), f _G (x ₄₂₅ ) obtained by substituting (x ₄₂₅ , y ₄₂₅ ) for the coordinates (x ₄₂₅ , y ₄₂₅ ) of the pixel of interest into the approximate function of equation (2). , y ₄₂₅ ), f _B (x ₄₂₅ , y ₄₂₅ ) are calculated as output image signals.

  Next, it is determined whether the output image signal has been calculated for all the pixels of the input image signal (step 5).

  When the output image signal is calculated for all the pixels of the input image signal (step 5, No), the process returns to step 2 and the next pixel is set as the target pixel. When the output image signal is calculated for all the pixels of the input image signal (step 5, Yes), the output image signal for one frame is output, and the process is terminated.

In the present embodiment, the constant term of the approximate function of the color component of the W pixel is defined independently of f _R (x, y), f _G (x, y), and f _B (x, y). This effect will be described.

For example, consider a case where an approximation function of W pixels having sensitivity to all RGB signals is simply modeled as a sum of approximation functions of all RGB. An approximation function of the W pixel based on the model is shown in Expression (8).

The W pixel is characterized by being easily saturated because of its high sensitivity to light. Saturation means a case where the image sensor exceeds a signal value of light that can be physically imaged.

If it is simply modeled as the sum of the approximate functions of all RGB, the image quality will deteriorate when saturation occurs in the W pixel. Specifically, the constant term of f ' _W (x, y) is represented by the weighted sum of the constant terms of f _R (x, y), f _G (x, y), f _B (x, y). The If the constant term of f ′ _W (x, y) is reduced to reduce the value taken by f _W (x, y) in accordance with the signal value of the saturated W pixel, f _R (x, y), f The values of _G (x, y) and f _B (x, y) become small. As a result, the RGB color component signal values that would have been obtained when a set of all the primary color components was actually acquired at the time of imaging were obtained. Therefore, in this embodiment, the constant term of f _W (x, y) is independent of f _R (x, y), f _G (x, y), and f _B (x, y) as shown in Equation (2). By defining it, the change of f _W (x, y) has less influence on f _R (x, y), f _G (x, y), f _B (x, y). Since the relationship of the derivative of the approximate function is maintained between the W pixel and the R / G / B pixel, the effect of suppressing the false color is not lost.

  In the image processing unit 100 of the present embodiment, the case where the color of the color filter of the pixel of interest at the center of the block is R is exemplified, but any of G, B, and W may be used. The point that the signal value of the primary color is calculated by the method using the equations (1) to (6) remains the same.

  It is also possible to carry out with the following changes.

  The approximation function representing the primary color signal value is not necessarily in the form of equation (1). For example, equation (1) is a quadratic polynomial, but the order may be second or higher. In addition, equation (1) is a form in which an arbitrary function is expanded by Macrolin, but other forms may also be used. For example, if Fourier series expansion is used, Equation (1) becomes a weighted sum of trigonometric functions. In addition to trigonometric functions, basis functions used for wavelet analysis may be used. However, when the weighted sum of a plurality of basis functions is used, it is easy to find a solution that satisfies Equation (5). Further, it is desirable that the sum of the number of parameters of the approximate function representing the signal value of the primary color and the number of parameters of the approximate function of the color component expressed by the combination of the primary color components is equal to or less than the number of pixels used for parameter estimation.

  The shape of the processing block is not necessarily a rectangle as shown in FIG. For example, a process in which four pixels corresponding to the corners of a rectangular block are not included in the processing block may be used.

  Signal values of primary colors other than the target pixel at the block center may be calculated. In the first embodiment, the primary color signal calculation unit 103 substitutes the coordinates of the pixel of interest in the approximate function. However, the primary color signal value at an arbitrary position in the block can be calculated by changing the substituted coordinates. Is possible. By substituting the coordinates of the pixel gap, it can also be used for pixel interpolation.

  The arrangement rule of each color component of the color filter of the image sensor 102 used in the imaging unit of the present embodiment is not limited to the arrangement rule of the color component shown in FIG. For example, the WRGB color components may be arranged as shown in FIG.

For example, if the color component expressed by the combination of the primary color components is not W but magenta (Mg), magenta is the sum of R and B, and therefore w _R = w _B in equation (2). By setting = 1 and w _G = 0, demosaic processing can be performed similarly to the W pixel.

Various methods can be used to solve Equation (5). For example, if equation (5) is transformed

And can. K is a diagonal matrix with weight w _i

It is. S is a vector of signal values

It is. The matrix W is a matrix that describes the arrangement of pixels.

It is. Δ varies depending on the arrangement rule of the color components of the color filter,

It is represented by The solution of equation (9) can be calculated by equation (14) with the generalized inverse matrix of matrix W as W ⁺ .

Since the matrix W depends on the arrangement of pixels and color filters, only a finite number of patterns appear in an image sensor having a periodic structure. For example, even if the arrangement structure of the color components of the image sensor shown in FIG. 2 is arranged on the entire surface, only four patterns depending on the color filter of the central pixel appear. Further, the value of the matrix K can also be limited to a finite pattern even when the matrix C of Expression (7) is changed finely. In that case, a matrix (KW) ⁺ K corresponding to each color filter and C is calculated in advance and stored in the memory, and the matrix W corresponding to the color filter is read from the memory and convolved according to the equation (14) at the time of imaging. By performing the calculation, it is possible to reduce processing with high calculation costs such as a generalized inverse matrix for each pixel and iterative calculation.

[Second Embodiment]
FIG. 5 is a diagram illustrating the imaging apparatus 2 of the present embodiment. The imaging apparatus 1 of FIG. 1 is different in that it further includes a saturation determination unit 601 and parameter calculation units 602 and 603.

  The saturation determination unit 601 determines how many saturated W pixels whose signal value exceeds an imageable value are included in the processing block corresponding to the target pixel. According to the determination result, it is determined which of the parameter calculation units 201, 602, and 603 should derive the approximate function of the target pixel. The parameter calculation units 602 and 603 derive the approximation function by a method different from that of the parameter calculation unit 201.

The parameter calculation unit 602 refers to only the signal values obtained from the RGB pixels, that is, the primary color components, and approximate functions f _R (x, y), f _G (x, y), f _B (x , Y) parameters are calculated. That is, the parameter shown in the following formula (15) that satisfies the formula (5) is calculated.

Note that f (x _i , y _i , z _i ) shown in Expression (5) is changed to Expression (16).

The parameter calculation unit 603 refers to the input image signal of each RGBW pixel, and approximates the function f _R (x, y), f _G (x, y), f _B (x, y) of equation (1) and the equation The parameter of the approximate function f ′ _W (x, y) in (8) is calculated. As described above, the approximate function of Equation (8) is a function defined by the sum of the approximate functions of the primary color components.

  Next, the operation of the imaging device 2 of the present embodiment will be described.

  FIG. 6 is a flowchart illustrating the operation of the imaging apparatus according to the present embodiment.

  First, an input image signal is acquired from the imaging unit 100 (step 11). The arrangement rule of the color components of the image sensor of the imaging unit 100 of the present embodiment is as shown in FIG. 4 as in the first embodiment.

  Next, a pixel in the input image signal is set as a target pixel (step 12).

  Next, the saturation determination unit 601 counts the number of pixels reaching the saturation level among the W pixels of the input image signal included in the processing block (hereinafter referred to as a saturated W pixel prime number) (step 13). In the method of determining the saturation level, a predetermined threshold value is provided and a pixel having a pixel value equal to or higher than the threshold value is set as a saturation pixel. When saturated, the signal value is clipped and becomes a constant value. Note that since a saturated signal value may slightly change during reading, a value obtained by subtracting a predetermined value from a constant value when clipping is preferably used as a threshold value.

  The saturation determination unit 601 determines whether the counted number of saturated W pixels is larger than a predetermined reference N1 (step 14).

  When the number of saturated W pixels is larger than N1 (Yes in step 14), the parameter calculation unit 602 refers to only the primary color component signal values and derives the parameters of the approximation function shown in the equation (1).

  When the number of saturated W pixels is smaller than N1 (No at Step 14), the saturation determination unit 601 further determines whether the number of saturated W pixels is larger than 0 (Step 16). Note that the criterion in step 16 may be N2 (0 <N2 <N1) instead of 0.

  If there is no saturated W pixel in the processing block (step 16, Yes), the parameter calculation unit 603 derives the parameters of the approximation function of the equations (1) and (8) (step 17).

  When there is a saturated W pixel in the processing block (No in step 16), the parameter calculation unit 201 derives the parameters of the approximation functions of the expressions (1) and (2) as in the first embodiment (step 18).

  The primary color component calculation unit 202 substitutes the coordinates of the target pixel for the approximate function (1) derived in steps 15, 17, and 18 to obtain an output image signal (step 19).

  Next, it is determined whether the output image signal has been calculated for all the pixels of the input image signal (step 20).

  When the output image signal is calculated for all the pixels of the input image signal (No at Step 20), the process returns to Step 12, and the next pixel is set as the target pixel. When the output image signal is calculated for all the pixels of the input image signal (step 20, Yes), the obtained output image signal for one frame is output (step 21), and the process is terminated.

  As described above, according to the imaging apparatus of the present embodiment, it is possible to output a color image with higher image quality by switching the method of deriving the approximate function according to the number of saturated pixels.

  Specifically, if the approximate function including the signal value of the W pixel is derived when the number of saturated W pixels in the processing block is large, a shift occurs in the approximate function of the primary color component, and the image quality deteriorates. In the present embodiment, when the number of saturated W pixels is larger than the reference, the approximation function is derived only with the primary color components without including the signal value of W pixels and the approximation function, and therefore, robustness against saturation is achieved.

  Further, when there is no saturated W pixel in the processing block, even if the condition that the constant term of the approximate function of the W pixel is the sum of the RGB pixels is added, the image is not deteriorated and is more preferable. An approximate function can be derived.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  In addition, the image processing unit of the imaging device of each of the above embodiments can be realized by using, for example, a general-purpose computer device as basic hardware. The program to be executed has a module configuration including each function described above. The program is an installable or executable file that is recorded on a computer-readable recording medium such as a CD-ROM, CD-R, DVD, etc. Also good.

DESCRIPTION OF SYMBOLS 1, 2 ... Imaging device 100 ... Imaging part 101 ... Lens 102 ... Image sensor 200, 210 ... Image processing part 201 ... Parameter calculation part 202 ... Primary color signal calculation part 601 ... Determination units 602, 603 ... Parameter calculation unit

Claims (9)

According to a predetermined arrangement rule for each pixel, the position of each pixel of the input image signal having a signal value of one color component of a plurality of primary color components or a color component expressed by a combination of the primary color components is used as a variable, An estimator for approximating the signal values of the respective color components and satisfying the following (a) to (c), and estimating an approximate function for each color component;
(A) the derivatives of the approximate functions of the primary color components are equal;
(B) The derivative of the approximate function of the color component expressed by the combination of the primary color components is equal to the sum of the derivatives of the approximate functions obtained for the primary color component corresponding to the combination;
(C) the constant term of the approximate function of the color component expressed by the combination of the primary color components is defined independently of the approximate function of the plurality of primary color components;
A signal value of the output image signal having a set of signal values of a plurality of primary color components is obtained for each pixel by substituting the position of each pixel in the output image signal for the obtained approximate function of the plurality of primary color components. A calculation unit;
An image processing apparatus comprising: A determination unit that obtains the number of pixels whose signal value exceeds a reference among pixels having a signal value of a color component that is included in a peripheral region of a target pixel to be processed and expressed by a combination of the primary color components; ,
The estimation unit expresses the plurality of primary color components by expressing a constant term of an approximate function of a color component expressed by a combination of the primary color components when the number of signal values exceeding a reference is equal to or less than a first reference. The weighted sum of the constant terms of the approximate function
The image processing apparatus according to claim 1. The estimation unit derives only an approximate function of the primary color component when the number of signal values exceeding a reference is equal to or greater than a second reference greater than the first reference;
The image processing apparatus according to claim 2.   The image according to any one of claims 1 to 3, wherein the approximation function that expresses the plurality of primary color components and the approximation function that expresses a color component expressed by a combination of the primary color components are polynomials. Processing equipment.   The approximate function expressing the plurality of primary color components and the approximate function expressing the color component expressed by the combination of the primary color components are weighted sums of trigonometric functions. The image processing apparatus according to claim 1. According to a predetermined arrangement rule for each pixel, the position of each pixel of the input image signal having a signal value of one color component of a plurality of primary color components or a color component expressed by a combination of the primary color components is used as a variable, Approximating the signal values of the respective color components and estimating an approximate function satisfying the following (a) to (c) for each color component;
(A) the derivatives of the approximate functions of the primary color components are equal;
(B) The derivative of the approximate function of the color component expressed by the combination of the primary color components is equal to the sum of the derivatives of the approximate functions obtained for the primary color component corresponding to the combination;
(C) the constant term of the approximate function of the color component expressed by the combination of the primary color components is defined independently of the approximate function of the plurality of primary color components;
A signal value of the output image signal having a set of signal values of a plurality of primary color components is obtained for each pixel by substituting the position of each pixel in the output image signal for the obtained approximate function of the plurality of primary color components. Steps,
An image processing method comprising: An image sensor having sensitivity to light of a color component of one primary color component among a plurality of primary color components or light of a color component expressed by a combination of the primary color components is arranged for each pixel according to a predetermined arrangement rule. An imaging unit having an image sensor;
The position of each pixel in the input image signal imaged by the imaging unit is a variable, the signal value of each color component is approximated, and an approximate function that satisfies the following (a) to (c) is set for each color component: An estimation unit for estimating
(A) the derivatives of the approximate functions of the primary color components are equal;
(B) The derivative of the approximate function of the color component expressed by the combination of the primary color components is equal to the sum of the derivatives of the approximate functions obtained for the primary color component corresponding to the combination;
(C) the constant term of the approximate function of the color component expressed by the combination of the primary color components is defined independently of the approximate function of the plurality of primary color components;
A signal value of the output image signal having a set of signal values of a plurality of primary color components is obtained for each pixel by substituting the position of each pixel in the output image signal for the obtained approximate function of the plurality of primary color components. A calculation unit;
An imaging apparatus comprising: The imaging apparatus according to claim 7, wherein the imaging unit includes an imaging device having sensitivity to W that is light of a color component expressed by a combination of the primary color components. According to a predetermined arrangement rule for each pixel, from an input image signal having a signal value of one color component among a plurality of primary color components or a color component signal value expressed by a combination of the primary color components at each pixel position An estimation unit that approximates the signal value of each of the color components and estimates an approximation function that satisfies the following (a) to (c) for each color component;
(A) the shapes represented by the approximate functions of the primary color components are substantially equal;
(B) The approximate function of the color component expressed by the combination of the primary color components is substantially equal to the shape expressed by the function obtained by adding the approximate functions obtained for the primary color components corresponding to the combination;
(C) the constant term of the approximate function for each color component is defined independently;
A calculation unit for obtaining a signal value of the output image signal having a set of signal values of a plurality of primary color components for each pixel from the approximate function of the plurality of primary color components obtained;
An image processing apparatus comprising:

Images