JP2012169989A - Imaging device and interpolation processing method for imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device capable of obtaining a favorable interpolation image with no false color even when an image contains many high frequency components.SOLUTION: An imaging device 1 comprises: a monochrome image sensor 4 that images a subject to output a luminance signal; a color image sensor 5 that images the subject to output each color signal of an R signal, a G signal, and a B signal; a correlation direction index calculation unit 11 that calculates a correlation direction index representing a high correlation direction at each pixel based on the luminance signal; a signal interpolation unit 12 that acquires correlation direction pixels positioned in a high correlation direction based on the correlation direction index for a target pixel as a target of interpolation value acquisition, and acquires an interpolation value of a color signal at the target pixel using color signals of the correlation direction pixels; and an interpolation value calculation unit 13 that calculates an interpolation value of a color signal which is not acquired using the interpolation value acquired by the signal interpolation unit 12, the luminance signal, and the color signal output from the color image sensor 5.

Description

  The present invention relates to an imaging apparatus and an interpolation processing method of the imaging apparatus that prevent generation of false colors even in an image including a lot of high-frequency components in a two-plate imaging apparatus.

  In a color imaging apparatus, incident light (subject image) is separated into three colors of RGB using a color separation optical system (prism) and imaged on three monochrome imaging elements to obtain three primary color signals of each pixel. Incident light is imaged on a three-plate imaging device or a single color imaging device in which an on-chip color filter (hereinafter referred to as OCF) is formed on each pixel, and the three primary colors of each pixel are obtained by signal processing (interpolation calculation). There are single-plate imaging devices that obtain signals. Further, the incident light is separated by using a separation optical system (beam splitter), imaged on each of the monochrome imaging element and the color imaging element, and the output signals of the two imaging elements are subjected to signal processing. There is a two-plate imaging device that obtains three primary color signals of each pixel.

  The three-plate type image pickup device is large because of the use of a prism and is expensive because of the use of three image pickup devices. However, since the three primary color signals of each pixel can be obtained without using an interpolation operation, Thus, it is possible to accurately obtain each color signal of a subject image even for a subject image having a high spatial frequency having a fine repetitive pattern. On the other hand, the single-plate image pickup apparatus does not require a prism and uses only one image pickup device, so that the apparatus is small and inexpensive. However, a false color is generated when the spatial frequency of the subject image is high to perform interpolation calculation. Sometimes.

  Therefore, a two-plate type imaging device has been proposed that is smaller and cheaper than a three-plate type imaging device and can obtain a higher quality image than a single-plate type imaging device. For example, a two-plate type CCD camera disclosed in Patent Document 1 is proposed. is there.

JP-A-6-121326

 However, in the conventional two-plate CCD camera disclosed in Patent Document 1 described above, the interpolated value of the R signal is calculated from only the R signal in the vicinity of the pixel of interest regardless of the direction of high correlation in each pixel. Since the interpolated value of the B signal is calculated from only the B signal, an image having a high spatial frequency, for example, a stripe pattern having a narrower pitch than the arrangement pitch of the R or B color filters of the color image sensor. In the image, there is a problem that false color is generated because proper color demodulation cannot be performed. For example, a black-and-white striped pattern image with a narrow arrangement pitch is output as a cyan or yellow striped pattern.

 Therefore, the present invention has been proposed in view of the above-described circumstances, and there is provided an imaging apparatus and an imaging apparatus capable of obtaining a good interpolated image having no false color even if the image contains a lot of high-frequency components. An object is to provide an interpolation processing method.

  In order to achieve the above-described object, an imaging apparatus according to the present invention includes a first imaging device that images a subject and outputs a luminance signal, and each color signal of an R signal, a G signal, and a B signal by imaging the subject. A second image sensor having the same number of pixels as the first image sensor, and a correlation direction index calculation unit that calculates a correlation direction index representing a direction with high correlation in each pixel based on the luminance signal; Then, a correlation direction pixel located in a highly correlated direction is obtained based on the correlation direction index for the target pixel for which an interpolation value is to be obtained, and the color signal of the target pixel is interpolated by the color signal of the correlation direction pixel. A signal interpolation unit for obtaining a value, an interpolation value of the color signal not obtained using the interpolation value of the color signal obtained by the signal interpolation unit, the luminance signal, and the color signal output from the second image sensor Calculate value Characterized in that it comprises an interpolation value calculating section for.

  The signal interpolation unit of the imaging apparatus according to the present invention may be configured to calculate an average of the G signals of the correlation direction pixels when the target pixel is a B pixel that outputs the B signal and an R pixel that outputs the R signal. A G signal interpolating unit that obtains a value as an interpolation value, and the target pixel is a G pixel that outputs the G signal, and when the correlation direction pixel is the R pixel, the R signal of the correlation direction pixel An R signal interpolation unit for obtaining an average value as an interpolation value, and a B signal of the correlation direction pixel when the pixel of interest is a G pixel that outputs the G signal and the correlation direction pixel is the B pixel And a B signal interpolating unit for obtaining an average value as an interpolated value.

  Furthermore, the interpolation value calculation unit of the imaging apparatus according to the present invention calculates an interpolation value of the R signal using the interpolation value of the G signal, the luminance signal, and the B signal in the B pixel, and the G pixel. An R interpolation value calculation unit that calculates an R signal interpolation value using the B signal interpolation value, the luminance signal, and the G signal, and the G signal interpolation value and the luminance signal in the R pixel. B interpolation using the R signal and the R signal, and the B signal calculating the B signal interpolation value using the R signal interpolation value, the luminance signal, and the G signal in the G pixel. And a value calculation unit.

  According to the interpolation processing method of the imaging apparatus according to the present invention, the subject is imaged by the first imaging element and a luminance signal is output, and the subject is imaged by the second imaging element having the same number of pixels as the first imaging element. A step of outputting each color signal of the R signal, the G signal, and the B signal in accordance with a Bayer array, and a correlation direction index calculating step for calculating a correlation direction index indicating a highly correlated direction in each pixel based on the luminance signal And a correlation direction pixel located in a highly correlated direction based on the correlation direction index for the target pixel for which an interpolation value is to be calculated, and the color signal of the target pixel is determined by the color signal of the correlation direction pixel. A signal interpolation step for obtaining an interpolation value, an interpolation value of the color signal obtained in the signal interpolation step, the luminance signal, and the color signal output from the second image sensor are used. Te, characterized in that it comprises an interpolation value calculating step of calculating an interpolated value of the color signal which is not required.

  According to the imaging apparatus and the interpolation processing method of the imaging apparatus according to the present invention, a direction with high correlation in each pixel is obtained based on the luminance signal, and an interpolation value is obtained using the color signal of the pixel in the direction with high correlation. Therefore, even if the image contains a lot of high frequency components, a good interpolation image without false color can be obtained.

It is a block diagram which shows the structure of the imaging device which concerns on one Embodiment to which this invention is applied. It is a flowchart which shows the process sequence of the interpolation process by the imaging device which concerns on one Embodiment to which this invention is applied. It is a figure for demonstrating the interpolation process by the imaging device which concerns on one Embodiment to which this invention is applied. It is a figure for demonstrating the interpolation process by the imaging device which concerns on one Embodiment to which this invention is applied. It is a figure for demonstrating the interpolation process by the imaging device which concerns on one Embodiment to which this invention is applied. It is a figure for demonstrating the interpolation process by the imaging device which concerns on one Embodiment to which this invention is applied. It is a figure for demonstrating the interpolation process by the imaging device which concerns on one Embodiment to which this invention is applied. It is a figure for demonstrating the interpolation process by the imaging device which concerns on one Embodiment to which this invention is applied. It is a figure for demonstrating the interpolation process by the imaging device which concerns on one Embodiment to which this invention is applied. It is a figure for demonstrating the interpolation process by the imaging device which concerns on one Embodiment to which this invention is applied.

  Hereinafter, an embodiment to which the present invention is applied will be described with reference to the drawings.

[Configuration of imaging device]
FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus according to the present embodiment. As shown in FIG. 1, an imaging apparatus 1 according to the present embodiment includes a lens 2 on which light from a subject is incident, a separation optical system 3 that separates a subject image incident through the lens 2 in two directions, and a subject. Monochrome image pickup device 4 that picks up an image and outputs a luminance signal, and a color image pickup device having the same number of pixels as the monochrome image pickup device 4 that picks up a subject and outputs each color signal of an R signal, a G signal, and a B signal according to a Bayer array 5 and an interpolation processing unit 6 that performs interpolation processing to generate the three primary color signals P of each pixel.

  The imaging apparatus 1 according to the present embodiment separates the subject image incident through the lens 2 in two directions by the separation optical system 3 and forms it on the monochrome imaging element 4 and the color imaging element 5. Here, the monochrome imaging device 4 has characteristics almost equal to human visibility characteristics, and the color imaging device 5 has a Bayer array OCF (on-chip color filter) formed on pixels. The output SD1 of the monochrome imaging device 4 and the output SD2 of the color imaging device 5 are input to the interpolation processing unit 6, and the three primary color signals P of each pixel are generated by the interpolation calculation.

  Here, since the RGB balance (so-called white balance) changes depending on the illumination condition (color temperature), the gain of each color of the color image sensor 5 is adjusted so that R = G = B for an achromatic subject. The gain of the monochrome image sensor 4 is adjusted so that Y = R = G = B for an achromatic subject.

[Configuration of interpolation processing unit]
As shown in FIG. 1, the interpolation processing unit 6 according to the present embodiment includes a correlation direction index calculation unit 11 that calculates a correlation direction index that represents a highly correlated direction in each pixel based on the luminance signal, and an interpolation value. A signal interpolation unit that obtains a correlation direction pixel located in a highly correlated direction based on a correlation direction index for a target pixel to be obtained, and obtains an interpolated value of the color signal in the target pixel from the color signal of the correlation direction pixel 12 and the interpolation value calculation for calculating the interpolation value of the color signal not obtained using the interpolation value of the color signal obtained by the signal interpolation unit 12, the luminance signal, and the color signal output from the color image sensor 5. Part 13.

  Further, when the pixel of interest is a B pixel that outputs a B signal and an R pixel that outputs an R signal, the signal interpolation unit 12 obtains an average value of G signals of correlation direction pixels as an interpolation value. When the target pixel is a G pixel that outputs a G signal and the correlation direction pixel is an R pixel, the R signal interpolation unit 23 that obtains an average value of the R signals of the correlation direction pixel as an interpolation value, In the case of a G pixel that outputs a G signal and the correlation direction pixel is a B pixel, a B signal interpolation unit 22 that obtains an average value of the B signals of the correlation direction pixel as an interpolation value is provided.

  Further, the interpolation value calculation unit 13 calculates the interpolation value of the R signal using the interpolation value of the G signal, the luminance signal, and the B signal in the B pixel, and the interpolation value of the B signal, the luminance signal, and the G signal in the G pixel. The R interpolation value calculation unit 31 for calculating the interpolation value of the R signal using the R, and the interpolation value of the B signal using the interpolation value of the G signal, the luminance signal, and the R signal in the R pixel, A B interpolation value calculation unit 32 is provided that calculates an interpolation value of the B signal using the interpolation value of the R signal, the luminance signal, and the G signal.

[Interpolation procedure]
Next, the procedure of interpolation processing by the imaging apparatus 1 according to the present embodiment will be described with reference to the flowchart of FIG. Here, a case where a subject having a black and white vertical stripe pattern having a period equal to the pixel pitch of the image sensor is photographed will be described as an example. In order to facilitate understanding, the pixel arrangement of the monochrome imaging device 4 and the color imaging device 5 is shown as one schematic diagram on the same coordinate axis as shown in FIG. Is shown in FIG. Also, the coordinates in the horizontal direction x = i and the vertical direction y = j are indicated by a subscript (i, j).

  As shown in FIG. 2, in step S101, the correlation at each pixel is high based on the luminance signal output Y (i, j) of the monochrome imaging device 4 of the pixel located at the coordinate (i, j). A correlation direction index D (i, j) representing the direction is calculated. A specific calculation method is as follows. The luminance signal Y (i, j-1) of two adjacent pixels above and below is Y (i, j), where Y (i, j) is the luminance signal of the target pixel whose interpolation value is to be obtained. And Y (i, j + 1) average value Ya_v (i, j), and left and right adjacent two pixels Y (i-1, j) and Y (i + 1, j) average value Ya_h (i, j) is calculated, and the direction in which the difference between the average value and the luminance signal Y (i, j) of the pixel of interest is small is defined as a highly correlated direction. If D (i, j) = 1 and the correlation in the horizontal direction is high, the correlation direction index D (i, j) = 0 can be defined.

D (i, j) = 0 (when | Ya_v (i, j) −Y (i, j) |> | Ya_h (i, j) −Y (i, j) |)
= 1 ... (when | Ya_v (i, j) -Y (i, j) | = | Ya_h (i, j) -Y (i, j) |)
= 1 (when | Ya_v (i, j) −Y (i, j) | <| Ya_h (i, j) −Y (i, j) |)) (1)
Here, when | Ya_v (i, j) −Y (i, j) | = | Ya_h (i, j) −Y (i, j) |, the degree of correlation is equal in both the horizontal and vertical directions. In the present embodiment, priority is given to the vertical direction. However, the present invention is not limited to this.

  Next, in step S102, the interpolation value of the G signal at the coordinates of x = odd, y = even (hereinafter referred to as B pixel or coordinates (o, e)) where the B OCF is formed in the color image sensor 5. g (o, e) is calculated by the following equation. A specific calculation method is as follows. When the correlation direction index of the pixel of interest is D (i, j) = 1, that is, when the correlation in the vertical direction is strong, the G signals G (i, j− The average value of 1) and G (i, j + 1) is calculated as an interpolated value g (i, j) of the G signal, and when the correlation direction index of the pixel of interest is D (i, j) = 0, that is, horizontal When the direction correlation is strong, the average value of G signals G (i−1, j) and G (i + 1, j) of two pixels adjacent to the left and right is set as an interpolated value g (i, j) of the G signal. calculate.

g (i, j) = (G (i, j-1) + G (i, j + 1)) / 2 (when D (i, j) = 1)
g (i, j) = (G (i-1, j) + G (i + 1, j)) / 2 (when D (i, j) = 0) (2)
Next, in step S103, the interpolation value of the G signal at the coordinates of x = even and y = odd (hereinafter referred to as R pixels or coordinates (e, o)) where the R OCF is formed in the color image sensor 5. g (e, o) is calculated. As a specific calculation method, the average value of the G signals of the correlation direction pixels is calculated as the interpolation value g (i, j) of the G signal as in step S102. Specific calculation results obtained in steps S102 and S103 are shown in FIGS.

  Next, in step S104, an interpolation value r (o, e) of the R signal at the coordinates (o, e) where the B OCF is formed in the color image sensor 5 is calculated. A specific calculation method includes an output Y (i, j) of the monochrome image pickup element 4 of the target pixel, an output B (i, j) of the color image pickup element 5, and an interpolation value g (G signal calculated in step S102). i, j) and the following equation.

r (i, j) = (Y (i, j) −0.6 × g (i, j) −0.1 × B (i, j)) / 0.3 (3)
As a result, all three primary colors of the pixel at the coordinate (o, e) are calculated, so that the color demodulation of the pixel at the coordinate (o, e) is completed. The calculation results are shown in FIGS. 5 (a) and 5 (b).

  Next, in step S105, the interpolation value b (e, o) of the B signal at the coordinate (e, o) where the R OCF is formed in the color image sensor 5 is calculated. A specific calculation method is the same as in step S104, and R (i, j) is used instead of B (i, j), and the calculation is performed according to the following equation.

b (i, j) = (Y (i, j) −0.6 × g (i, j) −0.3 × R (i, j)) / 0.1 (4)
As a result, all three primary colors of the pixel at the coordinate (e, o) are calculated, so that the color demodulation of the pixel at the coordinate (e, o) is completed. The calculation results are shown in FIGS. 6 (a) and 6 (b).

  Next, in step S106, the correlation direction of the coordinates (e, e) where the G OCF is formed in the color image sensor 5 is determined using the correlation direction index D (e, e). If the correlation direction is vertical, the process proceeds to step S107a to calculate the interpolation value r (e, e) of the R signal at the coordinate (e, e). A specific calculation method is to calculate an average value of R signals R (i, j-1) and R (i, j + 1) of two pixels adjacent above and below the target pixel as an interpolation value r (e, e) of the R signal. ). This calculation formula is shown in the following formula, and the calculation results are shown in FIGS.

r (i, j) = (R (i, j-1) + R (i, j + 1)) / 2 (5)
In step S108a, the interpolation value b (e, e) of the B signal at the coordinates (e, e) is calculated. A specific calculation method is that the luminance signal Y (i, j) of the monochrome image pickup device 4 of the pixel of interest, the G signal G (i, j) of the color image pickup device 5, and the interpolation value of the R signal calculated in step S107a. Using r (i, j), the following equation is used.

b (i, j) = (Y (i, j) −0.6 × G (i, j) −0.3 × r (i, j)) / 0.1 (6)
As a result, all the three primary colors of the pixel at the coordinates (e, e) are calculated, so that the color demodulation of the pixel at the coordinates (e, e) is completed. The calculation results are shown in FIGS. 8 (a) and 8 (b).

  On the other hand, if the correlation direction is horizontal in step S106, the process proceeds to step S107b to calculate the interpolated value b (e, e) of the B signal at the coordinates (e, e), and in step S108b the R signal Interpolated value r (e, e) is calculated. The specific calculation method is the same as that in steps S107a and 108a in the case where the correlation direction is vertical. The calculation formula for the interpolation value b for the B signal and the calculation formula for the interpolation value r for the R signal are as follows. Become.

b (i, j) = (B (i-1, j) + B (i + 1, j)) / 2 (7)
r (i, j) = (Y (i, j) −0.6 × G (i, j) −0.1 × b (i, j)) / 0.3 (8)
In step S109, the correlation direction at the coordinates (o, o) where the G OCF is formed in the color image sensor 5 is determined using the correlation direction index D (o, o). If the correlation direction is vertical, the process proceeds to step S110a to calculate the interpolation value b (o, o) of the B signal at the coordinates (o, o). A specific calculation method is to calculate an average value of B signals B (i, j-1) and B (i, j + 1) of two pixels adjacent to the upper and lower sides of the target pixel as an interpolation value b (o of the B signal. , o). This calculation formula is shown in the following formula, and the calculation results are shown in FIGS.

b (i, j) = (B (i, j-1) + B (i, j + 1)) / 2 (9)
In step S111a, the interpolation value r (o, o) of the R signal at the coordinates (o, o) is calculated. A specific calculation method is that the luminance signal Y (i, j) of the monochrome image pickup device 4 of the target pixel, the G signal G (i, j) of the color image pickup device 5, and the interpolation value of the B signal calculated in step S110a. Using b (i, j), the following equation is used.

r (i, j) = (Y (i, j) −0.6 × G (i, j) −0.1 × b (i, j)) / 0.3 (10)
As a result, all the three primary colors of the pixel at the coordinates (o, o) are calculated, so that the color demodulation of the pixel at the coordinates (o, o) is completed. The calculation results are shown in FIGS. 10 (a) and 10 (b).

  On the other hand, if the correlation direction is horizontal in step S109, the process proceeds to step S110b to calculate the interpolated value r (o, o) of the R signal at the coordinates (o, o), and in step S111b the B signal Interpolated value b (o, o) is calculated. The specific calculation method is the same as that in steps S110a and 111a when the correlation direction is vertical, and the calculation formula for the interpolation value r for the R signal and the calculation formula for the interpolation value b for the B signal are as follows. Become.

r (i, j) = (R (i-1, j) + R (i + 1, j)) / 2 (11)
b (i, j) = (Y (i, j) −0.6 × G (i, j) −0.3 × r (i, j)) / 0.1 (12)
As a result, all the three primary colors of the pixel at the coordinates (o, o) are calculated, so that the color demodulation of the pixel at the coordinates (o, o) is completed.

  When the three primary color signals for all coordinates are calculated in this way, the interpolation processing by the imaging apparatus according to the present embodiment ends. As a result, according to the imaging apparatus according to the present embodiment, as shown in FIG. 10B, interpolation is also performed on a vertical stripe pattern including a high-frequency component that generates false colors in the conventional interpolation processing method. It can be seen that there is no false color in the processing result and the input image can be correctly reproduced.

  In the above-described embodiment, the correlation direction index of all coordinates is calculated in step S101. However, the timing for calculating the correlation direction index is not limited to this, and each reference to the correlation direction index is performed. You may calculate in step (S102, S103, S106, S109), respectively. In addition, after the interpolation values of the G signal are calculated for all coordinates in steps S102 and S103, the color demodulation of the coordinates (o, e) is completed in step S104, and then the color of the coordinates (e, o) in step S105. Although the demodulation is completed, the order of processing is not limited to this. First, the color demodulation of the coordinates (e, o) is completed in the order of steps S103 → S105, and then the order of steps S102 → S104. The color demodulation of the coordinates (o, e) may be completed with Alternatively, the color demodulation of the coordinates (e, e) may be completed first by processing steps S106 → S107 → S108 first, or the coordinates (o, o) Color demodulation may be completed first. In the present invention, even if the order of processing is changed as described above, the concept and the effect are not changed. Of course, the same applies even if parallel parallel processing is performed by hardware.

  In the above-described embodiment, the correlation direction index is calculated using the difference between the average value of two adjacent pixels that hold the pixel of interest and the pixel of interest as the method of calculating the correlation direction index in step S101. However, the calculation method is limited to this. However, any existing method can be used, such as calculation using a value obtained by appropriately filtering a larger number of adjacent pixels.

  Further, in the above-described embodiment, the interpolation value calculation method in the specific direction in steps S102, S103, S107, and S110 is the average value of two adjacent pixels that hold the target pixel, but the calculation method is limited to this. However, it is also possible to use a value obtained by appropriately filtering a larger number of adjacent pixels.

  The present invention is not limited to the case where the interpolation processing unit in FIG. 1 is configured by hardware, and the interpolation processing can also be performed by software by a computer program that executes the procedure in FIG. In this case, the computer program may be taken into the computer from a recording medium or may be taken into the computer via a network.

[Effect of the embodiment]
As described above in detail, according to the imaging apparatus and the interpolation processing method of the imaging apparatus according to an embodiment to which the present invention is applied, a direction having high correlation in each pixel is obtained based on the luminance signal, and this correlation is obtained. Since the interpolation value is obtained using the color signal of the pixel in the higher direction, a good interpolation image without false colors can be obtained even for an image containing a lot of high frequency components.

  Further, according to the imaging apparatus according to an embodiment to which the present invention is applied, when the target pixel is the B pixel and the R pixel, the G signal interpolation unit 21 that obtains the average value of the G signals of the correlation direction pixels as an interpolation value; When the pixel of interest is a G pixel, the R signal interpolating unit 23 and the B signal interpolating unit 22 that obtain an average value of the R signal and the B signal as an interpolation value according to the correlation direction pixel are provided. The interpolation value can be obtained using the color signal of the pixel in the higher direction.

  Furthermore, according to the imaging apparatus according to an embodiment to which the present invention is applied, the interpolation value of the R signal is calculated using the interpolation value of the G signal, the luminance signal, and the B signal, or the interpolation value of the B signal and the luminance. An R interpolation value calculation unit 31 that calculates an interpolation value of the R signal using the signal and the G signal, and an interpolation value of the B signal using the interpolation value, the luminance signal, and the R signal of the G signal, or R Since the B interpolation value calculation unit 32 that calculates the interpolation value of the B signal using the signal interpolation value, the luminance signal, and the G signal is provided, it is obtained using the color signal of the pixel in the direction with high correlation. The interpolation value can be calculated using the obtained interpolation value.

  The above-described embodiment is an example of the present invention. Therefore, the present invention is not limited to the above-described embodiment, and various forms other than this embodiment can be used depending on the design as long as they do not depart from the technical idea of the present invention. Of course, it is possible to change.

DESCRIPTION OF SYMBOLS 1 Image pick-up device 2 Lens 3 Separation optical system 4 Black and white image pick-up element 5 Color image pick-up element 6 Interpolation processing part 11 Correlation direction index | exponent calculation part 12 Signal interpolation part 13 Interpolation value calculation part 21 G signal interpolation part 22 R signal interpolation part 23 B signal interpolation Unit 31 R interpolation value calculation unit 32 B interpolation value calculation unit

Claims (4)

A first image sensor that images a subject and outputs a luminance signal;
A second image pickup device having the same number of pixels as the first image pickup device that picks up an image of a subject and outputs color signals of R, G, and B signals according to a Bayer array;
A correlation direction index calculating unit that calculates a correlation direction index representing a highly correlated direction in each pixel based on the luminance signal;
A correlation direction pixel located in a highly correlated direction is obtained based on the correlation direction index for a target pixel for which an interpolation value is to be calculated, and an interpolation value of a color signal at the target pixel is determined by a color signal of the correlation direction pixel. A signal interpolation unit for obtaining
An interpolation value calculation unit that calculates an interpolation value of a color signal that has not been obtained using the interpolation value of the color signal obtained by the signal interpolation unit, the luminance signal, and the color signal output from the second image sensor. An imaging device comprising: The signal interpolation unit
When the pixel of interest is a B pixel that outputs the B signal and an R pixel that outputs the R signal, a G signal interpolation unit that obtains an average value of the G signals of the correlation direction pixels as an interpolation value;
An R signal interpolation unit that obtains an average value of R signals of the correlation direction pixel as an interpolation value when the target pixel is a G pixel that outputs the G signal and the correlation direction pixel is the R pixel; ,
A B signal interpolation unit for obtaining an average value of B signals of the correlation direction pixel as an interpolation value when the target pixel is a G pixel that outputs the G signal and the correlation direction pixel is the B pixel; The imaging apparatus according to claim 1, further comprising: The interpolation value calculator is
An interpolation value of the R signal is calculated using the interpolation value of the G signal, the luminance signal, and the B signal in the B pixel, and the interpolation value of the B signal, the luminance signal, and the G signal in the G pixel An R interpolation value calculation unit for calculating an interpolation value of the R signal using
In the R pixel, an interpolation value of the B signal is calculated using the interpolation value of the G signal, the luminance signal, and the R signal. In the G pixel, the interpolation value of the R signal, the luminance signal, and the G signal are calculated. The imaging apparatus according to claim 2, further comprising: a B interpolation value calculation unit that calculates an interpolation value of the B signal using the B. Imaging a subject with a first image sensor and outputting a luminance signal;
Imaging a subject with a second image sensor having the same number of pixels as the first image sensor and outputting each color signal of an R signal, a G signal, and a B signal according to a Bayer array;
A correlation direction index calculating step for calculating a correlation direction index representing a highly correlated direction in each pixel based on the luminance signal;
A correlation direction pixel located in a highly correlated direction is obtained based on the correlation direction index for a target pixel for which an interpolation value is to be calculated, and an interpolation value of a color signal at the target pixel is determined by a color signal of the correlation direction pixel. A signal interpolation step to obtain
Interpolation value calculation step of calculating an interpolation value of a color signal that has not been obtained using the interpolation value of the color signal obtained in the signal interpolation step, the luminance signal, and the color signal output from the second image sensor. An interpolation processing method for an imaging apparatus, comprising:

Images