JP2014086742A - Solid-state image sensor, imaging apparatus and signal processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a highly sensitive color imaging technique suitable for generation of an IR image.SOLUTION: The solid-state image sensor includes: a photosensitive cell array in which a plurality of unit blocks of photosensitive cells are disposed two-dimensionally including a first photosensitive cell 2a disposed in first row-first column, a second photosensitive cell 2b disposed in first row-second column, a third photosensitive cell 2c disposed in second row-first column, and a fourth photosensitive cell 2d disposed in second row-second column; and a spectroscopic element array disposed opposed to the photosensitive cell array including a plurality of spectroscopic elements. The spectroscopic element array is arranged to allow light of different spectral distribution to enter each of the first to the fourth photosensitive cells 2a-2d so that total spectral distribution of visible light entering the first and the fourth photosensitive cells 2a and 2d becomes substantially equal to the total spectral distribution of visible light entering the second and third photosensitive cells 2b and 2c.

Description

  The present application relates to a technique for increasing the sensitivity and coloration of a solid-state imaging device.

  In recent years, there has been a remarkable increase in functionality and performance of digital cameras and digital movies using a solid-state image sensor such as a CCD or CMOS (hereinafter sometimes referred to as “image sensor”). In particular, due to rapid progress in semiconductor manufacturing technology, the pixel structure in an image sensor has been miniaturized. As a result, the pixels of the image sensor and the drive circuit are highly integrated, and the performance of the image sensor is increasing. In particular, in recent years, a camera using a backside illumination type image sensor that receives light on the back side rather than the surface (front surface) side on which the wiring layer of the solid-state image sensor is formed has been developed, and its characteristics are attracting attention. ing. On the other hand, with the increase in the number of pixels of the image sensor, the amount of light received by one pixel is reduced, which causes a problem that the camera sensitivity is reduced.

  The reduction in camera sensitivity is caused by the use of color filters for color separation in addition to the increase in the number of pixels. In a normal color camera, a subtractive color filter using an organic pigment as a coloring matter is arranged facing each photosensitive cell of the image sensor. Since the color filter absorbs light other than the color component to be used, when such a color filter is used, the light utilization rate of the camera is lowered. For example, in a color camera having a Bayer-type color filter array basically composed of one pixel of red (R), two pixels of green (G), and one pixel of blue (B), each color filter of R, G, B is Only the R, G, B light is transmitted and the remaining light is absorbed. Therefore, the light used in the color camera with the Bayer array is about 1/3 of the entire incident light. As described above, the use of the color filter causes a decrease in light use efficiency and causes a decrease in sensitivity of the color camera.

  On the other hand, Patent Document 1 discloses a colorization technique that increases a light utilization rate by using a spectral element that separates light according to a wavelength instead of a color filter. According to this technique, light is incident on different photosensitive cells depending on the wavelength range by the spectral elements arranged corresponding to the photosensitive cells. Each photosensitive cell receives light on which components in different wavelength ranges are superimposed from a plurality of spectral elements. As a result, a color signal can be generated by signal calculation using a photoelectric conversion signal output from each photosensitive cell.

  Although the above technique mainly uses visible light, a color image and an infrared light image (hereinafter also referred to as “IR image”) are created by using an imaging device that simultaneously receives visible light and infrared light. Imaging technology has also been devised. For example, Patent Document 2 discloses an R pixel using an R dye, a G pixel using a G dye, a B pixel using a B dye, and an infrared light receiving pixel (IR pixel) that detects only specific infrared light. An imaging technique for creating a color image and an infrared light image using an imaging device having four rows and two columns of 4 pixels as a basic configuration is disclosed.

International Publication No. 2009/153937 Japanese Patent Laid-Open No. 2005-6066 International Publication No. 2009/019818 JP S59-137909

  In the technique disclosed in Patent Document 2, since RGB organic dyes are used, there is a problem in that the sensitivity is lowered in obtaining a color image.

  On the other hand, a technique for obtaining a color image and an IR image at the same time and achieving high sensitivity in the color image is disclosed in Patent Document 1, but in this technique, it is necessary to arrange rod-shaped spectral elements in two layers. This complicates the manufacturing process.

  The present application provides a high-sensitivity color imaging technique suitable for generating an IR image with a relatively easy manufacturing process.

  A solid-state imaging device according to an aspect of the present disclosure includes a first photosensitive cell arranged in the first row and the first column, a second photosensitive cell arranged in the first row and the second column, and two rows and one column. A photosensitive cell array in which a plurality of unit blocks including a third photosensitive cell arranged in the eye and a fourth photosensitive cell arranged in the second row and the second column are two-dimensionally arranged. The photosensitive cell is a light-sensitive cell array configured to output a photoelectric conversion signal corresponding to the amount of received light, and a light-spectral element array that is disposed to face the light-sensitive cell array and includes a plurality of light-spectral elements. , Light having different spectral distributions is incident on the first to fourth photosensitive cells, and the total spectral distribution of visible light incident on the first and fourth photosensitive cells is the second and third. It is substantially equal to the total spectral distribution of visible light incident on the photosensitive cell. And a dispersing element array is configured to.

  According to the solid-state imaging device and the imaging apparatus according to the embodiments of the present disclosure, a color with relatively high light utilization efficiency is obtained by using a spectral element that causes incident light to enter different photosensitive cells according to wavelength ranges (color components). Imaging becomes possible. Furthermore, if the spectral element array is appropriately configured, an infrared light image can also be acquired.

FIG. 3 is a perspective view schematically showing an arrangement relationship between a light-sensitive cell 200 and a spectral element 100 in a solid-state imaging device according to an exemplary embodiment. (A) is a top view which shows an example of the unit block of the solid-state image sensor by exemplary embodiment, (b) is an AA 'sectional view, (c) is a BB' sectional view. It is. It is a figure which shows the example of the spectral distribution about the light of each color component of G, Cb, Yr, and M. FIG. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus according to Embodiment 1. FIG. It is a figure which shows the spectral transmittance characteristic of the optical filter in Embodiment 1. 2 is a diagram illustrating a lens and an image sensor in Embodiment 1. FIG. 2 is a diagram illustrating an example of a pixel array of an image sensor according to Embodiment 1. FIG. 6 is a diagram illustrating another example of the pixel array of the image sensor according to Embodiment 1. FIG. 1 is a plan view showing a basic structure of an image sensor in Embodiment 1. FIG. It is the sectional view on the AA 'line in Drawing 6A. FIG. 6B is a sectional view taken along line BB ′ in FIG. 6A. It is CC 'sectional view taken on the line in FIG. 6A. It is the DD 'sectional view taken on the line in FIG. 6A. (A)-(c) is process sectional drawing which shows an example of the manufacturing method of the spectroscopic element array 100. FIG. It is a perspective view which shows a part of structure where the pattern of the other low refractive index transparent layer 6a 'was formed on the 1st low refractive index transparent layer 6a. FIG. 6 is a flowchart illustrating a procedure of color information generation processing in the first embodiment. FIG. 6 is a plan view showing a basic structure of an image sensor in Embodiment 2. It is EE 'line sectional drawing in FIG. 11A. It is FF 'sectional view taken on the line in FIG. 11A. It is a GG 'line sectional view in Drawing 11A. It is HH 'sectional view taken on the line in FIG. 11A. It is a figure which shows the example of the spectral transmittance characteristic of the dichroic mirrors 3a and 3d in Embodiment 2. FIG. It is a figure which shows the example of the spectral transmittance characteristic of the dichroic mirrors 3b and 3c in Embodiment 2. FIG.

  First, the basic configuration and operation principle of the solid-state imaging device 10 according to an aspect of the present disclosure will be described with reference to FIGS. In the following description, spatial separation of light having different wavelength ranges or color components may be referred to as “spectral”. A spatial minimum unit for detecting light is referred to as a “photosensitive cell” or “pixel”. The xyz coordinates shown in the figure are used for the description. The imaging surface of the imaging device 10 is an “xy plane”, and the “x axis” is the horizontal direction on the imaging surface, the “y axis” is the vertical direction on the imaging surface, and the “z axis” is the direction perpendicular to the imaging surface. Note that “horizontal direction” and “vertical direction” mean directions on the imaging surface corresponding to the horizontal direction and vertical direction of the generated image, respectively.

  FIG. 1 is a perspective view schematically showing a part of an image sensor 10 in an exemplary embodiment. The imaging device 10 includes a photosensitive cell array 200 including a plurality of photosensitive cells arranged two-dimensionally on an imaging surface, and a spectral element array including a plurality of spectral elements. The spectral element array 100 is disposed on the light incident side facing the photosensitive cell array 200. In FIG. 1, the spectral element array 100 is represented by a quadrangular prism for the sake of simplicity, but actually does not have such a shape but has a more detailed structure.

  When each light sensing cell 2 receives light, the light sensing cell 2 outputs an electrical signal (hereinafter, also referred to as “photoelectric conversion signal” or “pixel signal”) corresponding to the intensity (light reception amount) of the light received by photoelectric conversion. To do. Each photosensitive cell 2 receives the light separated by the spectral element array 100. As a result, each photosensitive cell 2 receives light having a spectral distribution different from that received when it is assumed that there is no spectral element. Here, the “spectral distribution” means a distribution of intensity with respect to the wavelength of the light.

  Hereinafter, an example of the basic structure of the image sensor 10 will be described with reference to FIG. Here, first, it is assumed that only visible light is incident on the image sensor 10.

  FIG. 2A is a plan view showing one of the basic pixel configurations (unit blocks) 40 of the photosensitive cell array 200. The photosensitive cell array 200 has a structure in which a plurality of unit blocks 40 each including four photosensitive cells 2a, 2b, 2c, and 2d are two-dimensionally arranged on the imaging surface. In one unit block, four photosensitive cells are arranged in 2 rows and 2 columns. In each unit block, the first photosensitive cell 2a in the first row and the first column, the second photosensitive cell 2b in the first row and the second column, the third photosensitive cell 2c in the second row and the first column, A fourth photosensitive cell 2d is arranged in the second row and the second column.

  2B and 2C are diagrams schematically showing a cross section taken along the line AA ′ and a cross section taken along the line BB ′ in FIG. 2B and 2C, the traveling direction changes depending on the color component when light incident on the image sensor 10 passes through the spectral element array 100, and as a result, the spectral distribution of the light received by each photosensitive cell is shown. It shows how they are different from each other. Here, when it is assumed that the spectral element array 100 does not exist, the light received by each photosensitive cell is referred to as “cell incident light” of the photosensitive cell. When the photosensitive cells 2a to 2d included in one unit block are close to each other, it can be considered that the light intensity and spectral distribution included in the cell incident light of the photosensitive cells are substantially the same. The intensity of the visible light component of the cell incident light of these photosensitive cells is represented by the symbol “W”. In this specification, a part of color components of visible light included in the cell incident light is set as a first color component, and the other part of color components is set as a second color component. The complementary color component of the second color component is the third color component, and the complementary color component of the first color component is the fourth color component. That is, when the intensities of the first to fourth color components are represented by C1, C2, C3, and C4, respectively, they can be represented as W = C1 + C4 and W = C2 + C3. Note that the wavelength range of the light of the first color component and the wavelength range of the light of the fourth color component are completely different because the first color component and the fourth color component have a complementary color relationship. There is no need to separate them, and they may partially overlap. Similarly, the wavelength range of the light of the second color component and the wavelength range of the light of the third color component may partially overlap.

  The first to fourth color components are, for example, green (G), blue that includes half of green (B + 1 / 2G, hereinafter may be referred to as “Cb”), and red that includes half of green (for example, R + 1 / 2G, hereinafter referred to as “Yr”), and magenta (R + B, hereinafter referred to as “M”). The first to fourth color components are not limited to this example, and may be a combination of other color components as long as the four color components are included in visible light.

  FIG. 3 shows an example of the spectral distribution of light of each color component when the first to fourth color components are G, Cb, Yr, and M, respectively. As shown in the figure, the G light is a color component mainly containing a component in the green wavelength range (around 500 nm to 600 nm), but may contain some components in other wavelength ranges. The M light is a color component mainly containing components in the blue wavelength range (near 400 nm to 500 nm) and the red wavelength range (near 600 nm to 700 nm), but may contain some components in the green wavelength range. . The Cb light is a color component mainly containing a blue to short wavelength side green component (near 400 nm to 550 nm), but may contain some other wavelength range components. Yr light is a color component mainly containing components of green to red (near 550 nm to 700 nm) on the long wavelength side, but may contain some components in other wavelength regions. As described above, the light of the first to fourth color components may include light in a wavelength region that partially overlaps, although the wavelength regions where the intensity peaks are different from each other.

  The spectroscopic element array 100 typically includes a plurality of spectroscopic elements arranged to face each photosensitive cell. Each spectroscopic element can be constituted by, for example, a “high refractive index transparent portion” described in detail later and a “low refractive index transparent portion” provided therearound. Alternatively, it may be constituted by a microlens whose shape and refractive index are appropriately designed. Further, it can be realized by using a multilayer filter (dichroic mirror) that reflects light in a specific wavelength range and transmits light in other wavelength ranges. The spectroscopic element including the multilayer filter may be configured to totally reflect light reflected by the multilayer filter at a boundary between different media and guide the light to an adjacent photosensitive cell.

  The spectral element array 100 shown in FIG. 2 reduces the light of the first color component (intensity C1) from the cell incident light (intensity W) of the first photosensitive cell 2a, and the light of the second color component and the first light. The light whose intensity of the three color components (intensity (C2 + C3) / 2) is increased is incident on the first photosensitive cell 2a. Further, the light of the second color component (intensity C2) is decreased from the cell incident light (intensity W) of the second photosensitive cell 2b, and the light of the first color component and the light of the fourth color component (intensity). The light whose (C1 + C4) / 2) is increased is incident on the second photosensitive cell 2b. Further, the light of the third color component (intensity C3) is decreased from the cell incident light (intensity W) of the third photosensitive cell 2c, and the light of the first color component and the fourth color component (intensity (C1 + C4)). ) / 2), the light having increased light is made incident on the third photosensitive cell 2c. Further, the light of the fourth color component (intensity C4) is reduced from the cell incident light (intensity W) of the fourth photosensitive cell 2d, and the light of the second color component and the light of the third color component (intensity). The light whose (C2 + C3) / 2) is increased is incident on the fourth photosensitive cell 2d.

With the above-described configuration, the photosensitive cells 2a to 2d are, as shown in FIGS. 2B and 2C, W-C1 + (C2 + C3) / 2, W-C2 + (C1 + C4) / 2, and W-C3 +, respectively. The light of the intensity | strength represented by (C1 + C4) / 2 and W-C4 + (C2 + C3) / 2 is received. Each photosensitive cell outputs a photoelectric conversion signal corresponding to these intensities. Here, the photoelectric conversion signals output from the photosensitive cells 2a to 2d are S2a to S2d, respectively, the signal corresponding to the intensity W is Ws, the signal corresponding to the intensity C1 is C1s, the signal corresponding to the intensity C2 is C2s, and the intensity A signal corresponding to C3 is defined as C3s, and a signal corresponding to the intensity C4 is defined as C4s. However, Ws = C1s + C4s and Ws = C2s + C3s. Then, S2a-S2d can be represented by the following formulas 1-4.
(Formula 1) S2a = Ws-C1s + (C2s + C3s) / 2
(Formula 2) S2b = Ws-C2s + (C1s + C4s) / 2
(Formula 3) S2c = Ws-C3s + (C1s + C4s) / 2
(Formula 4) S2d = Ws−C4s + (C2s + C3s) / 2

Here, when the difference between the signals S2a and S2d is D1, and the difference between the signals S2b and S2c is D2, D1 and D2 are expressed by the following equations 5 and 6, respectively.
(Formula 5) D1 = S2a-S2d = C4s-C1s
(Formula 6) D2 = S2c-S2b = C2s-C3s

Furthermore, the following formulas 7 and 8 are obtained by adding and subtracting D1 and D2, respectively.
(Formula 7) D1 + D2 = C4s + C2s-C3s-C1s
(Formula 8) D1-D2 = C4s-C2s + C3s-C1s

  Since the calculations of Expressions 7 and 8 include difference calculations, it can be said that the signal (D1 + D2) and the signal (D1-D2) generated by these calculations indicate color difference signals. Here, red, green, and blue photoelectric conversion signals are represented by Rs, Gs, and Bs, respectively. As typical examples described above, C1s = Gs, C2s = Bs + 1 / 2Gs, C3s = Rs + 1 / 2Gs, and C4s = Rs + Bs. Assuming that there is, it can be seen from Equation 7 and Equation 8 that two color difference signals represented by (D1 + D2) = (2Rs−Gs) and (D1−D2) = (2Bs−Gs) are obtained.

On the other hand, by any one of the addition of S2a and S2d, the addition of S2b and S2c, and the addition of S2a to S2d, as shown in the following formulas 9 to 11, the intensity W of the visible light component of the cell incident light A signal corresponding to twice or four times the above is obtained.
(Formula 9) S2a + S2d = 2Ws
(Formula 10) S2b + S2c = 2Ws
(Formula 11) S2a + S2b + S2c + S2d = 4 Ws

  These signals correspond to signals obtained when all visible light components included in the incident light are photoelectrically converted without loss. Therefore, if any of these is used as a luminance signal, ideal characteristics can be obtained in terms of imaging sensitivity.

  If the luminance signal obtained by any one of Equations 9 to 11 and the two color difference signals obtained by Equations 7 and 8 are obtained, the RGB signal can be obtained by matrix operation. That is, a color signal can be calculated by signal calculation based on the four photoelectric conversion signals S2a to S2d output from the photosensitive cells 2a to 2d.

  According to the imaging device 10 described above, color information can be obtained by signal calculation by using a spectral element without using a color filter that absorbs part of light. Therefore, loss of light can be suppressed, and imaging sensitivity can be increased as compared with the conventional case. Further, as shown on the right side of Expressions 1 to 4, since any signal of C1s to C4s is not only added but also subtracted in the process of generating each pixel signal, the visible light having the light intensity W is received. In this case, the signal value does not particularly increase or decrease particularly for a certain pixel. This can be said to be favorable in terms of improving the dynamic range characteristics of the image sensor 10 with respect to incident light.

  Next, a case where the image sensor 10 receives only infrared light in a specific wavelength range will be described. This corresponds to a case where imaging is performed using an IR light source that emits infrared light in a specific wavelength range at night, for example. Due to the characteristics of the spectral element array 10 including spectral elements and other optical elements, the imaging element 10 has the same amount of received light in the first photosensitive cell 2a and the fourth photosensitive cell 2d for the specific infrared light. The second photosensitive cell 2b and the third photosensitive cell 2c are configured to have the same received light amount, and the first photosensitive cell 2a and the second photosensitive cell 2b have different received light amounts. That is, the spectral element array 100 is configured such that S2a = S2d ≠ S2b = S2c is established when only infrared light in the specific wavelength region enters each photosensitive cell. Therefore, D1 and D2 in Expressions 5 and 6 are 0, and as a result, the signals (D1 + D2) and (D1-D2) represented by Expressions 7 and 8 are also 0. This result shows that the influence of infrared light in the wavelength region does not appear in the color difference signals represented by Expression 7 and Expression 8. On the other hand, the influence of infrared light in the specific wavelength region appears in the luminance signals represented by Expressions 9 to 11. That is, the amount of received infrared light in the specific wavelength region differs between the first and fourth photosensitive cells 2a and 2d and the second and third photosensitive cells 2b and 2c. And a difference between the luminance signal expressed by Equation 10 and the luminance signal expressed by Equation 10.

  Here, an operation | S2a + S2d−S2b−S2c | is considered. As can be seen from Equations 9 and 10, when the imaging device 10 receives only visible light, the calculation result is 0. However, when infrared light in a specific wavelength region is received, the calculation result is 0. Don't be. This indicates that the amount of infrared light in the specific wavelength region can be estimated by the calculation. If the amount of infrared light in the specific wavelength region can be estimated, the influence of the infrared light in the specific wavelength region on the luminance signal can be removed. In addition, an IR image can be generated using the result of the operation | S2a + S2d-S2b-S2c |. That is, a visible light color image and an IR image can be obtained simultaneously. Specific processing for generating a color image and an IR image will be described later.

  As described above, the image sensor 10 according to the embodiment of the present disclosure has a configuration suitable for generating an IR image. By appropriately configuring the spectral elements and other optical elements, color imaging and IR images can be generated simultaneously. Further, unlike the prior art, since a color filter is not used, there is an excellent advantage that color imaging with high sensitivity is possible.

  The configuration of the spectral element array 100 is not limited to the above example. The spectral element array 100 reduces the C1 light from the cell incident light of the first light-sensitive cell 2a, and causes the light having the increased C2 light and C3 light to be incident on the first light-sensitive cell 2a. The C2 light is decreased from the cell incident light of the sensing cell 2b, the light obtained by increasing the C1 light and the C4 light is incident on the second photosensitive cell 2b, and the C3 light is emitted from the cell incident light of the third photosensitive cell 2c. , The light having the increased C1 light and the C4 light is made incident on the third photosensitive cell 2c, the C4 light is decreased from the cell incident light of the fourth photosensitive cell 2d, and the C2 light and the C3 light are reduced. As long as the increased light is configured to enter the fourth photosensitive cell 2d, the intensity of the light may be different from that in the above example. By appropriately correcting each signal according to the ratio of each color component in each photoelectric conversion signal, the same arithmetic processing as described above can be applied.

  Further, the spectroscopy by the spectral element array 100 is not limited to the above-described aspect. The spectral element array 100 allows light having different spectral distributions to enter the first to fourth photosensitive cells, and the total spectral distribution of visible light incident on the first and fourth photosensitive cells is the second and second. As long as it is configured to be substantially equal to the total spectral distribution of the visible light incident on the three photosensitive cells, any configuration may be used. In other words, the sum of the visible light component of the photoelectric conversion signal output from the first photosensitive cell and the visible light component of the photoelectric conversion signal output from the fourth photosensitive cell is the second value. The visible light component of the photoelectric conversion signal output from the photosensitive cell and the visible light component of the photoelectric conversion signal output from the third photosensitive cell are substantially equal to each other. It only has to be. With such a configuration, the visible light component can be canceled by the calculation | S2a + S2d−S2b−S2c |, and thus there is an advantage that infrared light information can be easily generated.

  In the embodiment of the present disclosure, it is effective to arrange an infrared cut filter that transmits only specific infrared light on the front surface of the image sensor 10. Further, with respect to infrared light in a specific wavelength range, the received light amounts of the first photosensitive cell 2a and the fourth photosensitive cell 2d are made equal, and the second photosensitive cell 2b and the third photosensitive cell 2c It is effective to equalize the amount of received light. Therefore, an optical filter having transparency in the wavelength band of the infrared light and visible light and not having transparency in the other wavelength bands may be disposed on any of the photosensitive cells.

  Hereinafter, more specific embodiments of the present disclosure will be described. In the following description, the same or similar elements are denoted by the same reference numerals.

(Embodiment 1)
FIG. 4 is a block diagram illustrating the overall configuration of the imaging apparatus according to the first embodiment. The imaging apparatus according to the present embodiment is a digital electronic camera, and includes an imaging unit 300 and a signal processing unit 400 that generates a signal (image signal) indicating an image based on a signal transmitted from the imaging unit 300. ing. Note that the imaging device may generate only a still image or may have a function of generating a moving image.

  The imaging unit 300 is an optical lens 12 for imaging a subject, an optical filter 11, and a solid-state imaging device 10 (converting optical information imaged through the optical lens 12 and the optical filter 11 into an electrical signal by photoelectric conversion ( Image sensor). The imaging unit 300 further generates an IR light source 19 that emits infrared light and a signal that generates a basic signal for driving the imaging device 10 and receives an output signal from the imaging device 10 and sends it to the signal processing unit 400. A generation / reception unit 13 and an element drive unit 14 that drives the image sensor 10 based on a basic signal generated by the signal generation / reception unit 13 are provided. The optical lens 12 is a known lens and may be a lens unit having a plurality of lenses. The image sensor 10 is typically a CMOS or a CCD, and is manufactured by a known semiconductor manufacturing technique. The signal generation / reception unit 13 and the element driving unit 14 are configured by an LSI such as a CCD driver, for example. The IR light source 19 is a general-purpose infrared light source, and is used for imaging in a dark situation such as at night.

  The optical filter 11 is a combination of a quartz low-pass filter for reducing moire patterns generated due to pixel arrangement and an infrared cut filter that removes most of infrared light. The infrared cut filter has a characteristic of transmitting only visible light and infrared light in a specific wavelength range and absorbing infrared light in other wavelength ranges. FIG. 5 shows an example of the spectral transmittance characteristics of this infrared cut filter. In this example, the specific wavelength range is set between 700 and 800 nm. Therefore, the infrared light transmitted through the infrared cut filter has an intensity peak between 700 and 800 nm. However, this is only an example, and the wavelength at which the intensity peaks can be set arbitrarily. Due to this spectral characteristic, only specific infrared light having a peak in the near-infrared band is incident on the image sensor 10 in addition to visible light.

  The signal processing unit 400 generates an image signal by processing a signal transmitted from the imaging unit 300, a memory 30 for storing various data generated in the process of generating the image signal, and a generated signal And an image signal output unit 16 for sending the image signal to the outside. The image signal generation unit 15 can be suitably realized by a combination of hardware such as a known digital signal processor (DSP) and software that executes image processing including image signal generation processing. The memory 30 is configured by a DRAM or the like. The memory 30 records the signal transmitted from the imaging unit 300 and temporarily records the image data generated by the image signal generation unit 15 and the compressed image data. These image data are sent to a recording medium (not shown) or a display unit via the image signal output unit 16.

  Note that the imaging apparatus according to the present embodiment may include known components such as an electronic shutter, a viewfinder, a power source (battery), and a flashlight, but a description thereof is omitted because it is not particularly necessary for understanding the present embodiment. . Further, the above configuration is merely an example, and in the present embodiment, publicly known elements can be appropriately combined and used for the constituent elements other than the image sensor 10 and the image signal generation unit 15.

  Hereinafter, the solid-state imaging device 10 in the present embodiment will be described.

  FIG. 6 is a diagram schematically illustrating a state in which light transmitted through the lens 12 is incident on the image sensor 10 during exposure. In FIG. 6, description of components other than the lens 12 and the image sensor 10 is omitted for simplicity. In addition, the lens 12 can be generally composed of a plurality of lenses arranged in the optical axis direction, but is drawn as a single lens for simplicity. A photosensitive cell array including a plurality of photosensitive cells (pixels) arranged two-dimensionally is disposed on the imaging surface 10a of the imaging element 10. Each photosensitive cell is typically a photodiode, and outputs a photoelectric conversion signal (pixel signal) corresponding to the amount of incident light by photoelectric conversion. Light (visible light and infrared light in a specific wavelength range) that has passed through the lens 12 and the optical filter 11 is incident on the imaging surface 10a. In general, the intensity of light incident on the imaging surface 10a and the distribution (spectral distribution) of the amount of incident light for each wavelength range differ depending on the incident position.

  7A and 7B are plan views illustrating an example of a pixel array in the present embodiment. For example, the photosensitive cell array 200 includes a plurality of photosensitive cells arranged in a square lattice pattern on the imaging surface 10a as shown in FIG. 7A. The photosensitive cell array 200 includes a plurality of unit blocks 40, and each unit block 40 includes four photosensitive cells 2a, 2b, 2c, and 2d. In each unit block, the first photosensitive cell 2a in the first row and the first column, the second photosensitive cell 2b in the first row and the second column, the third photosensitive cell 2c in the second row and the first column, A fourth photosensitive cell 2d is arranged in the second row and the second column. Note that the arrangement of the photosensitive cells is not such a square lattice arrangement, but may be, for example, an oblique arrangement shown in FIG. 7B or another arrangement. As shown in FIGS. 7A and 7B, the four photosensitive cells 2a to 2d included in each unit block are close to each other, but even if they are separated from each other, the spectral element array described later should be appropriately configured. It is possible to obtain color information.

  Opposite to the photosensitive cell array 200, a spectral element array including a plurality of spectral elements is arranged on the light incident side. In this embodiment, one spectral element is provided for each of the four photosensitive cells included in each unit block.

  Hereinafter, the spectral elements in the present embodiment will be described.

  The spectral element in the present embodiment is an optical element that directs incident light in different directions depending on the wavelength region by utilizing diffraction of light generated at the boundary between two types of translucent members having different refractive indexes. This type of spectroscopic element consists of a high refractive index transparent part (core part) made of a material having a relatively high refractive index and a low contact with each side face of the core part made of a material having a relatively low refractive index. It has a refractive index transparent part (cladding part). Due to the difference in refractive index between the core part and the clad part, a phase difference occurs between the light transmitted through the core part and diffraction occurs. Since this phase difference varies depending on the wavelength of light, it becomes possible to spatially separate light according to the wavelength range (color component). For example, the light of the first color component can be directed in half in the first direction and the second direction, and the light other than the first color component can be directed in the third direction. It is also possible to direct light in different wavelength ranges (color components) in the three directions.

  In the present specification, the terms “high refractive index” and “low refractive index” do not mean absolute high or low refractive index, but merely a result of relative comparison of refractive indexes. That is, “low refractive index” means that the refractive index of the low refractive index transparent portion is lower than the refractive index of the high refractive index transparent portion. On the other hand, “high refractive index” means that the refractive index of the high refractive index transparent portion is higher than the refractive index of the low refractive index transparent portion. Therefore, if the refractive index of the low refractive index transparent part is lower than the refractive index of the high refractive index transparent part, the value of each refractive index is arbitrary.

  The low refractive index transparent portion in the present embodiment is formed in layers within the spectral element array 100. For this reason, the low refractive index transparent portion is also referred to as “low refractive index transparent layer” or simply “transparent layer”. The individual high refractive index transparent portions embedded in the low refractive index transparent layer locally reduce the phase velocity of light incident on the low refractive index transparent portion. As a result, when the light incident on the upper surface of the low refractive index transparent portion propagates toward the lower surface, a phase shift that varies depending on the wavelength occurs, and the incident light is dispersed. For this reason, in this specification, each high refractive index transparent part is called a "spectral element." This spectral element may be called a “phase shifter”.

  The shape of the high refractive index transparent portion in each spectral element is a rectangular parallelepiped, the sides in the x-axis direction and the y-axis direction are approximately the same, and the side in the z-axis direction is the longest. The shape of the high refractive index transparent portion 3 does not have to be a strict rectangular parallelepiped, the edge may be rounded, and the side surface may have a taper or a reverse taper. By adjusting the size, shape, refractive index, and the like of the high refractive index transparent portion 3, it is possible to control how incident light is dispersed. Since the more detailed structure and function of the high refractive index transparent part 3 are disclosed in Patent Document 3, the entire contents of Patent Document 3 are incorporated herein.

  FIG. 8A is a plan view showing the basic structure of the image sensor 10. In each unit block, the spectroscopic elements 1a, 1b, 1c, and 1d are disposed to face the four photosensitive cells 2a, 2b, 2c, and 2d, respectively. A plurality of patterns having such a basic structure are repeatedly formed on the imaging surface 10a.

  8B, 8C, 8D, and 8E are diagrams respectively showing an AA ′ line cross section, a BB ′ line cross section, a CC ′ line cross section, and a DD ′ line cross section in FIG. 8A. As shown in these drawings, the image pickup device 10 includes a semiconductor substrate 7 made of a material such as silicon, photosensitive cells 2a to 2d disposed inside the semiconductor substrate 7, and a surface side (light) of the semiconductor substrate 7. Are formed on the wiring layer 5 and the transparent layer 6 made of a low refractive index transparent portion, and the spectral elements 1a, 1b, 1c, 1d made of the high refractive index transparent portion arranged inside the transparent layer 6. And. In addition, microlenses 4 for efficiently condensing light to each photosensitive cell are arranged corresponding to the individual photosensitive cells with a transparent layer 6 therebetween. Even if the microlens 4 is not arranged, it is possible to obtain the effect of this embodiment. In the present embodiment, red that transmits visible light and infrared light in a specific wavelength region and absorbs other infrared light facing the second photosensitive cell 2b and the fourth photosensitive cell 2d. Outer cut filters 8b and 8d are respectively arranged. These infrared cut filters 8b and 8d are configured so that the received light amounts of the photosensitive cells 2a and 2d are equalized and the received light amounts of the photosensitive cells 2b and 2c are equalized with respect to infrared light in a specific wavelength range. The transmittance is designed.

  The structure shown in FIGS. 8A to 8E can be manufactured by a known semiconductor manufacturing technique. 8A to 8E has a surface irradiation type structure in which light enters each photosensitive cell from the wiring layer 5 side, but is not limited to such a structure, and the wiring layer 5 You may have the structure of the back irradiation type which receives light from the other side.

  As shown in FIGS. 8A to 8E, the spectral elements 1a, 1b, 1c, and 1d have a rectangular cross section and a rectangular parallelepiped shape that is long in the light transmitting direction. Spectroscopy by refractive index difference. Here, first, it is assumed that only visible light is received. In this case, it may be considered that the infrared cut filters 8b and 8d shown in FIGS. 8B, 8C, and 8E do not exist.

  The light separating element 1a makes red (R) light and blue (B) light combined light, that is, magenta (M) light, enter the opposing photosensitive cell 2a. In addition, green (G) light is incident on four photosensitive cells adjacent in the horizontal direction (x direction) or the vertical direction (y direction) by ¼. That is, the spectroscopic element 1a causes the light represented by G / 4 to enter the photosensitive cells 2b and 2c and the two photosensitive cells included in the adjacent unit block.

  The light-splitting element 1b makes synthetic light (Cb light) composed of 1/2 of the B light and the G light incident on the opposing photosensitive cell 2b. Further, the combined light (Yr light) made up of ½ of the R light and the G light is made incident on four photosensitive cells adjacent in the horizontal direction and the vertical direction by 1/4. That is, the spectroscopic element 1b causes the light represented by Yr / 2 to enter the photosensitive cells 2a and 2d and the two photosensitive cells included in the adjacent unit block.

  The light-splitting element 1c causes Yr light to enter the opposing photosensitive cell 2c, and causes Cb light to enter each of four adjacent photosensitive cells in the horizontal direction and the vertical direction by ¼. That is, the spectroscopic element 1c causes light represented by Cb / 4 to enter the photosensitive cells 2a and 2d and the two photosensitive cells (not shown) included in the adjacent unit block.

  The light-splitting element 1d causes the G light to be incident on the opposing photosensitive cell 2d, and allows the M light to be incident on four photosensitive cells adjacent in the horizontal direction and the vertical direction one by one. That is, the spectroscopic element 1 causes light represented by M / 4 to enter the photosensitive cells 2b and 2c and the two photosensitive cells (not shown) included in the adjacent unit block. In the present embodiment, the length and thickness of the spectral elements 1a, 1b, 1c, and 1d are designed so as to have the above spectral characteristics.

The spectroscopic elements 1a to 1d can be made of, for example, silicon nitride (SiN). The low refractive index transparent layer 6 can be formed of, for example, silicon dioxide (SiO ₂ ). However, the material of the spectral element array 100 is not limited to this example. In order to correct the spectral characteristics, a substance that absorbs light in a specific wavelength region may be added to a part of the high refractive index transparent portion or the low refractive index transparent layer 6 constituting the spectral elements 1a to 1d.

  Next, an example of a method for manufacturing the spectral element array 100 used in the present embodiment will be described with reference to FIG.

  First, as shown in FIG. 9A, a first low refractive index transparent layer 6 a constituting a part of the low refractive index transparent layer 6 is deposited on the photosensitive cell array 200. The first low refractive index transparent layer 6a can be deposited using a known thin film deposition technique. For example, CVD (chemical vapor deposition) or sputtering can be used. A transparent layer 18 made of a material having a refractive index higher than that of the low refractive index transparent layer 6 is deposited on the first low refractive index transparent layer 6a. The transparent layer 18 can also be deposited using a known thin film deposition technique. Next, an etching mask pattern 17 is formed on the transparent layer 18 by lithography. By designing a photomask pattern used in the lithography technique, the etching mask pattern 17 having an arbitrary planar shape can be formed.

  Next, as shown in FIG. 9B, by etching the transparent layer 18 using the etching mask pattern 17 as a mask, unnecessary portions are removed from the transparent layer 18 and a high refractive index transparent portion (in the example shown in the figure). “Spectral elements 1a, 1b”). This etching can be performed by anisotropic dry etching. A taper of the high refractive index transparent portion may be formed during etching.

  Further, as shown in FIG. 9C, after the etching mask pattern 17 is removed, the region between the high refractive index transparent portions is buried with the second low refractive index transparent layer 6b constituting the low refractive index transparent layer 6. Then, the formation of the low refractive index transparent layer 6 is completed. The second low refractive index transparent layer 6b can be formed so as to cover the upper surface of the high refractive index transparent portion. In FIG. 9, only the spectral elements 1a and 1b are depicted, but the spectral elements 1c and 1d are also formed simultaneously by the above method.

  By adjusting the thickness of the first low refractive index transparent layer 6a, the distance between the lower end of the high refractive index transparent portion and the photosensitive cell array 200 can be controlled. Between the spectral elements 1a to 1d having different spectral characteristics, the distance between the lower end of the high refractive index transparent portion and the photosensitive cell array 200 may be different. For this reason, when manufacturing the spectral element array 100 in this embodiment, the process of changing the thickness of the 1st low refractive index transparent layer 6a according to a position may be performed. Such a process is performed by etching a part of the first low-refractive-index transparent layer 6a from the surface before depositing the transparent layer 18, or by another layer on the first low-refractive-index transparent layer 6a. What is necessary is just to form the pattern of a low refractive index transparent layer. The pattern of another low refractive index transparent layer can be formed by, for example, a lift-off method.

  FIG. 10 is a perspective view showing a part of a structure in which a pattern of another low refractive index transparent layer 6a 'is formed on the first low refractive index transparent layer 6a. The pattern of the other low refractive index transparent layer 6a 'may be formed with a different thickness depending on which of the photosensitive cells 2a to 2d is opposed to the pattern.

  Further, the spectral elements 1a to 1d having different spectral characteristics may have different heights (sizes in the z-axis direction). In order to change the height of the light-splitting elements 1a to 1d depending on the position, the transparent layer 18 having a different thickness depending on the location may be deposited. Such a transparent layer 18 is formed by depositing a transparent layer 18 having a substantially uniform thickness on the first low-refractive-index transparent layer 6a having a concavo-convex step as shown in FIG. It can be formed by planarizing the top surface of layer 18. When such flattening is performed, the upper surfaces of the high refractive index transparent portions constituting the spectral element array 100 are all at the same level. When changing the level of the upper surface of the high refractive index transparent part according to the type of spectral element, the upper surface of the high refractive index transparent part located in a specific region is masked, and the upper surface of the unmasked high refractive index transparent part is masked. Selective etching may be performed.

  In addition, the manufacturing method of said spectral element array is an example, and is not limited to such a method.

Next, signal processing by the image signal generation unit 15 will be described. As a result of the spectroscopy by the spectral elements 1a to 1d, the photosensitive cells 2a to 2d output photoelectric conversion signals S2a to S2d represented by the following formulas 12 to 15, respectively. Here, signals corresponding to the intensities of red light, green light, and blue light are represented as Rs, Gs, and Bs, respectively. Further, a signal corresponding to the intensity of magenta light is Ms (= Rs + Bs), a signal corresponding to the intensity of Yr light is Yrs (= Rs + 1 / 2Gs), and a signal corresponding to the intensity of Cb light is Cbs (= Bs + 1 / 2Gs). A signal corresponding to the intensity of the cell incident light is expressed as Ws (= Rs + Gs + Bs = Ms + Gs = Yrs + Cbs).
(Formula 12) S2a = Ms + (Yrs + Cbs) / 2 = (3/2) Rs + (1/2) Gs + (3/2) Bs
(Formula 13) S2b = Cbs + (Ms + Gs) / 2 = (1/2) Rs + Gs + (3/2) Bs
(Formula 14) S2c = Yrs + (Ms + Gs) / 2 = (3/2) Rs + Gs + (1/2) Bs
(Formula 15) S2d = Gs + (Yrs + Cbs) / 2 = (1/2) Rs + (3/2) Gs + (1/2) Bs

  Equations 12 to 15 correspond to those obtained by substituting C1s for Gs, C2s for Yrs, C3s for Cbs, and C4s for Ms in Equations 1-4, respectively.

  Next, a case where the image sensor 10 receives only infrared light in a specific wavelength range will be described. This corresponds to a case where, for example, at night, an infrared light image is acquired by irradiating a subject with infrared light using the IR light source 19 and detecting the reflected light.

  The spectral elements 1a to 1d in the present embodiment are configured such that part of the infrared light in the specific wavelength region is incident on the light-sensitive cell immediately below and the other part is incident on the adjacent light-sensitive cell. ing. More specifically, if the spectral elements 1a to 1d do not have the infrared cut filters 8b and 8d, the light reception amount Ir11 of the photosensitive cell 2a and the light reception of the photosensitive cell 2b for infrared light in the specific wavelength range. The amount Ir12, the amount of received light Ir21 of the photosensitive cell 2c, and the amount of received light Ir22 of the photosensitive cell 2d are different from each other, and the relationship of Ir11 <Ir22 and Ir21 <Ir12 is established. Therefore, in the present embodiment, with respect to the infrared light, the light receiving amounts of the light sensing cells 2a and 2d are equal, and the light receiving amounts of the light sensing cells 2b and 2c are equal. Infrared cut filters 8b and 8d are provided. When the transmittances of the infrared cut filters 8b and 8d for the infrared light are kb and kd, respectively, the amount of received light of the infrared light of the photosensitive cell 2b (hereinafter referred to as “infrared light reception amount”). Kb × Ir12, and the amount of infrared light received by the photosensitive cell 2d is kd × Ir22. Therefore, the infrared light reception amount Ir11 of the photosensitive cell 2a and the infrared light reception amount kd × Ir22 of the light detection cell 2d are equal, and the infrared light reception amount Ir21 of the light detection cell 2c and the infrared light reception amount kb of the light detection cell 2b. × Ir12 becomes equal.

  With the above configuration, when the imaging device 10 receives only infrared light in the specific wavelength range, photoelectric conversion signals corresponding to the infrared light reception amounts Ir11, Ir12, Ir21, and Ir22 are Ir11s, Ir12s, Ir21s, and Ir22s, respectively. In other words, a signal expressed by Ir11s from the photosensitive cell 2a, kb × Ir12s from the photosensitive cell 2b, Ir21s from the photosensitive cell 2c, and kd × Ir22s from the photosensitive cell 2d is obtained. However, Ir11s = kb × Ir22s and Ir21s = kb × Ir12s. That is, in this case, S2a = Ir11s, S2b = kb × Ir12s = Ir21s, S2c = Ir21s, S2d = kd × Ir22s = Ir11s.

  Here, the ratio of the amount of infrared light in the specific wavelength range incident on the photosensitive cells 2a to 2d via the spectral elements 1a to 1d and the infrared cut filters 8b and 8d can be obtained at the design stage. That is, the ratio of Ir11s, Kb × Ir12s, Ir21s, kd × Ir22s is measured by irradiating the imaging element 10 with infrared light in the specific wavelength range at the time of design and measuring the signal amounts of the four photosensitive cells 2a to 2d. Can be requested. Further, since the transmittances kb and kd for the infrared light of the infrared cut filters 8b and 8d are also known, the ratio of Ir11s, Ir12s, Ir21s, and Ir22s can also be obtained. For this reason, the ratio between the calculated value of the above calculation | S2a + S2d−S2b−S2c | and the average value (S2a + S2d + S2b + S2c) / 4 of the four pixel signals when the infrared cut filters 8b and 8d are not present is also optically designed. The above can be calculated. Therefore, in this embodiment, in order to generate an IR image, the ratio kv (= (Ir11s + Ir12s + Ir21s + Ir22s) / 4 | Ir11s + kd × Ir22s−Ir21s−kb × Ir12s |) is calculated and recorded in the memory 30 or the like. Yes. Since Ir11s = kd × Ir22s and Ir21s = kb × Ir12s, this ratio kv can also be calculated by (Ir11s + Ir12s + Ir21s + Ir22s) / 8 | Ir11s−Ir21s |.

  In the present embodiment, both an infrared light image and a visible light image (color image) can be generated even when visible light is incident on the image sensor 10 in addition to infrared light in a specific wavelength range. In order to generate a color image in this case, kw = (Ir11s + kb × Ir12s + Ir21s + kd × Ir22s) / | Ir11s + kd × Ir22s−Ir21s−kb × Ir12s | is also obtained in the design stage and recorded in the memory 30 or the like.

When receiving visible light and specific infrared light, photoelectric conversion signals of the photosensitive cells 2a to 2d are expressed by the following equations 16 to 19, respectively.
(Formula 16) S2a = (3/2) Rs + (1/2) Gs + (3/2) Bs + Ir11s
(Expression 17) S2b = (1/2) Rs + Gs + (3/2) Bs + kb × Ir12s
(Formula 18) S2c = (3/2) Rs + Gs + (1/2) Bs + Ir21s
(Formula 19) S2d = (1/2) Rs + (3/2) Gs + (1/2) Bs + kd × Ir22s

  In this case, the image signal generation unit 15 illustrated in FIG. 4 can generate color information by calculation using the photoelectric conversion signals represented by Expressions 16 to 19.

  Hereinafter, color information generation processing by the image signal generation unit 15 will be described with reference to FIG. FIG. 11 is a flowchart showing the procedure of color information generation processing in the present embodiment.

The image signal generation unit 15 holds the values of the ratios kv and kw obtained in advance at the time of design. First, in step S11, the image signal generation unit 15 acquires photoelectric conversion signals S2a to S2d. Subsequently, in step S12, the image signal generation unit 15 generates a signal (Rs−Gs + Bs) and a signal (Rs−Bs) by executing calculations represented by the following Expressions 20 and 21.
(Formula 20) S2a-S2d = Rs-Gs + Bs
(Formula 21) S2c-S2b = Rs-Bs
By these calculations, the infrared light components included in the signals S2a to S2d are removed.

Next, in step S13, color difference signals (2Rs-Gs) and (2Bs-Gs) are generated by addition / subtraction of the two generated difference signals (Rs-Gs + Bs) and (Rs-Bs). That is, the image signal generation unit 15 performs calculations represented by the following formulas 22 and 23.
(Formula 22) (S2a-S2d) + (S2c-S2b) = 2Rs-Gs
(Formula 23) (S2a-S2d)-(S2c-S2b) = 2Bs-Gs

Further, in step S14, a signal represented by the following Expression 24 is generated by adding the pixel signals S2a to S2d.
(Formula 24) S2a + S2b + S2c + S2d = 4 (Rs + Gs + Bs) + (Ir11s + kb × Ir12s + Ir21s + kd × Ir22s) = 4Ws + 4Irs
Here, Irs = (Ir11s + kb × Ir12s + Ir21s + kd × Ir22s) / 4 = (Ir11s + Ir21s) / 2.

Next, in step S15, the image signal generation unit 15 multiplies the calculated value of the calculation | S2a + S2d−S2b−S2c | by the design value kv and sets the result as an infrared light signal. That is, the following equation 25 is calculated.
(Equation 25) | S2a + S2d−S2b−S2c | × kv = | Ir11s + kd × Ir22s−kb × Ir12s−Ir21s | × kv

  This signal represents the average value of the intensity of infrared light in a specific wavelength range included in the cell incident light of each photosensitive cell. Therefore, this signal can be used as a signal indicating the image of the infrared light.

  Next, in step 16, the signal 4Irs is subtracted from the signal (4Ws + 4Irs) generated in step S14 to obtain a luminance signal. The signal 4Irs is obtained by multiplying the signal | S2a + S2d−S2b−S2c | by the design value kw. Therefore, the image signal generator 15 subtracts the signal | S2a + S2d−S2b−S2c | × kw from the signal (4Ws + 4Irs) as a luminance signal.

  Subsequently, in step 17, the image signal generation unit 15 generates an RGB color signal by matrix calculation from the two color difference signals generated in step S13 and the one luminance signal generated in step S16. Through the above processing, a color image and an IR image can be obtained.

  A specific method of colorization using the color difference signals (2Rs−Gs) and (2Bs−Gs) is disclosed in, for example, Patent Document 4. Also in this embodiment, the method disclosed in Patent Document 4 can be used. If there are two color difference signals and one luminance signal, that is, three pieces of information, an RGB signal can be calculated by matrix calculation.

  The image signal generation unit 15 performs the above signal calculation for each unit block 40 of the photosensitive cell array 2 to generate a signal (referred to as a “color image signal”) indicating an image of each of the R, G, and B color components. Then, a signal (referred to as “IR image signal”) indicating the image of the infrared light component is generated. The generated color image signal and IR image signal can be output to a recording medium (not shown) or a display unit by the image signal output unit 16.

  As described above, according to the imaging apparatus of the present embodiment, the color image signal and the IR image signal are obtained by the addition / subtraction process using the photoelectric conversion signals S2a to S2d. According to the imaging device 10 in the present embodiment, for visible light, an optical element that absorbs light is not used. Therefore, light loss can be significantly reduced as compared with the conventional technology using a color filter or the like.

  As described above, in the image sensor 10 of the present embodiment, the spectral element array having the basic configuration of 2 rows and 2 columns is arranged facing the photosensitive cell array. In the first row and the first column, a spectral element 1a that divides light into magenta light and non-magenta light is arranged. A spectral element 1b that divides light into Cb light and other than Cb light is disposed in the first row and the second column, and an infrared cut filter 8b that transmits a specific amount of specific infrared light is disposed immediately below the spectral element 1b. In the second row and the first column, a spectral element 1c that divides light into Yr light and other than Yr light is arranged. A spectral element 1d that divides light into green light and non-green light is disposed in the second row and the second column, and an infrared cut filter 8d that transmits a specific amount of specific infrared light is disposed immediately below. Since such an array pattern of spectral elements is repeatedly formed on the imaging surface, even if the selection method of the unit block 40 in the photosensitive cell array 200 is changed one row or one column at a time, the four photoelectric conversion signals obtained are: When only visible light is incident, it is always a combination of four signals represented by equations 12 to 15, and even when specific infrared light is incident, it is always a combination of four signals represented by equations 16 to 19. . That is, by performing the above signal calculation while shifting the pixel block to be calculated one row and one column at a time, it is possible to obtain the information of each RGB color component and the information of the infrared light component as many as the number of pixels. This means that the resolution of the imaging device can be increased to the number of pixels. Therefore, in addition to the higher sensitivity in the visible light region than the conventional imaging device, the imaging device of the present embodiment can generate a high-resolution color image and can also generate an IR image. Has the advantage.

  Note that the image signal generation unit 15 does not necessarily generate all the image signals of the three color components. It may be configured to generate only one or two color image signals depending on the application. Further, signal amplification, synthesis, and correction may be performed as necessary.

  In addition, it is ideal that each spectral element has the above-described spectral performance strictly, but the spectral performance may be slightly shifted. That is, the photoelectric conversion signal actually output from each photosensitive cell may be slightly deviated from the photoelectric conversion signals shown in Expressions 12 to 15. Even when the spectral performance of each spectral element deviates from the ideal performance, good color information can be obtained by correcting the signal according to the degree of deviation.

  Furthermore, the signal calculation performed by the image signal generation unit 15 in the present embodiment can be executed not by the imaging apparatus itself but by another device. For example, the color information can also be generated by causing an external device that has received an input of the photoelectric conversion signal output from the image sensor 10 to execute a program that defines the signal calculation processing in the present embodiment.

  Note that the basic structure of the image sensor 10 is not limited to the configuration shown in FIGS. For example, a configuration in which the spectral element 1a and the spectral element 1d are interchanged and the infrared cut filter 8d is also replaced with the arrangement, or a configuration in which the spectral element 1c and the spectral element 1b are interchanged and the infrared cut filter 8b is also replaced with the arrangement. The effect of the present embodiment is not changed even if it is arranged. Further, the arrangement of the first row and the arrangement of the second row shown in FIG. 8A may be interchanged, and any arrangement is possible as long as the diagonal combination of the basic light 2 rows 2 columns is similar to the above. But its effectiveness remains the same. The arrangement of the infrared cut filters 8b and 8d is also arranged above the photosensitive cells 2b and 2d in the present embodiment. However, what is important is not the arrangement itself, but when infrared light is received, It is only necessary that the photoelectric conversion signals of the sensing cells 2a and 2d are equal and the photoelectric conversion signals of the light sensing cells 2b and 2c by the light are equal.

  Further, each of the spectral elements does not have to be configured so that the light of the color component to be split is incident on the adjacent four pixels by ¼. For example, the first spectral element 1a causes part of the G light included in the incident light to enter the second photosensitive cell 2b and the other part of the G light to enter the third photosensitive cell 2c. It may be configured as follows. Similarly, the second spectral element 1b causes a part of the Yr light included in the incident light to be incident on the first photosensitive cell 2a and the other part of the Yr light is incident on the fourth photosensitive cell 2d. You may be comprised so that it may make. In addition, the third spectral element 1c causes part of the Cb light included in the incident light to enter the first photosensitive cell 2a, and causes the other part of the Cb light to enter the fourth photosensitive cell 2d. It may be configured as follows. Further, the fourth light separating element 1d causes part of the M light included in the incident light to enter the second photosensitive cell 2b and allows the other part of the M light to enter the third photosensitive cell 2c. It may be configured as follows. In this way, even if each spectral element is configured so that light does not enter the photosensitive cell of the adjacent unit block, by appropriately designing the calculation method according to the spectral distribution of the light received by each pixel, Information and infrared light information can be obtained.

  In the above description, an optical element that performs spectroscopy using the difference in refractive index between two members is used as the spectroscopic element. However, the spectroscopic element is not limited as long as light of a desired color component can be incident on each photosensitive cell. It may be anything. For example, a microprism or a dichroic mirror may be used as the spectral element. It is also possible to use different types of spectral elements in combination.

(Embodiment 2)
Next, a second embodiment will be described with reference to FIGS. The imaging device of this embodiment is different from the imaging device of Embodiment 1 only in the structure of the optical filter 11 and the imaging device 10, and the other components are the same. The following description will be focused on the differences from the imaging apparatus of the first embodiment, and the description of overlapping points will be omitted.

  The optical filter 11 in the present embodiment is designed to transmit visible light but not to transmit infrared light except for a part of a wavelength range (a narrow band centered around a wavelength of 810 nm). Except for this point, the optical filter 11 has the same characteristics as those of the first embodiment. For this reason, the infrared light handled in the present embodiment is narrow-band infrared light near 810 nm. In addition, the image sensor 10 according to the present embodiment includes a dichroic mirror that separates light into two color components, not a spectral element using diffraction. Further, the infrared cut filters 8b and 8d are not arranged. Hereinafter, the basic structure of the image sensor 10 in the present embodiment will be described.

  12A to 12E are diagrams illustrating a basic structure of the image sensor 10 in the present embodiment. The illustrated image sensor 10 is a back-illuminated image sensor. It is not important whether the type of the image sensor 10 is a backside illumination type or a front side illumination type, and the image sensor 10 may be a front side illumination type as in the first embodiment. FIG. 12A is a plan view showing the basic configuration of the image sensor 10 on the light receiving surface side. The arrangement of the photosensitive cells of the image sensor 10 in the present embodiment is the same as that in the first embodiment, and one unit block includes four photosensitive cells 2a to 2d. Dichroic mirrors 3a, 3b, 3c, and 3d are arranged to be inclined with respect to the imaging surface so as to face the photosensitive cells 2a, 2b, 2c, and 2d. The shape of the dichroic mirrors 3a to 3d is a square pyramid. Here, the inclination angle of each dichroic mirror is set so that the reflected light is totally reflected at the interface with the air layer outside the image sensor 10 and enters four pixels adjacent in the horizontal and vertical directions. Has been.

  12B to 12E are diagrams respectively showing a cross section taken along the line EE ′, a cross section taken along the line FF ′, a cross section taken along the line GG ′, and a cross section taken along the line HH ′ in FIG. 12A. As shown in the figure, the imaging device 10 includes a semiconductor substrate 7 made of a material such as silicon, photo-sensitive cells 2a to 2d disposed in the semiconductor substrate 7, and a back surface side of the semiconductor substrate 7 (a light incident side). ) And the dichroic mirrors 3 a, 3 b, 3 c, and 3 d disposed inside the transparent layer 6. A wiring layer 5 is formed on the surface side of the semiconductor substrate 7 (the side opposite to the light incident side). A fixed base 9 that supports the semiconductor substrate 7, the wiring layer 5, and the like is disposed on the surface side. The fixed substrate 9 is bonded to the semiconductor substrate 7 through the transparent layer 6.

  As shown in FIGS. 12B and 12D, the dichroic mirror 3a has characteristics of transmitting M light and reflecting G light. As a result, the M light transmitted through the dichroic mirror 3a is incident on the photosensitive cell 2a, and the G light reflected by the dichroic mirror 3a is totally reflected at the interface between the transparent layer 6 and the air to be horizontal and vertical. Is incident on four photosensitive cells adjacent to each other.

  12B and 12E, the dichroic mirror 3b has characteristics of transmitting Cb light and reflecting Yr light. As a result, the Cb light transmitted through the dichroic mirror 3b is incident on the photosensitive cell 2b, and the Yr light reflected by the dichroic mirror 3b is totally reflected at the interface between the transparent layer 6 and the air to be horizontal and vertical. Is incident on four photosensitive cells adjacent to each other.

  As shown in FIGS. 12C and 12D, the dichroic mirror 3c has a characteristic of transmitting Yr light and reflecting Cb light. As a result, the Yr light transmitted through the dichroic mirror 3c is incident on the photosensitive cell 2c, and the Cb light reflected by the dichroic mirror 3c is totally reflected at the interface between the transparent layer 6 and the air and is horizontally and vertically oriented. The incident light is incident on four adjacent photosensitive cells.

  12C and 12E, the dichroic mirror 3d has characteristics of transmitting G light and reflecting M light. As a result, the G light transmitted through the dichroic mirror 3d is incident on the photosensitive cell 2d, and the M light reflected by the dichroic mirror 3d is totally reflected at the interface between the transparent layer 6 and the air, in the horizontal and vertical directions. The incident light is incident on four adjacent photosensitive cells.

  Here, the spectral characteristics of the dichroic mirrors 3a to 3d are shown in FIGS. 13A and 13B. FIG. 13A is a diagram showing the transmittance characteristics of the dichroic mirrors 3a and 3d, and shows the spectral distribution of light incident on the photosensitive cells 2a and 2d. On the other hand, FIG. 13B is a diagram showing the transmittance characteristics of the dichroic mirrors 3b and 3c, and shows the spectral distribution of light incident on the photosensitive cells 2b and 2c. In the present embodiment, for visible light, the spectral distribution of light incident on each photosensitive cell is substantially the same as that in the first embodiment, but for infrared light, it is different from that in the first embodiment. In the present embodiment, the dichroic mirrors 3a to 3d are designed so that the received light amounts of the photosensitive cells 2a and 2d are substantially equal and the received light amounts of the photosensitive cells 2b and 2c are substantially equal for infrared light having a wavelength of about 810 nm. ing. As a result, when the infrared light is received, the relationships S2a = S2d and S2b = S2c are established for the photoelectric conversion signals of the photosensitive cells 2a to 2d. In the first embodiment, this relationship is realized by using the infrared cut filters 8b and 8d, but in the present embodiment, these relationships are realized by the spectral characteristics of the dichroic mirrors 3a, 3b, 3c, and 3d.

  By using such dichroic mirrors 3a to 3d, each of the photosensitive cells 2a to 2d receives light of the same color component as when the configuration in the first embodiment is adopted when receiving only visible light. That is, the photosensitive cell 2a receives the M light transmitted through the dichroic mirror 3a and the light (Yr + Cb) / 2 that is reflected by the four dichroic mirrors adjacent in the horizontal and vertical directions. The photosensitive cell 2b receives Cb light transmitted through the dichroic mirror 3b and light (M + G) / 2 reflected by four dichroic mirrors adjacent in the horizontal and vertical directions. The photosensitive cell 2c receives Yr light transmitted through the dichroic mirror 3c and light (M + G) / 2 reflected by the dichroic mirror adjacent in the horizontal and vertical directions. The photosensitive cell 2d receives the G light transmitted through the dichroic mirror 3d and the light (Yr + Cb) / 2 that is reflected by the four dichroic mirrors adjacent in the horizontal and vertical directions. As a result, the photoelectric conversion signals S2a to S2d respectively output from the photosensitive cells 2a to 2d can be expressed by equations 12 to 15 as in the case of adopting the configuration in the first embodiment. Further, in the case of receiving infrared light in the vicinity of a wavelength of 810 nm in addition to visible light, S2a to S2d are expressed by equations 16 to 19 as in the first embodiment. Therefore, color information and infrared light information can be obtained by the same processing as in the first embodiment.

  Thus, according to the imaging apparatus of the present embodiment, a color image signal and an IR image signal are obtained by signal calculation processing using the photoelectric conversion signals S2a to S2d, as in the imaging apparatus of the first embodiment. The imaging element 10 according to the present embodiment also does not use an optical element that absorbs visible light, so that it is possible to significantly reduce light loss as compared with the conventional technique using a color filter or the like. In addition, since three color signals are obtained by calculation using four photoelectric conversion signals, there is an effect that the amount of color information obtained with respect to the number of pixels is larger than in the case of a conventional image sensor.

  As described above, in the imaging device 10 of the present embodiment, the dichroic mirror 3a that divides light into magenta light and non-magenta light is arranged in the first row and the first column. A dichroic mirror 3b that divides light into Cb light and other than Cb light is arranged in the first row and the second column. A dichroic mirror 3c that divides light into Yr light and other than Yr light is arranged in the second row and the first column. A dichroic mirror 3d that divides light into green light and non-green light is arranged in the second row and the second column. Since such an array pattern of spectral elements is repeatedly formed on the imaging surface, the four photoelectric conversion signals obtained are always obtained even if the method of selecting unit blocks in the photosensitive cell array 200 is changed one row or one column at a time. This is a combination of four signals represented by Expressions 12 to 15 and Expressions 16 to 19. That is, by performing the above signal calculation while shifting the pixel block to be calculated one row and one column at a time, it is possible to obtain RGB color component information for almost the same number of pixels, and the infrared light component information is also the same. Can get to. Therefore, in addition to higher sensitivity than the conventional imaging device, the imaging device of the present embodiment can generate a high-resolution color image and IR image.

  Note that the basic structure of the image sensor 10 is not limited to the configuration shown in FIGS. For example, even if the dichroic mirror 3a and the dichroic mirror 3d are interchanged and the dichroic mirror 3c and the dichroic mirror 3b are interchanged, the effect of this embodiment is not changed. Further, even if the arrangement of the first row and the arrangement of the second row shown in FIG. 12A are interchanged, the effectiveness is not changed.

  In this embodiment, a dichroic mirror is used as the spectral element, but the spectral element may be any element as long as it separates into primary color light and its complementary color light. For example, as the spectroscopic element, a microprism or an optical element using diffraction used in the first embodiment may be used. It is also possible to use different types of spectral elements in combination.

  Moreover, it is not necessary for all of the branched light to enter the adjacent photosensitive cell for each spectral element, and a part of the branched light may be incident on the adjacent photosensitive cell. In that case, the image signal generation unit 15 may be configured to correct the signal processing based on the proportion of the branched light that enters the adjacent photosensitive cell.

  In the first and second embodiments, infrared light having an intensity peak at an intermediate wavelength of 700 to 800 nm and infrared light having an intensity peak near 810 nm are used. There is no limitation, and a wavelength band other than that may be used.

  The solid-state imaging device and the imaging apparatus of the present disclosure are effective for all cameras using the solid-state imaging device. For example, it can be used for consumer cameras such as digital still cameras and digital video cameras, and industrial solid-state surveillance cameras.

1a, 1b, 1c, 1d Spectral element 2, 2a, 2b, 2c, 2d Photosensitive cell 3a, 3b, 3c, 3d Spectral element (dichroic mirror)
4 Microlens 5 Wiring Layer of Image Sensor 6 Transparent Layer 8b, 8d Infrared Cut Filter 7 Silicon Substrate 9 Fixed Substrate 10 Image Sensor 11 Optical Filter 12 Optical Lens 13 Signal Generating / Receiving Unit 14 Element Driving Unit 15 Image Signal Generating Unit 16 Image signal output unit 19 IR light source 30 Memory 40 Photosensitive cell unit block 100 Spectral element array 200 Photosensitive cell array 300 Imaging unit 400 Signal processing unit

Claims (15)

Each of the first photosensitive cells arranged in the first row and the first column, the second photosensitive cells arranged in the first row and the second column, and the third photosensitive cells arranged in the second row and the first column. And a photosensitive cell array in which a plurality of unit blocks including a fourth photosensitive cell arranged in the second row and the second column are two-dimensionally arranged,
A spectral element array disposed opposite to the photosensitive cell array and including a plurality of spectral elements, wherein light having different spectral distributions is incident on the first to fourth photosensitive cells, and the first and fourth A spectral element configured such that a total spectral distribution of visible light incident on the photosensitive cell is substantially equal to a total spectral distribution of visible light incident on the second and third photosensitive cells. An array,
A solid-state imaging device. When it is assumed that the spectral element array does not exist, the light received by each photosensitive cell is the cell incident light of each photosensitive cell,
A part of the color component included in the visible light is a first color component, the other part of the color component included in the visible light is a second color component, and a complementary color component of the second color component is the first color component. 3 and when the fourth color component is a complementary color component of the first color component,
The spectral element array is:
The first color component light is decreased from the cell incident light of the first photosensitive cell, and the second color component light and the third color component light are increased. Incident on the light sensitive cell,
The second color component light is decreased from the cell incident light of the second photosensitive cell, and the first color component light and the fourth color component light are increased. Incident on the light sensitive cell,
The third color component light is decreased from the cell incident light of the third photosensitive cell, and the light of the first color component light and the fourth color component light is increased. Incident on the light sensitive cell,
The fourth color component light is decreased from the cell incident light of the fourth photosensitive cell, and the second color component light and the third color component light are increased. Configured to be incident on the photosensitive cell of the
The solid-state imaging device according to claim 1.   The spectral element array allows infrared light of a specific wavelength range to be incident on the first to fourth photosensitive cells, and infrared light of the specific wavelength range is incident on the first and fourth photosensitive cells. Intensities of light are substantially equal, and intensities of infrared light in the specific wavelength range incident on the second and third photosensitive cells are substantially equal, and the first and fourth photosensitive cells The infrared light intensity of the specific wavelength range that is incident is different from the intensity of infrared light of the specific wavelength range that is incident on the second and third photosensitive cells. Item 3. The solid-state imaging device according to Item 1 or 2. The spectral element array includes, in each unit block, a first spectral element disposed to face the first photosensitive cell and a second spectral element disposed to face the second photosensitive cell. A third spectral element disposed opposite to the third photosensitive cell, and a fourth spectral element disposed opposite to the fourth photosensitive cell,
The first spectral element causes a part of the light of the first color component included in incident light to enter the second photosensitive cell, and the light of the first color component included in the incident light. Other part of the light is incident on the third photosensitive cell, the light of the fourth color component included in the incident light is incident on the first photosensitive cell,
The second spectral element causes a part of the light of the second color component included in the incident light to enter the first photosensitive cell, and the light of the second color component included in the incident light. The other part of the light is incident on the fourth photosensitive cell, the light of the third color component included in the incident light is incident on the second photosensitive cell,
The third spectral element causes a part of the light of the third color component included in the incident light to enter the first photosensitive cell, and the light of the third color component included in the incident light. The other part of the light is incident on the fourth photosensitive cell, the light of the second color component included in the incident light is incident on the third photosensitive cell,
The fourth spectral element causes a part of the light of the fourth color component included in incident light to enter the second photosensitive cell, and the light of the fourth color component included in the incident light. The other part of the light is incident on the third photosensitive cell, and the light of the first color component included in the incident light is incident on the fourth photosensitive cell.
The solid-state image sensor according to any one of claims 1 to 3. The first spectral element converts the light of the first color component included in incident light into two light detection cells included in the second light detection cell, the third light detection cell, and an adjacent unit block. Making the same amount incident on the cell, causing the light of the fourth color component contained in the incident light to enter the first photosensitive cell,
The second spectral element converts the light of the second color component included in incident light into two light detection elements included in the first light detection cell, the fourth light detection cell, and the adjacent unit block. Making the same amount incident on the cell, causing the light of the third color component contained in the incident light to enter the second photosensitive cell,
The third spectral element converts the light of the third color component included in incident light into two light detection elements included in the first light detection cell, the fourth light detection cell, and the adjacent unit block. Making the same amount incident on the cell, causing the light of the second color component contained in the incident light to enter the third photosensitive cell,
The fourth spectral element converts the light of the fourth color component included in incident light into two light detection elements included in the second light detection cell, the third light detection cell, and the adjacent unit block. Making the same amount incident on the cell, and making the light of the first color component included in the incident light enter the fourth photosensitive cell,
The solid-state imaging device according to claim 4.   One of the first and fourth color components is a magenta color component, the other of the first and fourth color components is a green color component, and one of the second and third color components is The red color component including a part of green, and the other of the second and third color components is a blue color component including another part of green. Solid-state image sensor. Each of the first to fourth spectral elements has a high refractive index transparent portion and a low refractive index that is lower than the refractive index of the high refractive index transparent portion and is provided around the high refractive index transparent portion. A refractive index transparent portion,
At least one of the shape and the size of the high refractive index transparent part of the first to fourth spectral elements is different from each other.
The solid-state imaging device according to claim 1.   7. The solid-state imaging device according to claim 1, wherein each of the first to fourth light-splitting elements includes a dichroic mirror, and performs spectroscopy using the dichroic mirror. A solid-state imaging device according to any one of claims 1 to 8,
An optical system for forming an image on the solid-state imaging device;
An optical filter that transmits only visible light and infrared light in a specific wavelength range; and
A signal processing unit that processes a signal output from the solid-state imaging device, the first photoelectric conversion signal output from the first photosensitive cell, and the second output from the second photosensitive cell. Color image information by calculation using the photoelectric conversion signal, the third photoelectric conversion signal output from the third photosensitive cell, and the fourth photoelectric conversion signal output from the fourth photosensitive cell. A signal processing unit for generating infrared image information;
An imaging apparatus comprising:   The imaging apparatus according to claim 9, further comprising a light source that emits infrared light in the specific wavelength range.   The signal processing unit includes a first difference signal representing a difference between the first photoelectric conversion signal and the fourth photoelectric conversion signal, and the second photoelectric conversion signal and the third photoelectric conversion signal. A first color difference signal and a second color difference signal are generated by an operation including addition and subtraction with a second difference signal representing a difference, and the addition result of the first photoelectric conversion signal and the fourth photoelectric conversion signal The imaging device according to claim 9, wherein a difference signal between the second photoelectric conversion signal and the addition result of the third photoelectric conversion signal is generated.   The signal processing unit includes any of addition of the first and fourth photoelectric conversion signals, addition of the second and third photoelectric conversion signals, and addition of the first to fourth photoelectric conversion signals. The imaging device according to claim 9 or 10, wherein a luminance signal is generated by calculation. A method for processing a signal output from the solid-state imaging device according to claim 1,
A first photoelectric conversion signal output from the first photosensitive cell, a second photoelectric conversion signal output from the second photosensitive cell, and a third photoelectric signal output from the third photosensitive cell. Obtaining a photoelectric conversion signal and a fourth photoelectric conversion signal output from the fourth photosensitive cell; and
Step B for generating color information using the first to fourth photoelectric conversion signals;
Including methods. Step B includes
Generating a first difference signal indicating a difference between the first photoelectric conversion signal and the fourth photoelectric conversion signal;
Generating a second difference signal indicating a difference between the second photoelectric conversion signal and the third photoelectric conversion signal;
Generating a first color difference signal and a second color difference signal by an operation including addition / subtraction between the first difference signal and the second difference signal;
Generating a difference signal between the addition result of the first photoelectric conversion signal and the fourth photoelectric conversion signal, and the addition result of the second photoelectric conversion signal and the third photoelectric conversion signal;
14. The method of claim 13, comprising: Step B includes
A luminance signal is generated by an operation including any of the addition of the first and fourth photoelectric conversion signals, the addition of the second and third photoelectric conversion signals, and the addition of the first to fourth photoelectric conversion signals. And steps to
Using the luminance signal, the first color difference signal, the second color difference signal, and the difference signal, red, green, and blue color signals included in the cell incident light and a specific infrared light signal A step of generating
15. The method of claim 14, further comprising:

Images