JP2009257919A - Solid-state imaging device, imaging system, and detection device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a low cost solid-state imaging device capable of suitably imaging an image even of a moving object in which the object is highlighted. <P>SOLUTION: The solid-state imaging device generates an image in which a predetermined object having a distributed absorption spectrum is highlighted, and comprises: a plurality of optical filters 130 that transmit light in a first wavelength band including a peak of the distributed absorption spectrum, a plurality of optical filters 131 that transmit light in a second wavelength band of the same color as that of the first wavelength band without including the peak; a plurality of image elements 25 converting light transmitted through the optical filters 130 to image data 62; a plurality of image elements 26 formed on a semiconductor circuit 119 on which the image elements 25 are also formed, and converting light transmitted through the optical filters 131 to image data 63; and an image generation unit 30 for generating image data 68 in which the object is highlighted, utilizing the difference in the brightness between the image elements 25 and the image elements 26. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a solid-state imaging device, an imaging system, and a detection device, and more particularly to a solid-state imaging device that generates an image in which a predetermined substance having a distributed absorption spectrum is emphasized.

  A specific substance using the image of the part absorbed by the subject after passing through a narrowband filter and transmitting light such as reflected light, transmitted light, or scattered light from the subject having a distributed absorption spectrum Several methods have been proposed (see Patent Documents 1 to 4). Here, the dispersion-type absorption spectrum is an absorption spectrum having a steep absorption peak due to any one or more of symmetric stretching vibration, asymmetric stretching vibration, and bending vibration between atoms or molecules.

  The detection device described in Patent Document 1 irradiates light on a subject, and rotates only a filter having different transmission characteristics by a revolver method, thereby acquiring only transmitted light having a desired wavelength out of reflected light from the subject. . Thereby, the detection apparatus of patent document 1 can acquire several data corresponding to the light of a respectively different wavelength.

  Here, the wavelengths of light transmitted through the plurality of filters are set to wavelengths that match the local wavelength of the absorption peak of the subject and wavelengths that do not match. The detection device described in Patent Document 1 generates an image in which a portion where light is absorbed is emphasized by subtracting a plurality of pieces of data corresponding to the acquired light of different wavelengths.

  In addition, the camera device described in Patent Document 2 separates light using a prism, and images the separated light with an image sensor via an optical filter.

In addition, the techniques described in Patent Document 3 and Patent Document 4 change the wavelength band of light in the near-infrared region to be transmitted after arranging a simple spectroscope in front of the image sensor. Specifically, the simple spectroscope has a structure for adjusting the air gap between two spectral transmittance variable elements.
Japanese Patent No. 3733434 Japanese Patent No. 37772016 JP-A-2005-308688 JP 2007-316486 A

  However, the detection device described in Patent Document 1 has a problem that a system such as a rotary filter becomes large and a cost that a mechanism and the like increase.

  In addition, the camera device of Patent Document 2 can reduce the size of the system as compared with the detection device described in Patent Document 1, but has a problem that two or more image sensors are required and the cost is increased.

  In addition, the techniques described in Patent Document 3 and Patent Document 4 can be realized by a single image sensor, but in order to acquire a plurality of near-infrared lights in different wavelength bands, imaging is performed after moving the air gap in the small spectroscope. There is a need to. As a result, the techniques described in Patent Literature 3 and Patent Literature 4 are not suitable for imaging a moving subject, for example, because it takes time to switch wavelength bands.

  The present invention has been made to solve the above-described problems, and is a solid-state imaging device, an imaging system, and a detection device that are suitable for capturing a low-cost image that emphasizes a subject even with a moving subject. The purpose is to provide.

  In order to achieve the above object, a solid-state imaging device according to the present invention is a solid-state imaging device that generates an image in which a predetermined substance having a dispersion-type absorption spectrum is emphasized, and the peak of the dispersion-type absorption spectrum is obtained. First filtering means that transmits a first wavelength band including the second filtering means that does not include the peak and transmits a second wavelength band that is the same color band as the first wavelength band; and A plurality of first photoelectric conversion means for converting light transmitted by the filtering means into first image data, and the first photoelectric conversion means formed on the same semiconductor substrate and transmitted by the second filtering means A plurality of second photoelectric conversion means for converting light into second image data; and an image generation means for generating an image in which the substance is emphasized using a difference in luminance between the first image data and the second image data. Is provided.

  According to this configuration, the solid-state imaging device according to the present invention generates first image data and second image data obtained by imaging light in different wavelength bands by a plurality of photoelectric conversion units formed on a single semiconductor substrate. Therefore, compared with the case where a plurality of image sensors are provided, it is possible to realize downsizing and cost reduction. In addition, the solid-state imaging device according to the present invention can simultaneously generate the first image data and the second image data without requiring an operation such as moving an air gap in the small spectroscope. Therefore, the solid-state imaging device according to the present invention can appropriately capture an image in which a subject is emphasized even with respect to a moving subject.

  The image generating means may include subtracting means for generating an image in which the substance is emphasized by subtracting the first image data from the second image data.

  According to this configuration, in the image generated by the image generation means, the luminance signal of the portion where the light is not absorbed by the substance is almost 0, and the luminance signal of the portion where the light is absorbed by the subject is a value other than 0. Become. That is, the solid-state imaging device according to the present invention can generate an image in which only a region where a specific substance exists, that is, an image in which the specific substance is emphasized.

  The image generation means further includes a correction means for amplifying or attenuating at least one of the first image data and the second image data, and the subtraction means is amplified or attenuated by the correction means. An image in which the substance is emphasized may be generated by subtracting the first image data and the second image data.

  According to this configuration, the correction unit can correct a difference in sensitivity between the first photoelectric conversion unit and the second photoelectric conversion unit in the first wavelength band and the second wavelength band, a difference in illumination intensity, and the like. Thereby, the solid-state imaging device according to the present invention can generate an image in which a specific substance is more clearly emphasized.

  The plurality of first photoelectric conversion units and the plurality of second photoelectric conversion units are alternately arranged in a row direction or a column direction, and the image generation unit further includes the first photoelectric conversion unit in the first image data. By generating data corresponding to the pixel position of the two photoelectric conversion means from the pixel data of the first image data adjacent to the pixel position, the data of the pixel position of the second photoelectric conversion means in the first image data Is generated from the pixel data of the second image data adjacent to the pixel position, and data corresponding to the pixel position of the first photoelectric conversion means in the second image data is generated. Pixel interpolating means for interpolating the pixel position data of the first photoelectric conversion means, and the subtracting means is the second image data interpolated by the pixel interpolating means. It may generate an image that emphasizes the material by subtracting the first image data pixel interpolation by the pixel interpolating unit.

  According to this configuration, the pixel interpolation unit performs pixel interpolation of the first image data and the second image data. Thereby, the solid-state imaging device according to the present invention can generate a highly accurate image in which a specific substance is emphasized.

  The image generation means may include division means for generating an image in which the substance is emphasized by dividing the second image data by the first image data.

  According to this configuration, in the image generated by the image generation unit, the luminance signal of the portion where the light is not absorbed by the substance is small, and the luminance signal of the portion where the light is absorbed by the subject is light absorbed. Bigger than no part. That is, the solid-state imaging device according to the present invention can generate an image in which only a region where a specific substance exists, that is, an image in which the specific substance is emphasized.

  Further, the first filtering means and the second filtering means respectively include a first multilayer film and a second multilayer film in which a high refractive index film and a low refractive index film are alternately laminated, and the first multilayer film. And the third film laminated between the second multilayer film, and the high refractive index film and the low refractive index film included in the first filtering means are respectively the second filtering means. The third film included in the first filtering means is the same as the third film and the film included in the second filtering means. The thickness may be different.

  According to this configuration, two types of first filtering means and second filtering means that transmit light of different wavelength bands can be easily generated on a single semiconductor substrate. Furthermore, since the first filtering means and the second filtering means can narrow the transmission band by increasing the number of stacked high refractive index films and low refractive index films, the peak of the absorption spectrum is very close to single wavelength light. It can transmit light of wavelength.

  The solid-state imaging device further includes third filtering means that transmits light in a third wavelength band including the first wavelength band and the second wavelength band, and the first photoelectric conversion means includes the first photoelectric conversion means, The light transmitted by the three filtering means and the light transmitted by the first filtering means is converted into the first image data, the second photoelectric conversion means is transmitted by the third filtering means, and the The light transmitted by the second filtering means may be converted into the second image data.

  According to this configuration, the third filtering unit can block light in an unnecessary wavelength band that cannot be blocked by the first filtering unit and the second filtering unit. As a result, the solid-state imaging device according to the present invention does not assume that the light in the surrounding environment is turned off or uses a light source with a narrowed wavelength range, etc., and the light in the first wavelength band and the second wavelength. Only band light can be imaged.

  The imaging system according to the present invention includes the solid-state imaging device and illumination means for irradiating light in a fourth wavelength band including the first wavelength band and the second wavelength band.

  According to this configuration, the imaging system according to the present invention generates first image data and second image data obtained by imaging light in different wavelength bands by a plurality of photoelectric conversion means formed on a single semiconductor substrate. Therefore, compared with the case where a plurality of image sensors are provided, a reduction in size and cost can be realized. In addition, the imaging system according to the present invention can simultaneously generate the first image data and the second image data without requiring an operation such as moving the air gap in the small spectroscope. Therefore, the imaging system according to the present invention can appropriately capture an image in which a subject is emphasized even with respect to a moving subject.

  The detection device according to the present invention is a detection device that detects a predetermined substance having a dispersion-type absorption spectrum, and is a first filtered light that transmits a first wavelength band including a peak of the dispersion-type absorption spectrum. Means, a plurality of second filtering means that do not include the peak and pass through a second wavelength band that is the same color band as the first wavelength band, and light transmitted by the first filtering means First photoelectric conversion means for converting into one signal and second photoelectric conversion means for converting light transmitted through the second filtering means into a second signal formed on the same semiconductor substrate as the first photoelectric conversion means And signal generation means for generating a third signal indicating that the substance has been detected using a difference in luminance between the first signal and the second signal.

  According to this configuration, the detection device according to the present invention generates the first signal and the second signal obtained by imaging light of different wavelength bands by the two photoelectric conversion units formed on the single semiconductor substrate. Compared with the case where a plurality of image sensors are provided, it is possible to realize downsizing and cost reduction. Moreover, the detection apparatus according to the present invention can simultaneously generate the first signal and the second signal without requiring an operation such as moving an air gap in the small spectroscope. Therefore, the detection device according to the present invention can also suitably detect a moving subject.

  The present invention can be realized not only as such a solid-state imaging device, imaging system, and detection device, but also as an imaging method or detection method using characteristic means included in the solid-state imaging device, imaging system, or detection device as a step. It can also be realized as a program that causes a computer to execute part or all of such characteristic steps. Needless to say, such a program can be distributed via a recording medium such as a CD-ROM and a transmission medium such as the Internet.

  As described above, the present invention can provide a solid-state imaging device, an imaging system, and a detection device that can appropriately capture an image in which a subject is emphasized even with a moving subject at low cost.

  Hereinafter, embodiments of an imaging system according to the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
The imaging system according to Embodiment 1 of the present invention includes a one-chip solid-state imaging device that converts light of different wavelength bands into electrical signals. Thereby, the imaging system according to the present invention can realize downsizing and cost reduction of the system. Furthermore, the imaging system according to the present invention can suitably capture an image in which a subject is emphasized even with respect to a moving subject.

  First, the configuration of the imaging system according to the embodiment of the present invention will be described.

  FIG. 1 is a block diagram showing a configuration of an imaging system according to Embodiment 1 of the present invention.

  The imaging system 10 illustrated in FIG. 1 captures an image of a subject and generates image data 68 that highlights and displays a predetermined substance having a distributed absorption spectrum. In the following, an example in which image data 68 in which water is emphasized is generated will be described.

  The imaging system 10 includes an imaging unit 20, an image generation unit 30, and an illumination unit 40.

  The illumination unit 40 irradiates the subject 50 with near-infrared light 60 in a third wavelength band including a first wavelength band in the vicinity of a wavelength of 960 nm, which is an absorption peak of water, and a second wavelength band in the vicinity of a wavelength of 850 nm. For example, the illumination unit 40 is a light emitting diode.

  The imaging unit 20 images light in the first wavelength band near the wavelength of 960 nm among the reflected light 61 reflected by the subject 50, and generates image data 62. In addition, the imaging unit 20 images light in the second wavelength band near the wavelength of 850 nm among the reflected light 61 reflected by the subject 50, and generates image data 63. The imaging unit 20 is, for example, a CMOS image sensor formed on a single semiconductor substrate.

  FIG. 2 is a diagram illustrating a configuration of the imaging unit 20. The imaging unit 20 includes a pixel array 21, a vertical scanning unit 22, a horizontal scanning unit 23, and an A / D conversion unit 24.

  The pixel array 21 includes a plurality of pixels 25 and a plurality of pixels 26 arranged in a matrix. The plurality of pixels 25 and the plurality of pixels 26 accumulate signal charges corresponding to the amount of received light.

  The vertical scanning unit 22 sequentially selects the pixels 25 and 26 in each row among the plurality of pixels 25 and 26.

  The horizontal scanning unit 23 sequentially selects the pixels 25 and 26 in each column among the plurality of pixels 25 and 26.

  The signal charges accumulated in the pixels 25 or 26 whose rows are selected by the vertical scanning unit 22 and whose columns are selected by the horizontal scanning unit 23 are converted into voltage or current and input to the A / D conversion unit 24. . The A / D converter 24 converts the input voltage or current from an analog signal to a digital signal, and outputs the converted digital signal as image data 62 and 63.

  FIG. 3 is a diagram illustrating a cross-sectional structure of the pixel 25 and the pixel 26.

  The pixel 25 and the pixel 26 are formed on the substrate 119. The substrate 119 is a semiconductor substrate, for example, a silicon substrate.

  The pixel 25 includes a photodiode 128, wirings 117, 118, 125 and 126, an interlayer insulating film 129, an optical filter 130, a planarization film 121, and a microlens 103.

  The pixel 26 includes a photodiode 127, wirings 123, 124, 125 and 126, an interlayer insulating film 129, an optical filter 131, a planarization film 121, and a microlens 120.

  The configurations of the pixel 25 and the pixel 26 are the same except that the configurations of the optical filters 130 and 131 are different.

  The photodiodes 127 and 128 are photoelectric conversion elements that convert incident light into signal charges.

  The wirings 117, 118, 123, 124, 125, and 126 are metal wirings that are formed in the interlayer insulating film 129 and also function as a light shielding film.

  The planarization film 121 is a film for planarizing the surface so that there is no step between the pixel 25 and the pixel 26 when the microlenses 103 and 120 are formed.

  The optical filters 130 and 131 selectively transmit light in different wavelength bands. The optical filters 130 and 131 are multilayer filters in which a plurality of dielectric films are stacked.

  The optical filters 130 and 131 include a first multilayer film in which high refractive index films 104, 106, and 108 and low refractive index films 105 and 107 are alternately stacked, and high refractive index films 110, 112, 114, and 16 and low refractive index. A second multilayer film in which the rate films 111, 113, and 115 are alternately stacked. The optical filter 130 includes a low-refractive index film 109 stacked between the first multilayer film and the second multilayer film, and the optical filter 131 is stacked between the first multilayer film and the second multilayer film. The low refractive index film 122 is included.

  The optical filter 130 and the optical filter 131 are different in film thickness between the low refractive index film 109 and the low refractive index film 122. The low refractive index films 105, 107, 111, 113 and 115 other than the low refractive index films 109 and 122 and the high refractive index films 104, 106, 108, 110, 112, 114 and 116 are the optical filter 130 and 131 is commonly used and has the same film thickness.

The high refractive index films 104, 106, 108, 110, 112, 114, and 116 are made of a high refractive index material, for example, titanium oxide (TiO ₂ ). The low refractive index films 105, 107, 109, 111, 113, 115, and 122 are made of a material having a lower refractive index than the high refractive index films 104, 106, 108, 110, 112, 114, and 116. For example, silicon It is composed of an oxide (SiO ₂ ).

  In FIG. 4, the high refractive index films 104, 106, 108, 110, 112, and 114 have a thickness of 86 nm, and the low refractive index films 105, 107, 111, 113, and 115 have a thickness of 149 nm. It is a figure which shows the permeation | transmission characteristic of the optical filters 130 and 131 when the film thickness of the refractive index film | membrane 109 is 5 nm, and the film thickness of the low refractive index film | membrane 122 is 58 nm.

  As shown in FIG. 4, the transmission characteristic 502 of the optical filter 130 has a peak near the wavelength of 960 nm, and the transmission characteristic 501 of the optical filter 131 has a peak near the wavelength of 850 nm.

  5 and 6 are diagrams showing an absorption spectrum 701 of water. FIG. 5 is a diagram showing the water absorption spectrum 701 with the vertical axis representing logarithm, and FIG. 6 is a diagram showing the water absorption spectrum 701 with the vertical axis representing linear.

  As shown in FIGS. 5 and 6, the water absorption spectrum 701 has a peak 702 at a wavelength of 960 nm.

  Thus, the optical filter 130 selectively transmits light in a wavelength band near the wavelength 960 nm of the peak 702 of the water absorption spectrum 701.

  The optical filter 131 selectively transmits light in a wavelength band near a wavelength of 850 nm in a region 704 that does not include the peak 702 of the water absorption spectrum 701.

  As described above, the configuration of the optical filters 130 and 131 includes the high-refractive-index films 104, 106, 108, 110, 112, and 114 whose optical film thickness is a quarter of the wavelength of the peak 702 of the absorption spectrum of water. A multilayer film in which a plurality of low refractive index films 105, 107, 111, 113, and 115 whose optical film thickness is a quarter of the wavelength of the peak 702 of the absorption spectrum of water is alternately laminated is The low refractive index films 109 and 122 having an optical film thickness different from the wavelength are sandwiched. Further, in the optical filters 130 and 131, the optical film thicknesses of the low refractive index films 109 and 122 are different.

  Thereby, the optical filter 130 that transmits only light around the peak wavelength of the water absorption spectrum and the optical filter 131 that transmits only light around the wavelength different from the peak wavelength can be realized. This is because the dielectric multilayer film, in which a low-refractive index material with a certain film thickness and a high-refractive index material with a certain film thickness are periodically laminated, has a non-transmissive photonic band gap that does not transmit light. By making one of the dielectric multilayer films different in film thickness, a physical phenomenon that can form a wavelength band to be transmitted in the photonic band gap is utilized.

  In addition, such a multilayer filter is more transmissive by increasing the number of layers of the high refractive index films 104, 106, 108, 110, 112, and 114 and the low refractive index films 105, 107, 111, 113, and 115. Since the band can be narrowed, it is possible to transmit light that is very close to light having a single wavelength of the peak wavelength in the water absorption spectrum. Further, in configuring a plurality of types of filters on the photoelectric conversion element, the high refractive index films 104, 106, 108, 110, 112 and 114 and the low refractive index films 105, 107, 111, 113 and 115 are respectively There is also an advantage that the filter manufacturing process is facilitated because the film thickness and the number of layers are common to these types of filters.

  The description will be given again with reference to FIG.

  The microlenses 103 and 120 are formed on the optical filters 130 and 131 via the planarization film 121, respectively.

  The optical filter 140 is formed on the pixels 25 and 26. The optical filter 140 transmits light in a wavelength band including the wavelength band that the optical filter 130 transmits and the wavelength band that the optical filter 131 transmits. For example, the optical filter 140 is a band-pass filter that blocks visible light and transmits infrared light.

  FIG. 7 is a diagram showing a transmission characteristic 601 of light incident on the photodiode 127 via the optical filters 140 and 131 and a transmission characteristic 602 of light incident on the photodiode 128 via the optical filters 140 and 130.

  As shown in FIG. 7, by providing an optical filter 140 that blocks visible light, only light having a wavelength of about 850 nm is incident on the photodiode 127, and only light having a wavelength of about 960 nm is incident on the photodiode 128. .

  Next, the arrangement of the pixels 25 and the pixels 26 in the pixel array 21 will be described.

  8, FIG. 9 and FIG. 10 are diagrams showing examples of arrangement of the pixels 25 and 26. FIG.

  The plurality of pixels 25 and the plurality of pixels 26 are alternately arranged in the row direction or the column direction so that each is adjacent to the other. For example, as shown in FIG. 8, the pixels 25 and the pixels 26 are arranged in a staggered pattern. In addition, the pixel units 217, 218, 219, and 220 configured by four pixels are each configured by the same pixel pattern. In the pixel array 21, pixel units 217, 218, 219, and 220 are arranged two-dimensionally.

  Further, as shown in FIGS. 9 and 10, the pixels 25 and the pixels 26 may be arranged in a stripe shape. In the pixel array 21, pixel units 317 and 319 including four pixels 25 and pixel units 318 and 320 including four pixels 26 are alternately arranged two-dimensionally.

  Note that the pixels 201, 204, 205, 208, 209, 212, 213, and 216 shown in FIG. 8 correspond to the pixel 25, and the pixels 202, 203, 206, 207, 210, 211, 214, and 215 correspond to the pixel 26. To do. Further, the pixels 301 to 304 and 309 to 312 shown in FIG. 9 and the pixels 401 to 404 and 409 to 412 shown in FIG. 10 correspond to the pixel 25, and the pixels 305 to 308 and 313 to 316 shown in FIG. Pixels 405 to 408 and 413 to 416 shown in FIG. Note that the combination of the two types of pixels may be reversed.

  The image generation unit 30 generates image data 68 in which water is emphasized using the difference in luminance between the image data 62 and the image data 63 generated by the imaging unit 20. The image generation unit 30 includes a pixel interpolation unit 31, a luminance correction unit 32, and a subtraction unit 33.

  The pixel interpolation unit 31 generates image data 64 and image data 65 by performing pixel interpolation on the image data 62 and the image data 63, respectively.

  The brightness correction unit 32 generates image data 66 and image data 67 by amplifying or attenuating at least one of the image data 64 and the image data 65.

  The subtracting unit 33 subtracts the image data 66 from the image data 67 to generate image data 68 in which water is emphasized.

  Further, the imaging unit 20 and the image generation unit 30 are formed as a one-chip semiconductor integrated circuit on a single semiconductor substrate 119, for example. Note that the imaging unit 20 and the image generation unit 30 may be formed as individual semiconductor integrated circuits. Further, the imaging unit 20 and some of the functional blocks included in the image generation unit 30 may be formed as a one-chip semiconductor integrated circuit, or the image generation unit 30 may be configured by a plurality of semiconductor integrated circuits. Good. Further, the function of the image generation unit 30 may be realized by a processor such as a CPU executing a program.

  Next, the operation of the imaging system 10 according to Embodiment 1 of the present invention will be described.

  FIG. 11 is a flowchart showing a flow of image signal generation processing by the imaging system 10.

  First, the near infrared light 60 irradiated by the illumination unit 40 is absorbed and reflected by the subject 50. The imaging unit 20 captures light in the first wavelength band in the vicinity of a wavelength of 960 nm that is a water absorption peak in the reflected light 61 from the subject, and generates image data 62. Further, the imaging unit 20 images light in the second wavelength band near the wavelength of 850 nm, which is different from the water absorption peak, from the reflected light 61 reflected by the subject 50, and generates image data 63 (S101). Specifically, the plurality of pixels 25 convert the light in the first wavelength band transmitted by the optical filters 140 and 130 out of the reflected light 61 into an electrical signal. Further, the plurality of pixels 26 convert the light in the second wavelength band transmitted by the optical filters 140 and 131 out of the reflected light 61 into an electric signal.

  The electrical signals converted by the pixels 25 and 26 sequentially selected by the vertical scanning unit 22 and the horizontal scanning unit 23 are converted from analog signals to digital signals by the A / D conversion unit 24. The imaging unit 20 outputs image data 62 that is digital signals corresponding to the plurality of pixels 25 and image data 63 that is digital signals corresponding to the plurality of pixels 26 to the image generation unit 30.

  12 and 13 are diagrams showing examples of the image data 62 and 63, respectively.

  As shown in FIG. 12, in the image data 62, the light in the vicinity of the wavelength of 960 nm is absorbed by the water, so that the brightness of the region where the water exists is low. Further, as shown in FIG. 13, in the image data 63, light in the vicinity of a wavelength of 850 nm is not absorbed by water, so that the brightness of the region where water is present does not become lower than that of the image data 62.

  Further, in an area where no water exists, both the image data 62 and 63 have the same luminance. This is because the absorption spectrum of a normal substance (a substance that does not have a dispersion-type absorption spectrum) has a smaller change in absorption coefficient with respect to the wavelength than the dispersion-type absorption spectrum.

  Next, the pixel interpolation unit 31 interpolates the image data 62 and the image data 63 (S102). Hereinafter, pixel interpolation by the pixel interpolation unit 31 will be described.

  Since there is no pixel signal in the first wavelength band in the pixel 26, there is no pixel signal in the pixel position corresponding to the pixel 26 in the image data 62. The pixel interpolation unit 31 generates a pixel signal corresponding to the pixel position of the pixel 26 in the image data 62 from the pixel signal of the pixel 25 of the image data 62 adjacent to the pixel position, so that the pixel of the pixel 26 in the image data 62 Interpolate the pixel signal at the position.

  Similarly, the pixel interpolation unit 31 generates a pixel signal corresponding to the pixel position of the pixel 25 in the image data 63 from the pixel signal of the pixel 26 of the image data 63 adjacent to the pixel position, so that the pixel in the image data 63 Interpolate pixel signals at 25 pixel positions.

  Hereinafter, an example of interpolating the pixel signal at the pixel position of the pixel 204 of the image data 63 when the arrangement of the pixels 25 and 26 is the staggered pattern shown in FIG. 8 will be described.

  The pixel interpolation unit 31 uses the pixel signals in the second wavelength band of the pixels 202, 203, 206, and 211 to generate a pixel signal in the second wavelength band at the pixel position of the pixel 204. Specifically, the pixel interpolation unit 31 calculates an average value of the pixel signals of the four pixels 202, 203, 206, and 211. The pixel interpolation unit 31 uses the calculated average value as a pixel signal in the second wavelength band at the pixel position of the pixel 204.

  As another method, the pixel interpolation unit 31 compares the absolute value of the difference between the pixel signals of the pixel pair composed of the pixels 202 and 206 with the absolute value of the difference between the pixel signals of the pixel pair composed of the pixels 203 and 211. To do. The pixel interpolation unit 31 calculates an average value of pixel signals of two pixels included in a pixel pair having a small absolute value, and sets the calculated average value as a pixel signal in the second wavelength band at the pixel position of the pixel 204. The latter processing method has an advantage that the edge can be easily reproduced with respect to a steep luminance change generated at the edge of the subject as compared with the former processing method.

  Next, a pixel interpolation method in the case where the pixels 25 and 26 are arranged in stripes as shown in FIG. 9 or FIG. 10 will be described. Here, an example in which the pixel signal at the pixel position of the pixel 306 in the image data 62 is interpolated will be described.

  The pixel interpolation unit 31 calculates an average value of the six pixels 301, 302, 303, 309, 310, and 311, and uses the calculated average value as a pixel signal in the second wavelength band at the pixel position of the pixel 306.

  As another method, the pixel interpolation unit 31 calculates the absolute value of the difference between the pixel signals of the pixel pair including the pixel 301 and the pixel 311 and the absolute value of the difference between the pixel signals of the pixel pair including the pixel 303 and the pixel 309. Compare The pixel interpolation unit 31 calculates an average value of pixel signals of two pixels included in a pixel pair having a small absolute value, and sets the calculated average value as a pixel signal in the first wavelength band at the pixel position of the pixel 306.

  Next, the luminance correction unit 32 corrects the luminance of the image data 66 and 67 subjected to pixel interpolation by the pixel interpolation unit 31 (S103).

  FIG. 14 is a diagram showing the wavelength dependence of relative sensitivity to light in a photoelectric conversion element made of silicon. As shown in FIG. 14, sensitivity A at a wavelength of 960 nm is different from sensitivity B at a wavelength of 850 nm. The luminance correction unit 32 corrects the difference between the sensitivity A and the sensitivity B. That is, the luminance correction unit 32 corrects the image data 64 and 65 so that the pixel signals when the same amount of light is incident are the same.

  Specifically, when the photoelectric conversion element is made of silicon, the sensitivity A is 7% and the sensitivity B is 21%. For example, the brightness correction unit 32 performs brightness correction by multiplying the image data 64 by 3 times. Data 66 is generated. Further, the luminance correction unit 32 outputs the image data 65 as it is as the image data 67.

  Note that the wavelength dependence of relative sensitivity as shown in FIG. 14 also exists for photoelectric conversion elements made of materials other than silicon. Therefore, the luminance correction unit 32 amplifies at least one of the image data 64 and 65 so that the pixel signals when the same amount of light is incident are the same according to the wavelength dependence of the relative sensitivity of the photoelectric conversion element. Alternatively, it may be attenuated.

  Next, the subtraction unit 33 subtracts the image data 66 from the image data 67 to generate image data 68 that emphasizes water (S104).

  FIG. 15 is a diagram illustrating an example of the image data 68.

  As shown in FIG. 15, in the image data 68, an area where water is present is emphasized.

  FIG. 16 is a diagram illustrating the luminance of the image data 62, 63, and 68 with respect to the pixel position on the xy line in FIGS. 12, 13, and 15. In FIG.

  As shown in FIG. 16, in an area where water does not exist, that is, if the subject does not have a dispersion-type absorption spectrum, the image data 66 obtained by multiplying the image data 62 by 3 times and the image data 63 have substantially the same luminance. Therefore, the luminance of the image data 68 is almost zero. On the other hand, in the region where water is present, the image data 66 obtained by multiplying the image data 62 by three times has a lower luminance than the image data 63, and therefore the luminance is a value other than zero. Here, the luminance of the area where water exists in the image data 68 represents the luminance of the light absorbed in the water.

  As described above, the imaging system 10 according to Embodiment 1 of the present invention can generate the image data 68 that emphasizes the region where light is absorbed by the subject (water).

  Furthermore, since the imaging system 10 generates the image data 62 and the image data 63 obtained by imaging light in different wavelength bands by the imaging unit 20 formed on a single semiconductor substrate, compared to a case where a plurality of imaging elements are provided. In addition, the imaging system 10 and the imaging unit 20 can be reduced in size and cost. Further, the imaging system 10 can simultaneously generate the image data 62 and the image data 63 in one frame without requiring an operation such as moving the air gap in the small spectroscope. Therefore, the imaging system 10 can appropriately capture an image in which the subject is emphasized even for a moving subject.

  In the imaging system 10, the pixel interpolation unit 31 performs pixel interpolation of the image data 62 and 63. Thereby, the imaging system 10 can generate | occur | produce the highly accurate image which emphasized water.

  In the imaging system 10, the luminance correction unit 32 can correct a difference in sensitivity between photoelectric conversion elements in the first wavelength band and the second wavelength band, a difference in illumination intensity, and the like. Thereby, the imaging system 10 can generate | occur | produce the image which emphasized water more.

(Embodiment 2)
The imaging system according to the second embodiment of the present invention divides the two image data 62 and 63 generated by the imaging unit 20 to generate image data in which water is emphasized.

  First, the configuration of the imaging system according to Embodiment 2 of the present invention will be described.

  FIG. 17 is a block diagram showing a configuration of an imaging system according to Embodiment 2 of the present invention. Elements similar to those in FIG. 1 are denoted by the same reference numerals, and description thereof is omitted.

  The imaging system 11 shown in FIG. 17 differs from the imaging system 10 shown in FIG.

  The image generation unit 35 includes a pixel interpolation unit 31, a division unit 36, and a luminance correction unit 37.

  The pixel interpolation unit 31 is the same as the pixel interpolation unit 31 according to the first embodiment.

  The division unit 36 generates image data 69 by dividing the image data 65 by the image data 64.

  The brightness correction unit 37 amplifies the image data 69 with a predetermined gain to generate image data 70 in which water is emphasized.

  Next, the operation of the imaging system 11 will be described.

  FIG. 18 is a flowchart showing a flow of image signal generation processing by the imaging system 11. The operations in steps S101 and S102 shown in FIG. 18 are the same as those in the first embodiment, and a description thereof is omitted.

  After step S102, the dividing unit 36 generates image data 69 by dividing the image data 65 by the image data 64 (S203).

  Next, the brightness correction unit 37 amplifies the image data 69 with a predetermined gain, and corrects the brightness so that the image data 69 whose absolute value is reduced by the division can be appropriately displayed on the output device (S204). .

  FIG. 19 is a diagram illustrating an example of the image data 70.

  As shown in FIG. 19, in the image data 70, an area where water is present is emphasized.

  FIG. 20 is a diagram illustrating the luminance of the image data 62, 63, and 70 with respect to the pixel position on the xy line in FIGS. 12, 13, and 19. In FIG.

  As shown in FIG. 20, in a region where there is no water, that is, if the subject does not have a dispersion-type absorption spectrum, the value obtained by dividing the image data 63 by the image data 62 is relatively small. On the other hand, in a region where water exists, the value obtained by dividing the image data 63 by the image data 62 is relatively large. That is, the image data 70 is an image in which only an area where water exists is highlighted.

  As described above, the imaging system 11 according to Embodiment 2 of the present invention can generate the image data 70 in which the region where light is absorbed by the subject (water) is emphasized. Thereby, the imaging system 11 which concerns on Embodiment 2 of this invention can acquire the effect similar to the imaging system 10 which concerns on Embodiment 1 mentioned above.

  Although the imaging apparatus according to Embodiments 1 and 2 of the present invention has been described above, the present invention is not limited to this embodiment.

  For example, in the first and second embodiments, the example in which the present invention is applied to the imaging systems 10 and 11 that generate image data in which water is emphasized has been described. However, the present invention is not limited to a point sensor that detects water. It can also be applied to detection devices. That is, the imaging unit 20 may include only one pixel 25 and one pixel 26, respectively. In this case, the detection device according to the present invention may not include the pixel interpolation unit 31.

  Specifically, in the detection device according to the present invention, light in the first wavelength band is photoelectrically converted into a first signal by the pixel 25, and light in the second wavelength band is photoelectrically converted into a second signal by the pixel 26. The image generation unit 30 generates a signal indicating that water has been detected using the difference in luminance between the first signal and the second signal. The operation of the image generation unit 30 is the same as the operation of the image generation unit 30 or 35 described above except that pixel interpolation is not performed.

  In the first and second embodiments, the transmission characteristics of the optical filters 130 and 131 are shown in FIG. 4, but the transmission characteristics of the optical filters 130 and 131 are not limited to this.

  For example, in the above description, the center wavelength of the first wavelength band transmitted by the optical filter 130 is the peak wavelength 960 nm of the water absorption spectrum, but the first wavelength band may be a wavelength band including the peak wavelength 960 nm.

  The second wavelength band transmitted by the optical filter 131 may be a wavelength band that does not include the first wavelength band, that is, a wavelength band that does not include the peak wavelength 960 nm. Here, the first wavelength band and the second wavelength band are wavelength bands in which the transmittance is 50% or more in the transmission characteristics 502 and 501.

  In addition, it is preferable that the water absorption coefficient of the center wavelength of the second wavelength band is sufficiently smaller than the water absorption coefficient of the peak wavelength. For example, the water absorption coefficient at the center wavelength of the second wavelength band may be less than or equal to half the water absorption coefficient at the peak wavelength. More preferably, the water absorption coefficient at the center wavelength of the second wavelength band is preferably smaller by one digit or more than the water absorption coefficient at the peak wavelength.

  The second wavelength band is preferably a wavelength band close to the first wavelength band. Although the change is gentle compared to the peak in the dispersion-type absorption spectrum, the absorption rate of substances other than water also changes depending on the wavelength. Therefore, by making the first wavelength band and the second wavelength band closer, the difference in absorption rate between the first wavelength band and the second wavelength band of substances other than water can be reduced. Thereby, it can suppress that substances other than water are emphasized in the image data 68 or 70.

  Specifically, it is preferable that the first wavelength band and the second wavelength band are included in the wavelength band of light of the same color. Here, the first wavelength band and the second wavelength band are included in the wavelength band of light of the same color (for example, blue has a wavelength of 430 nm to 460 nm, green has a wavelength of 500 nm to 570 nm, and red has a wavelength of Not only when the first wavelength band and the second wavelength band are included in the wavelength band of 610 nm to 780 nm, etc.), but also near infrared light other than visible light (780 nm to 2500 nm), mid infrared light (2.5 μm) To 4 [mu] m) or far-infrared light (4 [mu] m to 1000 [mu] m), etc., including the case where the first wavelength band and the second wavelength band are included.

  In the above description, the peak wavelength of water is 960 nm, but another peak 703 or the like in the water absorption spectrum 701 may be used for the first wavelength band. Further, a peak (not shown) existing on the longer wavelength side than 960 nm may be used for the first wavelength band. As shown in FIG. 14, the longer the wavelength, the lower the relative output of the photoelectric conversion element. However, the relative output at the long wavelength can be improved by configuring the photoelectric conversion element with SiGe or the like.

  In the first and second embodiments, water has been described as an example of a substance having a dispersion-type absorption spectrum. However, the present invention can also be applied to substances having a dispersion-type absorption spectrum other than water. For example, the present invention may be applied to an imaging system that generates an image that emphasizes oil, hemoglobin, or a pathogen, or a detection device that detects oil, hemoglobin, a pathogen, or the like. In this case, the first wavelength band may be set to a wavelength band near the peak wavelength of the absorption spectrum of the substance to be emphasized or detected, and the second wavelength band may be set to a wavelength band near the wavelength different from the peak wavelength.

  FIG. 21 is a diagram showing an absorption spectrum of fat. FIG. 22 is a diagram showing an absorption spectrum of hemoglobin. As shown in FIGS. 21 and 22, the absorption peak of fat is 930 nm, the absorption peak of oxyhemoglobin is 580 nm, and the absorption peak of reduced hemoglobin is 550 nm.

  Further, the imaging systems 10 and 11 may irradiate the subject 50 with near-infrared light in a light-shielded container. In this case, since it is not affected by visible light, the optical filter 140 may not be provided.

  In the first and second embodiments, the imaging unit 20 is a CMOS image sensor, but may be a MOS image sensor or a CCD image sensor.

  In the description of the first embodiment, the image generation unit 30 performs the luminance correction (S103) after performing the pixel interpolation (S102). However, after performing the luminance correction (S103), the pixel interpolation is performed. (S102) may be performed.

  In FIG. 3, the optical filter 140 is formed above the microlenses 103 and 120. However, the optical filter 140 may be formed between the microlenses 103 and 120 and the optical filters 130 and 131. .

  The present invention can be applied to a solid-state imaging device and an imaging system, and in particular to an imaging device that detects an object having a dispersed absorption spectrum such as water, oil, hemoglobin, or a pathogen.

It is a block diagram which shows the structure of the imaging system which concerns on Embodiment 1 of this invention. It is a figure which shows the structure of the imaging part which concerns on Embodiment 1 of this invention. It is sectional drawing which shows the structure of the pixel which concerns on Embodiment 1 of this invention. It is a figure which shows the transmission characteristic of the optical filter which concerns on Embodiment 1 of this invention. It is a figure which shows the absorption spectrum of water. It is a figure which shows the absorption spectrum of water. It is a figure which shows the transmission characteristic when a band pass filter is inserted in the image sensor front surface in the optical filter which concerns on Embodiment 1 of this invention. It is a top view which shows the example of arrangement | positioning of the pixel in the pixel array which concerns on Embodiment 1 of this invention. It is a top view which shows the example of arrangement | positioning of the pixel in the pixel array which concerns on Embodiment 1 of this invention. It is a top view which shows the example of arrangement | positioning of the pixel in the pixel array which concerns on Embodiment 1 of this invention. It is a flowchart which shows the flow of the image generation process in the imaging system which concerns on Embodiment 1 of this invention. It is a figure which shows an example of the image data corresponding to wavelength 960nm which concerns on Embodiment 1 of this invention. It is a figure which shows an example of the image data corresponding to wavelength 850nm which concerns on Embodiment 1 of this invention. It is a figure which shows the wavelength dependence with respect to the relative sensitivity of a photoelectric conversion element. It is a figure which shows an example of the image data which emphasized the water produced | generated by the imaging system which concerns on Embodiment 1 of this invention. It is a figure which shows the brightness | luminance of each image data in the imaging system which concerns on Embodiment 1 of this invention. It is a block diagram which shows the structure of the imaging system which concerns on Embodiment 2 of this invention. It is a flowchart which shows the flow of the image generation process in the imaging system which concerns on Embodiment 2 of this invention. It is a figure which shows an example of the image data which emphasized the water produced | generated by the imaging system which concerns on Embodiment 2 of this invention. It is a figure which shows the brightness | luminance of each image data in the imaging system which concerns on Embodiment 2 of this invention. It is a figure which shows the absorption spectrum of fat. It is a figure which shows the absorption spectrum of hemoglobin.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10, 11 Imaging system 20 Imaging part 21 Pixel array 22 Vertical scanning part 23 Horizontal scanning part 24 A / D conversion part 25, 26 Pixel 30, 35 Image generation part 31 Pixel interpolation part 32, 37 Luminance correction part 33 Subtraction part 36 Division Section 40 Illumination section 50 Subject 60 Near infrared light 61 Reflected light 62, 63, 64, 65, 66, 67, 68, 69, 70 Image data 103, 120 Microlens 104, 106, 108, 110, 112, 114, 116 High refractive index film 105, 107, 109, 111, 113, 115, 122 Low refractive index film 117, 118, 123, 124, 125, 126 Wiring 119 Substrate 121 Planarization film 127, 128 Photodiode 129 Interlayer insulating film 130 131, 140 Optical filters 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416 pixels 217, 218, 219, 220, 317, 318, 319, 320, 417, 418, 419, 420 Pixel unit 501, 502, 601, 602 Transmission characteristics 701 Water absorption spectrum 702, 703 Peak 704 area

Claims (9)

A solid-state imaging device for generating an image in which a predetermined substance having a distributed absorption spectrum is emphasized,
First filtering means that transmits a first wavelength band including the peak of the dispersion type absorption spectrum;
Second filtering means that does not include the peak and transmits a second wavelength band that is the same color band as the first wavelength band;
A plurality of first photoelectric conversion means for converting light transmitted by the first filtering means into first image data;
A plurality of second photoelectric conversion means which are formed on the same semiconductor substrate as the first photoelectric conversion means and convert the light transmitted by the second filtering means into second image data;
A solid-state imaging device comprising: an image generation unit configured to generate an image in which the substance is emphasized using a difference in luminance between the first image data and the second image data. The image generating means includes
The solid-state imaging device according to claim 1, further comprising a subtracting unit that generates an image in which the substance is emphasized by subtracting the first image data from the second image data. The image generation means further includes:
Correction means for amplifying or attenuating at least one of the first image data and the second image data;
The subtracting unit generates an image in which the substance is emphasized by subtracting the first image data and the second image data amplified or attenuated by the correcting unit. Solid-state imaging device. The plurality of first photoelectric conversion means and the plurality of second photoelectric conversion means are alternately arranged in a row direction or a column direction,
The image generation means further includes:
By generating data corresponding to the pixel position of the second photoelectric conversion means in the first image data from the pixel data of the first image data adjacent to the pixel position, the second in the first image data. Data of the pixel position of the photoelectric conversion means is interpolated, and data corresponding to the pixel position of the first photoelectric conversion means in the second image data is generated from the pixel data of the second image data adjacent to the pixel position. In this way, the image processing apparatus includes pixel interpolation means for interpolating data of pixel positions of the first photoelectric conversion means in the second image data,
The subtracting unit generates an image in which the substance is emphasized by subtracting the first image data pixel-interpolated by the pixel interpolation unit from the second image data pixel-interpolated by the pixel interpolation unit. The solid-state imaging device according to claim 2 or 3. The image generating means includes
The solid-state imaging device according to claim 1, further comprising a dividing unit that generates an image in which the substance is emphasized by dividing the second image data by the first image data. The first filtering means and the second filtering means are respectively
A first multilayer film and a second multilayer film in which a high refractive index film and a low refractive index film are alternately laminated;
A third film laminated between the first multilayer film and the second multilayer film,
The high refractive index film and the low refractive index film included in the first filtering means have the same film thickness as the high refractive index film and the low refractive index film included in the second filtering means, respectively.
The said 3rd film | membrane contained in a said 1st filtration means differs in the said 3rd film | membrane contained in a said 2nd filtration means, The film thickness is any one of Claims 1-5 characterized by the above-mentioned. Solid-state imaging device. The solid-state imaging device further includes:
A third filtering means for transmitting light in a third wavelength band including the first wavelength band and the second wavelength band;
The first photoelectric conversion means converts the light transmitted by the third filtering means and transmitted by the first filtering means into the first image data;
The said 2nd photoelectric conversion means converts the light permeate | transmitted by the said 3rd filtration means and permeate | transmitted by the said 2nd filtration means to the said 2nd image data. The solid-state imaging device according to any one of the above. A solid-state imaging device according to any one of claims 1 to 7,
An imaging system comprising: illumination means for irradiating light in a fourth wavelength band including the first wavelength band and the second wavelength band. A detection device for detecting a predetermined substance having a distributed absorption spectrum,
First filtering means that transmits a first wavelength band including the peak of the dispersion type absorption spectrum;
A plurality of second filtering means that does not include the peak and transmits a second wavelength band that is the same color band as the first wavelength band;
First photoelectric conversion means for converting light transmitted by the first filtering means into a first signal;
Second photoelectric conversion means that is formed on the same semiconductor substrate as the first photoelectric conversion means and converts the light transmitted by the second filtering means into a second signal;
A detection apparatus comprising: a signal generation unit configured to generate a third signal indicating that the substance is detected using a difference in luminance between the first signal and the second signal.