WO2018207817A1

WO2018207817A1 - Solid-state image capture device, image capture system, and object identification system

Info

Publication number: WO2018207817A1
Application number: PCT/JP2018/017925
Authority: WO
Inventors: 光一朗石神; 角　博文; 基史祖父江
Original assignee: 株式会社ナノルクス
Priority date: 2017-05-11
Filing date: 2018-05-09
Publication date: 2018-11-15
Also published as: JP2019165447A; US20200128197A1; CN110574368B; JP6600800B2; JPWO2018207817A1; US10863116B2; CN110574368A

Abstract

Provided are a solid-state image capture device and an image capture system with which a color image can be captured with high sensitivity in a wide illuminance environment of from conventional illuminance to darkness (0 lux). The solid-state image capture device 1 is provided with: a first detection unit 11 in which a first visible light pixel for receiving first visible light and a first near-infrared pixel for receiving first near-infrared light are disposed adjacent to each other; a second detection unit 12 in which a second visible light pixel for receiving second visible light different in wavelength from the first visible light and a second near-infrared pixel for receiving second near-infrared light are disposed adjacent to each other; and a third detection unit 13 in which a third visible light pixel for receiving third visible light different in wavelength from the first visible light and the second visible light and a third near-infrared pixel for receiving third near-infrared light are disposed adjacent to each other.

Description

Solid-state imaging device, imaging system, and object identification system

The present invention relates to a solid-state imaging device and an imaging system for color image shooting. More specifically, the present invention relates to a technique for generating a color image by detecting near-infrared light having a specific wavelength.

There has been proposed an image capturing device that detects infrared light reflected by a subject or emitted from the subject and forms a color image of the subject (see Patent Document 1). The image capturing apparatus described in Patent Document 1 generates a color image from near-infrared light by using the same subject spectral reflectance characteristic as that in the visible wavelength region even in the near-infrared wavelength region. Yes. Specifically, light in the near infrared region having a high correlation with the color when the same subject is viewed under visible light is detected, and a pseudo display color is generated from the detection information. If this technique is used, a color image can be taken even in an extremely low illumination environment or in the dark.

On the other hand, in order to realize the technique described in Patent Document 1, three types of near-infrared light (NIR-R, -G) corresponding to red light (R), green light (G), and blue light (B) are used. , -B) must be detected. Conventionally, as a solid-state imaging device capable of detecting both visible light and near infrared light, for example, a pixel for detecting red light (R), green light (G), or blue light (B) In other words, pixels in which pixels for detecting near-infrared light (NIR) are arranged on the same substrate have been proposed (see, for example, Patent Documents 2 to 4).

Also, red light (R), green light (G) or blue light (B) and the corresponding three types of near-infrared light (NIR-R, -G, -B) are detected by the same pixel. A photodetection device has also been proposed (see Patent Documents 5 and 6). For example, in the photodetector described in Patent Document 5, an optical filter that transmits only red light (R) and the corresponding near infrared light (NIR-R), green light (G) and the corresponding near infrared light. Pixels having optical filters that transmit only light (NIR-G) and optical filters that transmit only blue light (B) and the corresponding near-infrared light (NIR-B) are periodically arranged. .

International Publication No. 2011-013765 International Publication No. 2007/086155 JP 2008-244246 A JP 2016-174028 A International Publication No. 2015/159651 International Publication No. 2016/158128

However, the conventional solid-state imaging device described above has the following problems. First, the near-infrared light detection pixels in the solid-state imaging devices described in Patent Documents 2 to 4 are configured not to provide an infrared filter, but to provide only an RGB color filter, thereby blocking the incidence of visible light. Therefore, these solid-state imaging devices cannot selectively detect near-infrared light having a specific wavelength. That is, with the solid-state imaging devices described in Patent Documents 2 to 4, it is difficult to capture a color image in an extremely low illumination environment or in the dark.

On the other hand, the light detection devices described in

Patent Documents

5 and 6 are assumed to be applied to the technique described in Patent Document 1, but transmit only a specific wavelength in the visible region and a specific wavelength in the near-infrared region. Other optical filters that reflect light are difficult to design. In addition, when such an optical filter has a structure in which a high refractive layer and a low refractive layer are laminated as described in Patent Document 5, high-precision film thickness control is required, and the manufacturing process is complicated. Become. For this reason, the photodetectors described in

Patent Documents

5 and 6 are required to be further improved in terms of manufacturing cost and manufacturing process.

Therefore, an object of the present invention is to provide a solid-state imaging device and an imaging system capable of capturing a color image with high sensitivity in a wide range of illuminance environments from normal illuminance to darkness (0 lux).

In the solid-state imaging device according to the present invention, the first visible light pixel that receives the first visible light and the first near-infrared pixel that receives the first near-infrared light are provided adjacent to each other. A first detection unit, a second visible light pixel that receives second visible light having a wavelength different from that of the first visible light, and a second near infrared pixel that receives second near infrared light are mutually connected. A second detection unit provided adjacent to the first visible light, a third visible light pixel that receives the first visible light and a third visible light having a wavelength different from that of the second visible light, A third detection unit provided with a third near-infrared pixel that receives infrared light is provided adjacent to each other.
In this solid-state imaging device, the first near-infrared pixel also receives the first visible light, the second near-infrared pixel also receives the second visible light, and the third near-infrared pixel The third visible light may also be received.
For example, the first visible light, the second visible light, and the third visible light are red light, green light, and blue light, respectively.
In the solid-state imaging device according to the present invention, the first near-infrared light is light in a near-infrared region correlated with the first visible light, and the second near-infrared light is the second visible light. The near-infrared light having a correlation with the light may be used, and the third near-infrared light may be the near-infrared light having a correlation with the third visible light.
Alternatively, the solid-state imaging device of the present invention can receive two or three types of near infrared light having different wavelengths by the first to third near infrared pixels. That is, two of the first near-infrared light, the second near-infrared light, and the third near-infrared light may be light having the same wavelength.
Alternatively, in the solid-state imaging device of the present invention, in the second near-infrared pixel and / or the third near-infrared pixel, the light in the near-infrared region correlated with the first visible light, the second It is also possible to receive light in a band including two or more of light in a near infrared region correlated with visible light and light in a near infrared region correlated with the third visible light.
In the solid-state imaging device of the present invention, the first detection unit, the second detection unit, and the third detection unit may be provided on the same element.
Alternatively, in the solid-state imaging device of the present invention, the first to third detection units are provided in different elements, and the light from the subject is dispersed and emitted toward the first to third detection units. It can also be set as the structure which has a spectroscopic element.
Furthermore, the solid-state imaging device of the present invention may be provided with an image generation unit that generates a color image using the signals acquired by the first to third detection units.
In the image generation unit, for example, the visible light component detected by the first to third visible light pixels and the near infrared light component detected by the first to third near infrared pixels are combined at an arbitrary ratio, Generate a color image.

An imaging system according to the present invention includes the solid-state imaging device described above and a light irradiation device that irradiates a subject with near infrared light.
In this imaging system, the solid-state imaging device includes an illumination light control unit that controls the amount of near-infrared light emitted from the light irradiation device based on detection results of visible light and near-infrared light at each detection unit. May be provided.

An object identification system according to the present invention includes a light source that is attached to a target object and emits near-infrared light having a specific wavelength, a photodetector that detects near-infrared light emitted from the light source, and the photodetector. An object determination unit that determines whether or not the target object exists in a detection region from the detection result of the detection, and an object identification unit that identifies the target object from wavelength information of near-infrared light detected by the photodetector A data processing device.
In that case, the above-described solid-state imaging device is used as the photodetector, and the object identification unit of the data processing device is applied to the target object from the wavelength of near-infrared light detected by the photodetector. Color information may be acquired and the target object may be identified from the color information.
The object identification system further includes an imaging device that captures a visible light image of the detection target region. The visible light image captured by the imaging device is acquired by the object identification unit in the data processing device. An image composition unit for adding color information of the target object or arbitrary information associated with the color information may be provided.
The object identification system of the present invention may include a display device that displays a processing result in the data processing device.

According to the present invention, high-sensitivity shooting is possible in a wide range of illuminance environments from normal illuminance to darkness (0 lux), and a color image can be obtained even under low illuminance.

It is a conceptual diagram which shows the basic composition of the solid-state imaging device of the 1st Embodiment of this invention. A is a diagram illustrating a pixel arrangement example of a solid-state imaging device including detection units 11 to 13, and B is a diagram illustrating spectral characteristics of an optical filter provided on each pixel of A. FIG. A is a diagram illustrating another pixel arrangement example of the solid-state imaging device including the detection units 11 to 13, and B is a diagram illustrating spectral characteristics of an optical filter provided on each pixel of A. FIG. A is a diagram illustrating another pixel arrangement example of the solid-state imaging device including the detection units 11 to 13, and B is a diagram illustrating spectral characteristics of an optical filter provided on each pixel of A. FIG. FIGS. 8A to 8C are diagrams illustrating other pixel arrangement examples of the solid-state imaging device including the detection units 11 to 13; FIGS. It is a figure which shows the detection component obtained when the solid-state image sensor of the pixel arrangement | sequence shown in FIG. 2 is used. It is a figure which shows the detection component obtained when the solid-state image sensor of the pixel arrangement | sequence shown in FIG. 3 is used. It is a figure which shows the detection component obtained when the solid-state image sensor of the pixel arrangement | sequence shown in FIG. 4 is used. It is a figure which shows typically the structure of the solid-state imaging device of the 1st modification of the 1st Embodiment of this invention. FIGS. 8A to 8C are diagrams illustrating pixel arrangement examples in the

detection units

21, 22, and 23, respectively. It is a conceptual diagram which shows the solid-state imaging device of the 2nd Embodiment of this invention. It is a figure which shows the structural example of the image generation part 35 shown in FIG. It is a conceptual diagram which shows the imaging system of the 3rd Embodiment of this invention. It is a figure which shows typically the structure of the object identification system which concerns on the 4th Embodiment of this invention. It is a figure which shows the pixel arrangement example of the solid-state image sensor which concerns on the 5th Embodiment of this invention. It is a figure which shows the other pixel arrangement example of the solid-state image sensor which concerns on the 5th Embodiment of this invention.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments described below.

(First embodiment)
First, the solid-state imaging device according to the first embodiment of the present invention will be described. In the solid-state imaging device of the present embodiment, a pixel that receives visible light (visible light pixel) and a pixel that receives near-infrared light (near-infrared pixel) are provided adjacent to each other, and a wavelength range to be detected Have three or more types of detectors. FIG. 1 is a conceptual diagram showing a basic configuration of the solid-state imaging device of the present embodiment.

As shown in FIG. 1, the solid-state imaging device 1 according to this embodiment includes a first detection unit 11, a second detection unit 12, and a third detection unit 13 that detect light reflected from a subject or light emitted from the subject. It has. These detection units 11 to 13 detect visible light and near-infrared light having specific wavelengths, for example, signals S ₁ , S ₂ and S ₃ based on visible light, and signals S _IR1 and S _IR2 based on near-infrared light. S _IR3 is output.

[Visible light pixel]
The first visible light pixel provided in the first detection unit 11, the second visible light pixel provided in the second detection unit 12, and the third visible light pixel provided in the third detection unit 13 are different from each other. Receive visible light of wavelength. For example, when the first visible light pixel receives red light R, the second visible light pixel receives green light G, and the third visible light pixel receives blue light B.

In that case, each visible light pixel of the detectors 11 to 13 has a red light filter that reflects and / or absorbs visible light other than the red light R on a photoelectric conversion layer that detects incident light as an electrical signal, and green. A green light filter that reflects and / or absorbs visible light other than the light G and a blue light filter that reflects and / or absorbs visible light other than the blue light B may be provided.

The photoelectric conversion layer is a substrate in which a plurality of photoelectric conversion units are formed on a substrate such as silicon, and the red light filter, the green light filter, and the blue light filter are respectively formed on the corresponding photoelectric conversion units. The structure of the photoelectric conversion layer is not particularly limited, and a CCD (Charge-Coupled Device), a CMOS (Complementary Metal-Oxide Semiconductor) structure, or the like can be employed.

The transmission wavelength of each color filter provided on the photoelectric conversion layer is not limited to the above-described red light R, green light G, and blue light B, and is appropriately selected according to the specifications of the solid-state imaging device. be able to. Further, the material for forming each color filter is not particularly limited, and a known material can be used.

[Near infrared pixel]
On the other hand, a first near-infrared pixel provided in the first detector 11, a second near-infrared pixel provided in the second detector 12, and a third near-infrared provided in the third detector 13 The pixel detects light having two or more wavelengths. That is, the near-infrared pixels of the detectors 11 to 13 may receive near-infrared light of different wavelengths, but the near-infrared pixels of different detectors receive near-infrared light of the same wavelength. Also good.

When each detector receives near-infrared light having a different wavelength, the first to third near-infrared pixels are correlated with the visible light received by the first to third visible light pixels described above, respectively. Light in the outer region can be received. For example, when the first to third visible light pixels receive red light (R), green light (G), and blue light (B), the first to third near-infrared pixels are correlated with RGB. Receives near-infrared light.

Each of the near-infrared pixels of the detection units 11 to 13 can be configured such that an optical filter that selectively transmits near-infrared light having a specific wavelength, such as an interference filter, is provided on the photoelectric conversion layer. When the first to third near-infrared pixels receive near-infrared light correlated with RGB, for example, the first near-infrared pixel has a transmittance of near-infrared light having a wavelength longer than 800 nm of 50 nm. % Short pass filter, a band pass filter having a center wavelength of 850 nm is provided in the second near infrared pixel, and a transmittance of near infrared light having a wavelength shorter than 890 nm is provided in the third near infrared pixel. What is necessary is just to set it as the structure which provided the 50% or less long pass filter.

Further, for example, the first near-infrared pixel receives light in the near-infrared region correlated with visible light received by the first visible light pixel, and the second near-infrared pixel and the third near-infrared pixel are The near-infrared region light correlated with the visible light received by the third visible light pixel, the near-infrared region light correlated with the visible light received by the second visible light pixel, or both It can also receive broadband light including.

Furthermore, in the solid-state imaging device 1 of the present embodiment, the first to third near-infrared pixels may be configured to receive only near-infrared light, but are configured to receive visible light together with near-infrared light. You can also. Accordingly, it is possible to realize color photographing using near infrared light without securing sensitivity during photographing using visible light and reducing the S / N (Signal / Noise) ratio.

[Configuration example]
In the solid-state imaging device 1 of the present embodiment, each of the detection units 11 to 13 can be formed on one solid-state imaging device. 2A, 3A, and 4A are pixel arrangement examples of the solid-state imaging device including the detection units 11 to 13, and FIGS. 2B, 3B, and 4B are diagrams illustrating the spectral characteristics of the optical filter provided on each pixel. FIGS. 6 to 8 are diagrams showing detection components obtained when the solid-state imaging device having the pixel arrangement shown in FIGS. 2 to 4 is used.

When the three types of detection units 11 to 13 are provided on the same element, for example, a pixel arrangement having a repeating unit shown in FIGS. 2A, 3A, and 4A can be employed. In the solid-state imaging device having the pixel arrangement shown in FIG. 2A, as shown in FIG. 2B, the first detector 11 detects the near-infrared light IRR correlated with the red light R and the red light. Further, the second detection unit 12 detects the green light G and the near infrared light IRG correlated with the green light. Further, the third detector 13 detects the blue light B and the near-infrared light IRB that is correlated with the blue light.

Here, for example, the near infrared light IRR is light having an arbitrary wavelength in the range of 700 to 830 nm, and the near infrared light IRG is light having an arbitrary wavelength in the range of 880 to 1200 nm. Is light of an arbitrary wavelength in the range of 830 to 880 nm, each having a different wavelength.

In the case of this pixel arrangement, red light R, green light G, and blue light B are directly detected by visible light pixels for visible light, but both visible light and near infrared light are detected by near-infrared pixels. Therefore, as shown in FIG. 6, by removing the components detected by the visible light pixels from the components detected by the near-infrared pixels, the near-infrared light IRR, IRG, IRB correlated with RGB. Can be extracted.

As described above, since the pixel arrangement shown in FIG. 2A can detect three wavelength components for visible light and three wavelength components for near-infrared light, color reproduction can be achieved even under extremely low illuminance or in darkness (0 lux). A color image with excellent properties can be obtained. In addition, since the solid-state imaging device having this pixel arrangement is detected by different pixels for each wavelength, the design of each pixel is easy and the film configuration can be simplified, so that it can be manufactured more easily than conventional products. Become.

On the other hand, FIGS. 3A and 4A are examples of pixel arrangements with an emphasis on sensitivity. In the pixel arrangement shown in FIG. 3A, as shown in FIG. 3B, the first detection unit 11 detects the red light R and the near-infrared light IRR correlated with the red light. Further, the second detection unit 12 detects the green light G and the near infrared light IRG correlated with the green light. Further, the third detector 13 detects near infrared light IRG that is correlated with the blue light B and the green light.

In the pixel arrangement shown in FIG. 4A, as shown in FIG. 4B, the first detection unit 11 detects the red light R and the near infrared light IRR correlated with the red light. Further, the second detection unit 12 detects a broadband light IRW including the near-infrared light IRR correlated with the green light G and the red light to the near-infrared light IRG correlated with the green light. Further, the third detector 13 detects near infrared light IRG that is correlated with the blue light B and the green light.

In the solid-state imaging device having the pixel arrangement shown in FIGS. 3 and 4, as shown in FIGS. 7 and 8, the near-infrared component includes the near-infrared light IRR correlated with the red light R, and the green light G. Only the near-infrared light IRG having a correlation is separated and extracted, and the near-infrared light IRB having a correlation with only the blue light B is not detected. Thereby, since the selection range of the detection wavelength of each near-infrared pixel is expanded, the sensitivity of color photographing under low illuminance can be improved.

2 to 4 show an example in which one detection unit is configured by four pixels. However, the present invention is not limited to this, and a visible light pixel and a near-infrared pixel are adjacent to each other. The number of pixels constituting each detection unit is not limited as long as it is arranged. 5A to 5C are diagrams illustrating other pixel arrangement examples of the solid-state imaging device including the detection units 11 to 13. FIG. Specifically, as in the pixel arrangement having the repeating unit shown in FIGS. 5A and 5B, one detection unit can be configured by two pixels. In that case, an arrangement of visible light pixels and near infrared pixels is arranged. Can be either vertical or horizontal.

Also, when one detection unit is configured with four pixels, it is only necessary that the visible light pixel and the near-infrared pixel are arranged adjacent to each other in the detection unit, as shown in FIG. 5C. The visible light pixel and the near-infrared pixel do not have to be adjacently disposed between the pixels of the detection unit. That is, in the solid-state imaging device according to the present embodiment, the visible light pixel that receives the red light R provided in the detection unit 11 and the green light G that is provided in the detection unit 12, as in the pixel arrangement illustrated in FIG. 5C. Visible light pixels that receive light may be arranged adjacent to each other.

1 to 4 show an example in which three detectors 11 to 13 are provided. However, the present invention is not limited to this, and a visible light pixel and a near-infrared pixel are adjacent to each other. It is also possible to provide four or more types of detectors that are arranged in such a manner that detection wavelengths at least in the visible light region are different from each other.

The solid-state imaging device of this embodiment has three or more types of detection units in which visible light pixels and near-infrared pixels are arranged adjacent to each other, and has three or more wavelengths of visible light and two or more wavelengths of near red. Since external light is detected, a color image can be generated using visible light, near infrared light, or both detected by each detection unit. For this reason, by using the solid-state imaging device of the present embodiment, it is possible to capture a color image in a wide range of illuminance environments from normal illuminance to darkness (0 lux). In particular, in environments where the amount of visible light is significantly reduced, such as at night without artificial lighting, color images are generated using both visible light and near-infrared light, resulting in less noise and color. A color image with excellent reproducibility can be obtained.

In the solid-state imaging device of the present embodiment, near-infrared light is detected even at normal illuminance, and correction can be performed using the detection signal, so there is no need to provide a separate infrared cut filter. Similarly, when a color image is generated using a signal from a near-infrared pixel, correction can be performed using a visible light component and color adjustment can be performed. Therefore, a separate visible light cut filter (infrared path) is provided. There is no need to provide it. Thereby, the apparatus configuration can be simplified.

(First modification of the first embodiment)
Next, a solid-state imaging device according to a first modification of the first embodiment of the present invention will be described. The solid-state imaging device of the first embodiment described above employs a configuration in which three detection units are formed on the same pixel. However, the present invention is not limited to this, and each detection unit includes Each may be formed in a different solid-state image sensor.

FIG. 9 is a diagram schematically showing the configuration of the solid-state imaging device of this modification, and FIGS. 10A to 10C are diagrams showing examples of pixel arrangement in the

detection units

21, 22, and 23, respectively. As illustrated in FIG. 9, the solid-state imaging device 2 according to the present modification includes

detection units

21, 22, and 23 that include visible light pixels that receive visible light and near-infrared pixels that receive near-infrared light. Each is provided in an individual solid-state imaging device. The solid-state imaging device 2 is provided with a spectroscopic element 24 that splits light (visible light / near infrared light) from a subject for each specific wavelength and emits the light toward the

detection units

21, 22, and 23. ing.

[Detection units 21 to 23]
The detection units 21 to 23 of the solid-state imaging device 2 of the present modification can be arranged in a pixel arrangement having a repeating unit as shown in FIGS. 10A to 10C, for example. In this case, the first detector 21 detects the near-infrared light IRR that is correlated with the red light R and the red light. The second detector 22 detects green light G and near-infrared light IRG correlated with green light, and the third detector 13 detects near-infrared correlated with blue light B and blue light. Optical IRB is detected.

In the solid-state imaging device 2 of the present modification, since light dispersed by the spectroscopic element 24 for each specific wavelength is incident on each pixel, it is not necessary to provide a color filter for each pixel of the detection units 21 to 23. The visible light pixels R, G, and B may be provided with an infrared cut filter. Further, the pixel arrangement of each of the detection units 21 to 23 is not limited to that shown in FIGS. 10A to 10C, and the visible light pixel and the near-infrared pixel are adjacent to each other as in the first embodiment described above. If arranged, the number of repeating units and the number of pixels per unit are not particularly limited.

[Spectroscopic element 24]
As the spectroscopic element 24, for example, a dichroic prism or the like can be used. However, the type and characteristics are not particularly limited as long as the light to be detected can be split.

Since the solid-state imaging device of this modification has a solid-state imaging device for each detection unit, the pixel configuration is simpler than the configuration in which a plurality of detection units are provided on the same element shown in FIGS. The manufacturing cost of the solid-state imaging device can be reduced. Moreover, in the solid-state imaging device of this modification, since the light dispersed from one light is detected by the corresponding pixels of the three detection units, it is possible to detect three types of light having different wavelengths per region, Furthermore, since the density between pixels of the same wavelength is increased, higher resolution can be ensured. The configuration and effects other than those described above in the solid-state imaging device of the present modification are the same as those in the first embodiment described above.

(Second Embodiment)
Next, a solid-state imaging device according to the second embodiment of the present invention will be described. FIG. 11 is a conceptual diagram showing the solid-state imaging device of the present embodiment. In FIG. 11, the same components as those in the solid-state imaging device shown in FIG. 1 are denoted by the same reference numerals, and detailed description thereof is omitted. As shown in FIG. 11, the solid-state imaging device 3 of the present embodiment is provided with an image generation unit 31 in addition to the first to

third detection units

11, 12, and 13.

[Image generation unit 31]
FIG. 12 is a diagram illustrating a configuration example of the image generation unit 31. The image generation unit 31 performs color based on the visible light signals S ₁ , S ₂ , S ₃ and the near-infrared light signals S _IR1 , S _IR2 , S _IR3 output from the first to

third detection units

11, 12, 13. An image is generated. For example, in the “day mode”, the image generation unit 35 generates a color image using only the visible light signals S ₁ , S ₂ and S ₃ from the visible light pixels, and in the “night mode”, the near red color is generated. using only near infrared signal _{_S _IR1,} _S _IR2, _S _IR3 from the outer pixels to generate a color image.

The image generation unit 31 can also generate a color image using both the visible light signals S ₁ , S ₂ , S ₃ and the near-infrared signals S _IR1 , S _IR2 , S _IR3 . When combining the visible light signal and the near-infrared signal, the image generating unit 31 determines the combination ratio from the ratio of the visible light component and the near-infrared light component, for example, as shown in FIG. It can be.

For example, when determining one composition ratio for all pixel positions, an integral value of a certain pixel range can be used. At that time, the target pixel range is preferably a position within the effective region that is important at the time of observation, but when it is difficult to provide a detection unit of each response level for the effective region, as a simple alternative method, A small detection-dedicated region may be provided at the end portion outside the effective pixel, and determination may be made from the detection result of only that region. Further, when combining at different ratios for each pixel region, it is possible to use the response of adjacent pixels or use the result of locally performing the filtering process on the response of surrounding pixels. The two types of pairs to be compared are not limited to one set of information but may be a plurality of sets, or may be used after being converted into one representative component ratio information by an arbitrary weighted average or the like.

In addition, as the composition ratio of the visible light component and the near infrared component, for example, the calculated component ratio may be applied as it is, but may be intentionally modulated. As modulation methods, for example, “only component ratio information is applied non-linearly”, “near infrared component ratio is not given unless near infrared light component ratio exceeds a certain level”, “combined with other information "Additional condition setting", "The near-infrared composite ratio is given as the incident light quantity determined by the auto exposure mechanism with only the visible light component is small, or as the gain value (for exposure correction) is large." Is mentioned.

Since the solid-state imaging device of the present embodiment detects visible light having three or more wavelengths and near-infrared light having two or more wavelengths, the solid-state imaging device can capture images in a wide illuminance environment from normal illuminance to darkness (0 lux). And a high-resolution color image can be generated. In addition, since the solid-state imaging device of the present embodiment can arbitrarily set the composition ratio of the visible light component and the near infrared component, it can be set optimally according to the usage environment, etc. Is excellent.

Note that the image generation unit may not be built in the solid-state imaging device, and for example, the above-described processes can be executed by a program installed in a computer. In addition, the configuration and effects other than those described above in the solid-state imaging device of the present embodiment are the same as those of the first embodiment and the modifications thereof described above.

(Third embodiment)
Next, an imaging system according to a third embodiment of the present invention will be described. FIG. 13 is a diagram illustrating a configuration example of the imaging system of the present embodiment. As shown in FIG. 13, the imaging system of this embodiment includes a solid-state imaging device 4 and a light irradiation device 5 that irradiates a subject 6 with near infrared light that is illumination light.

In the solid-state imaging device 4 of this imaging system, the illumination light (near-infrared light) emitted from the light irradiation device 5 is based on the detection results of visible light and near-infrared light in the

detection units

11, 12, and 13. An illumination light control unit 41 that controls the amount of light is provided. In this imaging system, for example, incident light can be preferentially optimized, and the composition ratio can be determined according to the result.

For example, when the image generation unit 31 determines that the signal level of visible light is low, the illumination light control unit 41 emits near-infrared light, or the signal level of visible light and the signal level of near-infrared light are The amount of irradiation light is modulated so as to satisfy a certain standard. In addition, when the light irradiation device 5 can manipulate the light amount for each of a plurality of types of light sources having different wavelength regions, the spectral distribution can be optimized.

In the imaging system of the present embodiment, the light irradiation device 5 that irradiates the subject with illumination light (near infrared light) is controlled based on the visible light signal and the near infrared signal output from the detection unit. Color photography can be performed under optimum conditions. The imaging system of the present embodiment can keep the signal at the imaging stage before the signal processing more appropriate as compared with the case where the image is taken without controlling the light irradiation and the correction is performed only by the signal processing later. Therefore, for example, the range in which the color reproducibility and the S / N ratio are compatible and the balance of the image quality can be maintained is expanded. Thereby, color photography can be stably performed in a wide range of illuminance environments up to darkness (0 lux).

The configuration and effects other than those described above in the imaging system of the present embodiment are the same as those of the first embodiment, its modification, and the second embodiment described above.

(Fourth embodiment)
Next, an object identification system according to a fourth embodiment of the present invention will be described. FIG. 14 is a diagram schematically showing the configuration of the object identification system of this embodiment. As shown in FIG. 14, the object identification system of the present embodiment is attached to a target object 50 and emits near-infrared light having a specific wavelength, and near-infrared light emitted from the light sources 52a-52c. And a data processing device (not shown) including an object determination unit and an object identification unit.

In the object identification device according to the present embodiment, a plurality of target objects 50 including any one of the light sources 52a to 52c are imaged by the photodetector 53, the movement trajectory is recorded, and the near light emitted from the light sources 52a to 52c is recorded. The target object 50 is identified by the wavelength of infrared light. Note that FIG. 7 also shows objects (such as people and trees) other than the target object 50.

[Light sources 52a to 52c]
The light sources 52a to 52c are light-emitting elements that emit near-infrared light having mutually different wavelengths. For example, infrared light-emitting diodes can be used. Instead of the light emitting element, a material that reflects only near-infrared light having a specific wavelength or a paint containing such a material may be used as the light source.

The light sources 52a to 52c attached to the target object 50 need not all have different emission wavelengths. For example, light sources that emit the same wavelength may be attached to a plurality of target objects 250, and identification or detection of a movement locus may be performed for each group. Further, the emission wavelength of the light source is not limited to three types, and can be set as appropriate according to the number of target objects 50 or the number of target object groups. For example, one target object 50 or one group of target objects 50 can be set. In this case, one type of light-emitting element that emits near-infrared light can be used.

[Photodetector 53]
The photodetector 53 is not limited as long as it can detect and record near-infrared light emitted from the light sources 22a to 22c for each wavelength. For example, the solid-state imaging device of the first and second embodiments described above can be used. Can be used. When the

light sources

52a, 52b, and 52c are infrared light emitting diodes that emit near-infrared light NIR-R, -G, and -B, respectively, the photodetector 53 has red light (R) and green light in visible light. As in the case of light (G) and blue light (B), it is possible to use a device provided with an imaging device capable of distinguishing and detecting near-infrared light NIR-R, -G, -B.

The light detector 53 may be capable of detecting at least near-infrared light. For example, a near-infrared image and a visible image may be captured at the same time, and the respective images may be superimposed or recorded or displayed. In that case, although both a near-infrared image and a visible image may be imaged with one imaging device, you may image with another device, respectively.

[Data processing device]
The data processing apparatus includes an object determination unit that determines whether or not the target object 50 exists in the detection region from the detection result of the light detector 53, and the wavelength of the near-infrared light detected by the light detector 53. An object identification unit for identifying the target object 50 from the information is provided. For example, the object identification unit acquires color information given to the target object from the wavelength of near infrared light detected by the photodetector, and identifies the target object from the color information.

In addition, when the image processing apparatus further includes an imaging device that captures a visible light image of the detection target region, the data processing device adds the color information of the target object acquired by the object identification unit to the visible light image captured by the imaging device. An image composition unit for adding arbitrary information associated with the color information may be provided. Here, the information associated in advance with the color information includes the name and characteristics of the target object, but is not limited thereto, and can be set as appropriate according to the purpose.

The processing result in this data processing device is displayed on a display device provided separately, for example. Specifically, the position and movement locus of the target object 50 are displayed in a predetermined color on a still image or moving image captured with visible light, or a symbol or an illustration showing the target object 50 is displayed.

Since the object identification system according to the present embodiment identifies a target object using near-infrared light, an object other than the target object is detected, for example, when monitoring the movement of a person in a crowd. Even when there are a large number, it is possible to easily and reliably specify the presence or absence and position of the target object and acquire the movement trajectory. Further, since the near infrared light used in the object identification system of the present embodiment is not visible to humans, it can be used without giving a sense of incongruity even when used for humans.

(Fifth embodiment)
Next, a solid-state image sensor according to a fifth embodiment of the present invention will be described. 15 and 16 are diagrams showing an example of pixel arrangement of the solid-state imaging device of this embodiment. The solid-state imaging device of the present invention can also use a solid-state imaging device having a pixel arrangement having a repeating unit shown in FIGS. 15 and 16. Like the solid-state imaging device shown in FIGS. 15 and 16, it is possible to increase the luminance resolution by densely arranging the pixels that detect the green light G and the near infrared light IRG correlated with the green light. Become.

1 to 4 Solid-state imaging device 5 Light irradiation device 6 Subject 11 to 13, 21 to 23 Detection unit 24 Spectroscopic element 31 Image generation unit 41 Irradiation control unit 50 Target object 52a to 52c Light source 53 Photo detector

Claims

A first detection unit in which a first visible light pixel that receives first visible light and a first near-infrared pixel that receives first near-infrared light are provided adjacent to each other;
A second visible light pixel that receives second visible light having a wavelength different from that of the first visible light and a second near infrared pixel that receives second near infrared light are provided adjacent to each other. A second detection unit,
And a third visible light pixel that receives third visible light having a wavelength different from that of the first visible light and the second visible light, and a third near infrared pixel that receives third near infrared light. A solid-state imaging device having a third detector provided adjacent to each other.
The first near-infrared pixel also receives the first visible light,
The second near-infrared pixel also receives the second visible light,
The solid-state imaging device according to claim 1, wherein the third near-infrared pixel also receives the third visible light.
The solid-state imaging device according to claim 1 or 2, wherein the first visible light, the second visible light, and the third visible light are red light, green light, and blue light, respectively.
The first near-infrared light is light in a near-infrared region correlated with the first visible light;
The second near-infrared light is light in a near-infrared region that is correlated with the second visible light,
The solid-state imaging device according to any one of claims 1 to 3, wherein the third near-infrared light is light in a near-infrared region having a correlation with the third visible light.
The solid-state imaging device according to any one of claims 1 to 3, wherein two or three types of near-infrared light having different wavelengths are received by the first to third near-infrared pixels.
The second near-infrared pixel and / or the third near-infrared pixel are light in a near-infrared region correlated with the first visible light, and near-infrared correlated with the second visible light. The solid-state imaging device according to any one of claims 1 to 3, which receives light in a band including two or more of light in a region and near-infrared light having a correlation with the third visible light.
The solid-state imaging device according to any one of claims 1 to 6, wherein the first detection unit, the second detection unit, and the third detection unit are provided on the same element.
The first to third detection units are provided in mutually different elements,
7. The solid-state imaging device according to claim 1, further comprising a spectroscopic element that divides light from a subject and emits the light toward the first to third detection units.
9. The solid-state imaging device according to claim 1, further comprising an image generation unit that generates a color image using the signals acquired by the first to third detection units.
The image generation unit synthesizes the visible light component detected by the first to third visible light pixels and the near infrared light component detected by the first to third near-infrared pixels at an arbitrary ratio to generate a color The solid-state imaging device according to claim 9, which generates an image.
A solid-state imaging device according to any one of claims 1 to 10,
An imaging system comprising a light irradiation device that irradiates a subject with near infrared light.
The solid-state imaging device is provided with an illumination light control unit that controls the amount of near infrared light emitted from the light irradiation device based on the detection result of visible light and near infrared light in each detection unit. The imaging system according to claim 11.
A light source attached to a target object and emitting near-infrared light of a specific wavelength;
A photodetector for detecting near-infrared light emitted from the light source;
Identifying the target object from the wavelength information of the near-infrared light detected by the object determination unit that determines whether or not the target object exists in the detection region from the detection result of the photodetector and the photodetector An object identification system having a data processing device including an object identification unit.
The light detector is the solid-state imaging device according to any one of claims 1 to 8,
The said object identification part acquires the color information provided to the said target object from the wavelength of the near infrared light detected with the said photodetector, and identifies the said target object from the said color information. Object identification system.
An image pickup device for picking up a visible light image of the detection target region;
Image synthesis in which the data processing device adds color information of the target object acquired by the object identification unit or arbitrary information associated with the color information to a visible light image captured by the imaging device The object identification system according to claim 14, further comprising a unit.