JP2009049525A - Imaging apparatus and method for processing signal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus which can reproduce an object image faithfully. <P>SOLUTION: A digital camera comprising a solid state image sensor 200 having a large number of pixels 202R, G and B each including a photoelectric conversion film 226 formed on a semiconductor substrate 201 to absorb light in a specific wavelength zone and to generate corresponding charges, and a photoelectric conversion element 214 formed in the semiconductor substrate 201 below the photoelectric conversion film 226 is further provided with a means for determining exposure conditions of the photoelectric conversion element 214 and a means for regulating a voltage applied to the photoelectric conversion film 226 so that the signal from the photoelectric conversion film 226 included in each pixel does not exceed a saturation level when imaging is performed under the exposure conditions determined by the exposure condition determination means, and imaging is performed based on the exposure conditions in a state that a voltage regulated by the applying voltage regulation means is applied to the photoelectric conversion film 226. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging apparatus including a solid-state imaging device having a large number of pixels.

In recent years, various imaging devices capable of wide dynamic range imaging have been proposed (see, for example, Patent Document 1). Patent Document 1 acquires a high-sensitivity image and a low-sensitivity image with little temporal and spatial deviation using a solid-state imaging device in which each light-receiving cell is divided into a high-sensitivity light-receiving region and a low-sensitivity light-receiving region, Of these two images, an image quality deterioration area such as overexposure or blackout is automatically determined from one image, and the corresponding area is cut out from the other image according to the information on the image quality deterioration area, and the cut image portion is image quality deterioration area Synthesize the part. Thereafter, an image composition method is disclosed in which low-pass filter processing is performed on the entire region of the composite image or a region in the vicinity of the composite boundary, and after the images are smoothly connected, contour enhancement processing is performed.
JP 2004-48445 A

  In the above conventional method, the image picked up by the low sensitivity light receiving cell and the image picked up by the high sensitivity light receiving cell are partially cut and pasted to synthesize one wide dynamic range image. Since the S / N ratio is lowered with respect to the light receiving cell, when they are partially combined, an unnatural image is obtained. In addition, although the low-sensitivity light-receiving cell and the high-sensitivity light-receiving cell are close to each other, they are arranged at different positions. Also, a spatially subtle shift occurs, resulting in an unnatural image.

  In addition, it is possible to correct overexposure or underexposure by replacing the image quality degradation area of one image with the area corresponding to the image quality degradation area of the other image. If both the light receiving cell and the output of the low sensitivity light receiving cell are saturated, it is difficult to perform such correction.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide an imaging device capable of faithfully and reliably reproducing a subject image.

  The imaging device of the present invention is an imaging device including a solid-state imaging device having a large number of pixels, and each of the large number of pixels absorbs light in a specific wavelength region formed above the semiconductor substrate. An exposure condition determining means for determining an exposure condition of the photoelectric conversion element, the photoelectric conversion film generating a corresponding charge, and a photoelectric conversion element formed in the semiconductor substrate below the photoelectric conversion film; In imaging under the exposure condition determined by the condition determining means, the voltage applied to the photoelectric conversion film is adjusted so that there is no signal exceeding the saturation level in the signals from the photoelectric conversion film included in the large number of pixels. An applied voltage adjusting unit is provided, and imaging based on the exposure condition is performed in a state where the voltage adjusted by the applied voltage adjusting unit is applied to the photoelectric conversion film.

  In the imaging device of the present invention, the photoelectric conversion film absorbs light in an infrared wavelength region and generates a charge corresponding to the light.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device which can reproduce a to-be-photographed object faithfully and reliably can be provided.

  Embodiments of the present invention will be described below with reference to the drawings. In the following description, light in the blue (B) wavelength region (generally about 380 nm to about 520 nm) of the incident light is referred to as B light, and the green (G) wavelength region (generally about 450 nm). (About 610 nm) is called G light, red (R) wavelength region (generally about 550 nm to about 700 nm) light is called R light, and infrared (IR) wavelength region (generally) Is about 680 nm to about 3000 nm) is called IR light.

(First embodiment)
FIG. 1 is a schematic plan view showing a schematic configuration of the solid-state imaging device according to the first embodiment of the present invention.
A solid-state imaging device 200 illustrated in FIG. 1 includes a pixel 202R that can output an R signal according to R light of incident light, a pixel 202G that can output a G signal according to G light of incident light, Three types of pixels, a pixel 202B capable of outputting a B signal corresponding to B light of incident light, are provided, and these pixels are arranged in a row direction X on a substrate 201 such as silicon and a column direction Y orthogonal thereto. Are two-dimensionally arranged.

  As shown in FIG. 1, the pixel array of the solid-state imaging device 200 includes a GR pixel row, which is a pixel row in which pixels 202G and 202R are alternately arranged in the row direction X, and pixels 202B and 202G in the row direction X. BG pixel rows which are pixel rows alternately arranged in the column direction Y are alternately arranged. The pixel arrangement of the solid-state imaging device 200 is not limited to that shown in FIG. 1, and the pixels 202R, 202G, and 202B may be arranged in a vertical stripe or a horizontal stripe.

  A row selection scanning unit 203 is provided on the side of the substrate 201, and an image signal processing unit 204 is provided on the lower side. In addition, the substrate 201 is provided with a control unit 205 at appropriate positions for generating a timing pulse for selecting a pixel and generating various control signals for driving the pixel.

  Two signal lines of a reset signal line 206 and a row selection signal line 207 are provided in the upper portion of each pixel row so as to correspond to the pixel row and extend in the row direction X. The reset signal line 206 and the row selection signal line 207 are connected to each pixel of the corresponding pixel row and the row selection scanning unit 203.

  On the right side of the pixel column composed of a plurality of pixels arranged in the column direction Y, two signal lines of a column signal line 208 and a column signal line 209 extend in the column direction Y corresponding to the pixel column. Is provided. The column signal line 208 and the column signal line 209 are connected to each pixel of the corresponding pixel column and the image signal processing unit 204.

FIG. 2 is a diagram illustrating the four pixels surrounded by a broken line (A) in FIG. 1 combined into one.
The substrate 201 is, for example, an n-type silicon substrate, and a p-well layer 211 is formed thereon. The combined portion of the substrate 201 and the p-well layer 211 constitutes a semiconductor substrate. On the p-well layer 211, an insulating layer 220 that is transparent to incident light is formed. On the insulating layer 220, a color filter 222R that transmits R light that is a component of the pixel 202R, a color filter 222G that transmits G light that is a component of the pixel 202G, and B light that is a component of the pixel 202B. A color filter layer made of a color filter 222B that transmits is formed. On the color filter layer, a pixel electrode 225 made of a material transparent to incident light divided for each pixel (for example, ITO or a thin metal film) is formed. On the pixel electrode 225, a photoelectric conversion film 226 that absorbs IR light and generates a charge corresponding thereto is formed. On the photoelectric conversion film 226, a common electrode 227 made of a material transparent to incident light (for example, ITO or a thin metal film) is formed. On the common electrode 227, a protective layer 228 that is transparent to incident light is formed. The photoelectric conversion film 226, the common electrode 227, and the protective layer 228 have a single configuration common to all pixels. At a position corresponding to each pixel on the protective layer 228, a microlens 229 for condensing incident light on each pixel is formed.

  The photoelectric conversion film 226 is made of an organic photoelectric conversion material (for example, a phthalocyanine-based organic material or a naphthalocyanine-based organic material) that absorbs IR light and generates a charge corresponding to the light, and transmits visible light. Yes. The photoelectric conversion film 226 may be composed of an inorganic material. When a predetermined bias voltage is applied to the pixel electrode 225 and the common electrode 227 to apply an electric field to the photoelectric conversion film 226, the photoelectric conversion film 226 generates signal charges corresponding to the amount of incident IR light. The photoelectric conversion film 226 is formed by laminating the photoelectric conversion material on the pixel electrode 225 by a sputtering method, a laser ablation method, a printing technique, a spray method, or the like.

  The photoelectric conversion film 226 and the common electrode 227 may be divided corresponding to each pixel. In the case where the common electrode 227 is divided for each pixel, the divided common electrode 227 may be connected by a common wiring so that the same bias voltage can be applied to each.

  The pixel 202R, the pixel 202G, and the pixel 202B each include a part of the p-well layer 211, a color filter, a pixel electrode 225, a part of the photoelectric conversion film 226, a part of the common electrode 227, and a microlens. 229. Since the structure of each of the pixel 202R, the pixel 202G, and the pixel 202B is common except for the color filter, the common structure will be described below by using the pixel 202R as a representative.

  A pixel electrode 225 is formed on the color filter 222R of the pixel 202R. The pixel electrode 225, the common electrode 227 facing the pixel electrode 225, and the photoelectric conversion film 226 sandwiched between the pixel electrode 225 constitute an organic photoelectric conversion element that detects IR light.

  In the p well layer 211 of the pixel 202R, an n-type impurity layer (hereinafter referred to as an n layer) 212 formed from the surface to the inside is formed. A p-type impurity layer (hereinafter referred to as a p layer) 213 for suppressing dark current having an impurity concentration higher than that of the n layer 212 is formed on the inner side from the surface of the n layer 212. The p well layer 211, the n layer 212, and the p layer 213 constitute a photodiode (PD) 214 that is a photoelectric conversion element. Electric charges generated in the photodiode 214 are accumulated in the n layer 212.

  A signal readout circuit 215 is formed on the left side of the photodiode 214 with a little separation. The signal readout circuit 215 is, for example, a transistor circuit having a 3-transistor configuration or a 4-transistor configuration used in an existing CMOS type image sensor, and the row selection scanning unit 203 and the image signal processing unit 204 are also an existing CMOS type image sensor. The same ones used can be used.

  The output of the signal readout circuit 215 is connected to the column signal line 209. The signal readout circuit 215 is also connected to the row selection signal line 207, and when a pulse is applied from here, outputs a voltage signal corresponding to the electric charge accumulated in the n layer 212 to the column signal line 209. The signal readout circuit 215 is also connected to the reset signal line 206. When a pulse is applied from here, the signal readout circuit 215 performs a reset operation for sweeping out the electric charge accumulated in the n layer 212 to the reset drain.

  Next to the right side of the photodiode 214, a charge storage unit 216 made of, for example, an n-layer for storing charges generated in the photoelectric conversion film 226 of the organic photoelectric conversion element of the pixel 202R a little away from the inside thereof. Is formed. A signal readout circuit 217 is formed on the right side of the charge storage unit 216 with a slight separation.

  As the signal readout circuit 217, for example, a transistor circuit having a 3-transistor configuration or a 4-transistor configuration used in an existing CMOS type image sensor can be used.

  The output of the signal readout circuit 217 is connected to the column signal line 208. The signal readout circuit 217 is also connected to the row selection signal line 207, and when a pulse is applied therefrom, a voltage signal corresponding to the charge accumulated in the charge accumulation unit 216 is output to the column signal line 208. The signal readout circuit 217 is also connected to the reset signal line 206. When a pulse is applied from there, the signal readout circuit 217 performs a reset operation for sweeping out the charge accumulated in the charge accumulation unit 216 to the reset drain.

  Note that the signal readout circuit 215 and the signal readout circuit 217 are configured to transfer a signal charge detected in each pixel to a charge transfer path (vertical charge transfer path VCCD, horizontal charge) in the same manner as an existing CCD (charge coupled device) type solid-state imaging device. A structure may be employed in which the signal is transferred to the amplifier via the transfer path HCCD and the signal is output by the amplifier.

  A contact portion 224 made of aluminum or the like is formed through the pixel electrode 225 in the inorganic layer 220 and the color filter 222R on the charge storage portion 216. The contact portion 224 is directly connected to the pixel electrode 225 and the charge storage portion 216, and functions as a connection means for electrically connecting them.

  A poly wiring layer 218 and a three-layer metal wiring layer 219 are provided in the insulating layer 220. The number of these wiring layers is determined by the number of wirings necessary for the circuit operation of the signal readout circuits 215 and 217. The three-layer metal layer 219 also functions as a light shielding film that shields the signal readout circuits 215 and 217. The upper surface of the insulating layer 220 is flattened, and a color filter layer is formed thereon.

  The structure described above is a structure common to each pixel. Next, a different structure portion for each pixel will be described.

  A color filter 222R corresponding to the photodiode 214 is formed above the photodiode 214 of the pixel 202R. A color filter 222G corresponding to the photodiode 214 is formed above the photodiode 214 of the pixel 202G. A color filter 222B corresponding to the photodiode 214 is formed above the photodiode 214 of the pixel 202B.

  With such a configuration, the photodiode 214 of the pixel 202R functions as a first photoelectric conversion element that detects R light, and the photodiode 214 of the pixel 202G functions as a second photoelectric conversion element that detects G light. The photodiode 214 of the pixel 202B functions as a third photoelectric conversion element that detects B light. As described above, each pixel of the solid-state imaging device 200 includes the organic photoelectric conversion element and the photodiode 214.

  Next, the operation of the solid-state imaging device 200 having such a configuration will be described.

  When light is incident on the solid-state imaging device 200, IR light of incident light is absorbed by the photoelectric conversion film 226 and R light, G light, B light, and ultraviolet light are transmitted through the photoelectric conversion film 226 in the pixel 202R. Of the transmitted light, R light passes through the color filter 222R and is absorbed by the photodiode 214.

  In the pixel 202G, IR light of incident light is absorbed by the photoelectric conversion film 226, and R light, G light, B light, and ultraviolet light pass through the photoelectric conversion film 226. Of the transmitted light, G light passes through the color filter 222G and is absorbed by the photodiode 214.

  In the pixel 202B, IR light of incident light is absorbed by the photoelectric conversion film 226, and R light, G light, B light, and ultraviolet light pass through the photoelectric conversion film 226. Of the transmitted light, B light passes through the color filter 222B and is absorbed by the photodiode 214.

  In the photoelectric conversion film 226 in the pixel 202R, a signal charge corresponding to the amount of incident IR light is generated. The signal charges generated in the photoelectric conversion film 226 of the pixel 202R are collected on the pixel electrode 225 of the pixel 202R, pass through the contact portion 224 of the pixel 202R, converted into a signal by the signal readout circuit 217 of the pixel 202R, and the column signal line 208 is output as an IR signal.

  In the photoelectric conversion film 226 in the pixel 202G, signal charges corresponding to the amount of incident IR light are generated. The signal charges generated in the photoelectric conversion film 226 of the pixel 202G are collected on the pixel electrode 225 of the pixel 202G, pass through the contact portion 224 of the pixel 202G, converted into a signal by the signal readout circuit 217 of the pixel 202G, and the column signal line 208 is output as an IR signal.

  In the photoelectric conversion film 226 in the pixel 202B, signal charges corresponding to the amount of incident IR light are generated. The signal charges generated in the photoelectric conversion film 226 of the pixel 202B are collected on the pixel electrode 225 of the pixel 202B, passed through the contact portion 224 of the pixel 202B, converted into a signal by the signal readout circuit 217 of the pixel 202B, and the column signal line 208 is output as an IR signal.

  By processing IR signals obtained from the pixels 202R, 202G, and 202B output from the column signal line 208, infrared image data consisting of the total number of pixels of the solid-state imaging device 200 (IR data corresponding to each pixel is IR data). Data with a signal) can be generated.

  In the photodiode 214 of the pixel 202R, an R signal charge corresponding to the amount of incident R light is generated. The R signal corresponding to the R signal charge is output from the signal readout circuit 215 of the pixel 202R to the column signal line 209.

  In the photodiode 214 of the pixel 202G, a G signal charge corresponding to the amount of incident G light is generated. The G signal corresponding to the G signal charge is output from the signal readout circuit 215 of the pixel 202G to the column signal line 209.

  In the photodiode 214 of the pixel 202B, a B signal charge corresponding to the amount of incident B light is generated. The B signal corresponding to the B signal charge is output from the signal readout circuit 215 of the pixel 202B to the column signal line 209.

  Then, by using the R signal, G signal, and B signal output in this way, RGB color image data in which the pixel data corresponding to each pixel has three color information of RGB can be generated. .

  By the above method, it is possible to obtain RGB color image data and infrared image data with high accuracy by one imaging.

The solid-state imaging device 200 having the above-described configuration can be used by being mounted on an imaging apparatus such as a digital camera or a digital video camera. Below, the structure of the digital camera which is an example of the imaging device carrying the solid-state image sensor 200 is demonstrated.
FIG. 3 is a diagram illustrating a schematic configuration of a digital camera which is an example of an imaging apparatus in which the solid-state imaging device 200 illustrated in FIG. 1 is mounted.
The imaging system of the digital camera shown in the figure includes a photographic lens 1, a solid-state imaging device 200 having the above-described configuration, a diaphragm 2 provided between the two, and an optical low-pass filter 4.

  A system control unit 11 that performs overall control of the electrical control system of the digital camera controls the flash light emitting unit 12 and the light receiving unit 13 and controls the lens driving unit 8 to adjust the position of the photographing lens 1 to the focus position and zoom. The exposure amount is adjusted by adjusting the aperture amount of the aperture 2 via the aperture drive unit 9.

  Further, the system control unit 11 drives the solid-state image sensor 200 via the image sensor driving unit 10 and outputs a subject image captured through the photographing lens 1 as a color signal. An instruction signal from the user is input to the system control unit 11 through the operation unit 14.

  The electric control system of the digital camera further includes an analog signal processing unit 6 that performs analog signal processing such as correlated double sampling processing connected to the output of the solid-state imaging device 200, and a signal output from the analog signal processing unit 6. And an A / D conversion circuit 7 for converting the signal into a digital signal, which are controlled by the system control unit 11.

  Furthermore, the electric control system of this digital camera generates image data for recording by performing various signal processing such as interpolation calculation and gamma correction calculation with the main memory 16 and the memory control unit 15 connected to the main memory 16. A digital signal processing unit 17 that compresses the image data generated by the digital signal processing unit 17 into a JPEG format or expands the compressed image data, and a digital signal processing unit that integrates the photometric data. 17 for obtaining the gain of white balance correction performed by the display 17; an external memory control unit 20 to which a detachable recording medium 21 is connected; and a display control unit to which a liquid crystal display unit 23 mounted on the back of the camera is connected. 22 are connected to each other by a control bus 24 and a data bus 25, and controlled by a command from the system control unit 11. That.

FIG. 4 is a block diagram showing a detailed configuration of the digital signal processing unit 17 shown in FIG.
The digital signal processing unit 17 includes a color image data generation unit 171, an infrared image data generation unit 172, a contour image data generation unit 173, and an image composition unit 174. The functions of these blocks are realized by the digital signal processing unit 17 executing a predetermined program.

  The color image data generation unit 171 is a color image in which R, G, and B color signals are given to one pixel data from the R, G, and B signals obtained from the photodiode 214 of each pixel of the solid-state imaging device 200. Generate data. Specifically, the color image data generation unit 171 performs a synchronization process for interpolating other color signals that cannot be obtained from the photodiode 214 of each pixel at the pixel position corresponding to each pixel of the solid-state imaging device 200, Three color signals of R, G, and B are generated at the pixel position, and the three color signals are made up of pixel data of the same number as the total number of pixels of the solid-state imaging device 200 as pixel data corresponding to the pixel position. Generate color image data. For example, although the R signal exists at the pixel position corresponding to the pixel 202R, but the G and B signals do not exist, the G and B signals at the surrounding pixel positions are used for this pixel position. The G and B signals are interpolated to generate three color signals R, G, and B at the pixel position corresponding to the pixel 202R.

  The infrared image data generation unit 172 generates infrared image data in which one pixel data has an IR signal from the IR signal obtained from the organic photoelectric conversion element of each pixel of the solid-state imaging device 200. Specifically, the infrared image data generation unit 172 arranges the IR signal obtained from the organic photoelectric conversion element of each pixel at the pixel position corresponding to each pixel of the solid-state imaging device 200, and uses the IR signal as the pixel. As pixel data corresponding to the position, infrared image data composed of the same number of pixel data as the total number of pixels of the solid-state imaging device 200 is generated.

  The contour image data generation unit 173 generates contour image data obtained by extracting only contour information from the infrared image data generated by the infrared image data generation unit 172. The method for extracting the contour information is well known and will not be described.

  The image synthesizing unit 174 synthesizes the color image data generated by the color image data generating unit 171 and the contour image data generated by the contour image data generating unit 173, and image data for recording on the recording medium 21. Is output to the compression / decompression processing unit 18.

FIG. 5 is a diagram showing the relationship between the output signal (PD signal) from the photoelectric conversion element 214 and the output signal (film signal) from the organic photoelectric conversion element with respect to the subject illuminance.
Since the organic photoelectric conversion element has a larger light receiving area than the photoelectric conversion element 214, its dynamic range is larger than the dynamic range of the photoelectric conversion element 214. In other words, even if the subject has a high illuminance region in which the output signal is saturated in the photoelectric conversion element 214, the high illuminance region is imaged without causing the overexposure by the organic photoelectric conversion element. Is possible.

  For example, as shown in FIG. 6A, consider the case of imaging a building against a blue sky with clouds. In this subject, it is assumed that an area with clouds and a blue sky is a high illuminance area that is an illuminance level at which the output signal from the photoelectric conversion element 214 is saturated. When such a subject is photographed by the digital camera according to the present embodiment, the color image data generated by the color image data generation unit 171 has a high illuminance area overexposed as shown in FIG. 6B. As a result, the outline of the cloud portion is erased. On the other hand, the infrared image data generated by the infrared image data generation unit 172 has a high illuminance area that is not overexposed, as shown in FIG.

  In the digital camera of this embodiment, the contour image data generation unit 173 extracts contour information from the infrared image data shown in FIG. 6C and generates contour image data as shown in FIG. Then, the image synthesis unit 174 synthesizes the contour image data shown in FIG. 6 (d) and the color image data shown in FIG. 6 (b) to generate composite image data as shown in FIG. 6 (e). I am going to do it. Since the outline of the cloud remains in the outline image data shown in FIG. 6D, this is combined with the color image data shown in FIG. It becomes possible to reproduce the outline of the cloud that has been blown out and disappeared.

Next, the operation of the digital camera will be described. FIG. 7 is a flowchart for explaining the photographing operation of the digital camera of this embodiment.
When a photographing instruction is given, the electronic shutter of the photoelectric conversion element 214 is “opened” by the control of the image pickup element driving unit 10, and a predetermined bias voltage is applied between the pixel electrode 225 and the common electrode 227, and the photoelectric conversion element The imaging of the subject is simultaneously started by 214 and the organic photoelectric conversion element (steps S1 and S3). When the exposure period ends, color image data is generated from the R, G, and B signals obtained from the photoelectric conversion element 214 (step S2). Further, IR image data is generated from the IR signal obtained from the organic photoelectric conversion element, and contour image data obtained by extracting contour information from the IR image data is generated (step S4). Note that step S2 and step S4 may be performed simultaneously, or whichever may be performed first.

  After step S2 and step S4, the color image data and the contour image data are synthesized (step S5), and color image data for recording medium recording that faithfully reproduces the contour of the subject is output (step S6). After the image data is compressed, it is recorded on the recording medium 21.

  As described above, the digital camera according to the present embodiment generates the color image data generated from the signal obtained from the photoelectric conversion element 214 having a narrow dynamic range by using the signal obtained from the organic photoelectric conversion element having a wide dynamic range. The image data for recording on the recording medium 21 is generated by synthesizing the contour image data representing the contour of the subject. For this reason, even if the subject has a high illuminance region that is overexposed only by the photoelectric conversion element 214, image data that faithfully reproduces the contour of the subject in the high illuminance region can be obtained.

  In the digital camera according to the present embodiment, the photoelectric conversion film of the solid-state imaging device 200 is formed of a photoelectric conversion material that absorbs IR light and generates a charge corresponding to the IR light. For this reason, even when shooting in the dark, it is possible to obtain image data that faithfully reproduces the outline of the building in the dark. The digital camera of this embodiment can obtain image data that faithfully reproduces the outline of any subject (subject that is too bright or subject that is too dark). This makes it suitable for systems where it is important to be able to detect the movement of the subject even in situations.

  In addition, since the organic photoelectric conversion material constituting the photoelectric conversion film 226 of the solid-state imaging device 200 has a shorter mean free path of infrared light than silicon, the cross-pixel crossing compared to the case where infrared imaging is performed using silicon. It is possible to obtain infrared image data with little talk, that is, high resolution. It is preferable that the photoelectric conversion film 226 is made of an organic photoelectric conversion material rather than an inorganic material because the high-resolution infrared image data improves contour extraction accuracy and improves contour reproducibility.

  Further, since the solid-state imaging device 200 has a structure in which the photoelectric conversion element 214 and the organic photoelectric conversion element are stacked at the same position, the color image data generated by the color image data generation unit 171 and the infrared image The infrared image data generated by the data generation unit 172 can be spatially and temporally free from any deviation. Further, since the image data for recording is generated by synthesizing the entire color image data without deviation and the entire contour image data generated from the infrared image data, an image based on the image data for recording is generated. It can be made more uncomfortable.

  In the present embodiment, a large number of photoelectric conversion elements 214 included in the solid-state imaging device 200 include a photoelectric conversion element that detects R light, a photoelectric conversion element that detects G light, and a photoelectric conversion element that detects B light. However, the present invention is not limited to this, and a large number of photoelectric conversion elements 214 may be formed of four or more types of photoelectric conversion elements.

  The photoelectric conversion material constituting the photoelectric conversion film 226 is not limited to a material that absorbs IR light and generates a charge corresponding to the IR light, and can generate contour image data from a signal obtained from the photoelectric conversion film 226. Any material can be used. However, if a material that absorbs light that is to be absorbed by the photoelectric conversion element 214 is used, light is not incident on the photoelectric conversion element 214, and thus the material that transmits light to be absorbed by the photoelectric conversion element 214 is transmitted. It is desirable.

  In this embodiment, the image composition unit 174 always outputs the image data obtained by combining the color image data and the contour image data as recording image data. It is also possible to provide means for determining whether or not the signal from the photoelectric conversion element 214 has reached the saturation level, and to change the output contents in accordance with the determination result. In this case, if the signal from the photoelectric conversion element 214 has not reached the saturation level, the image composition unit 174 outputs the color image data generated by the color image data generation 371 as recording image data as it is, and performs photoelectric conversion. When the signal from the element 214 has reached the saturation level, a combination of the color image data generated by the color image data generation 371 and the contour image data may be output as recording image data.

(Second embodiment)
The overall configuration of the digital camera described in the present embodiment is almost the same as that shown in FIG. 3, but the configurations of the solid-state imaging device 200 and the digital signal processing unit 17 are slightly different.
The solid-state imaging device 200 of the digital camera according to the present embodiment absorbs light (hereinafter referred to as emerald (E) light) in a part of the wavelength range (about 480 nm to about 520 nm) of the G light as the photoelectric conversion film 226. The photoelectric conversion material which generates the electric charge according to is used. With such a configuration, E image data including only an E component is generated using the output signal of the organic photoelectric conversion element, and color reproduction is performed using the output signal of the photoelectric conversion element 214 and the output signal of the organic photoelectric conversion element. It is possible to generate RGB color image data with improved performance.

FIG. 8 is a block diagram showing a detailed configuration of the digital signal processing unit of the digital camera according to the second embodiment of the present invention.
The digital signal processing unit 17 illustrated in FIG. 8 includes a color image data generation unit 271, an E image data generation unit 272, a contour image data generation unit 273, an image synthesis unit 274, and a saturation level determination unit 275. ing.

  The color image data generation unit 271 converts R, G, and B signals obtained from the photodiode 214 of each pixel of the solid-state imaging device 200 and E signal obtained from the photoelectric conversion film 226 into one pixel data. , G, B color signal data is generated. Specifically, the color image data generation unit 271 performs a synchronization process of interpolating other color signals that cannot be obtained from the photodiode 214 of each pixel at the pixel position corresponding to each pixel of the solid-state imaging device 200, Four color signals of R, G, B, and E are generated at the pixel position, and the visibility characteristic of the R signal among the four color signals is corrected using the E signal. Then, using the three color signals of the corrected R signal, G signal, and B signal as pixel data corresponding to the pixel position, a color image composed of the same number of pixel data as the total number of pixels of the solid-state imaging device 200 Generate data.

  The advantage of detecting the light of emerald color with a wavelength of 480 to 520 nm is to correct red according to the human visual sensitivity. As for human visibility, even if only the R, G, and B positive sensitivities are detected by the photoelectric conversion element 214 of the solid-state imaging device 200 and color reproduction is performed, an image viewed by humans cannot be reproduced. Therefore, the signal processing described in Japanese Patent No. 2872759 is a signal processing in which the negative negative sensitivity with the highest negative sensitivity is detected by the emerald film, and the negative sensitivity is subtracted from the red sensitivity detected by the photoelectric conversion element 214. By performing in the same manner as described above, it is possible to reproduce the human sensitivity to red and improve color reproducibility.

  The E image data generation unit 272 generates E image data in which one pixel data has an E signal from the E signal obtained from the organic photoelectric conversion element of each pixel of the solid-state imaging element 200. Specifically, the E image data generation unit 272 arranges the E signal obtained from the organic photoelectric conversion element of each pixel at the pixel position corresponding to each pixel of the solid-state imaging device 200, and uses the E signal as the pixel position. E image data composed of the same number of pixel data as the total number of pixels of the solid-state imaging device 200 is generated as the pixel data corresponding to.

  The contour image data generation unit 273 generates contour image data obtained by extracting only contour information from the E image data generated by the E image data generation unit 272. The method for extracting the contour information is well known and will not be described.

  The saturation level determination unit 275 determines whether there is any signal that has reached the saturation level among the R, G, and B signals obtained from the photoelectric conversion element 214 and notifies the image composition unit 274 of the determination result.

  When the R, G, B signals obtained from the photoelectric conversion element 214 do not reach the saturation level, that is, the image illuminance does not saturate the output from the photoelectric conversion element 214. In the case of the level, the color image data generated by the color image data generation unit 271 is directly output as recording image data. On the other hand, in the case where some of the R, G, B signals obtained from the photoelectric conversion element 214 have reached a saturation level, that is, the subject illuminance is at a level at which the output from the photoelectric conversion element 214 is saturated. In this case, the color image data generated by the color image data generation unit 271 and the contour image data generated by the contour image data generation unit 273 are combined to generate image data for recording and output it.

Next, the operation of the digital camera will be described.
When a photographing instruction is given, the electronic shutter of the photoelectric conversion element 214 is “opened” by the control of the image pickup element driving unit 10, and a predetermined bias voltage is applied between the pixel electrode 225 and the common electrode 227, and the photoelectric conversion element The imaging of the subject is started simultaneously with 214 and the organic photoelectric conversion element. When the exposure period ends, color image data is generated from the R, G, B signals obtained from the photoelectric conversion element 214 and the E signal obtained from the organic photoelectric conversion element.

  Next, it is determined whether or not the R, G, and B signals obtained from the photoelectric conversion element 214 have reached the saturation level. If the saturation level has not been reached, the generated color image data is used as recording image data. Is output. On the other hand, when the saturation level is reached, E image data is generated from the E signal obtained from the organic photoelectric conversion element, and contour image data is generated therefrom. The contour image data and the color image data are combined and output as recording image data. The recording image data is compressed and then recorded on the recording medium 21.

  As described above, according to the digital camera of the present embodiment, when the illuminance of the subject is at a level where the output from the photoelectric conversion element 214 is not saturated, the R, G, B signal and E signal are used. , Color image data with improved color reproducibility can be generated and output as recording image data, and when the subject illuminance is at a level at which the output from the photoelectric conversion element 214 is saturated, Image data obtained by combining color image data with improved color reproducibility and contour image data can be output as recording image data. Therefore, when the subject illuminance is low and there is no white-out, color image data with high color reproducibility can be obtained. When the subject illuminance is high and there is white-out, the color reproducibility is high and the subject Color image data that faithfully reproduces the contour can be obtained.

  Note that the saturation level determination unit 275 illustrated in FIG. 8 may be omitted, and the image composition unit 274 may always output image data obtained by combining color image data and contour image data as recording image data. .

(Third embodiment)
The overall configuration of the digital camera described in the present embodiment is the same as that shown in FIG. 3, but the configurations of the solid-state imaging device 200 and the digital signal processing unit 17 are slightly different.
The solid-state imaging device 200 of the digital camera according to the present embodiment changes the pixel 202G in the GR pixel row shown in FIG. 1 to the pixel 202B, changes the pixel 202G in the BG pixel row shown in FIG. The conversion film 226 has a configuration using a photoelectric conversion material that absorbs G light and generates a charge corresponding thereto. With such a configuration, monochrome image data consisting of only the G component is generated using the output signal of the organic photoelectric conversion element, and RGB color data is generated using the output signal of the photoelectric conversion element 214 and the output signal of the organic photoelectric conversion element. Image data can be generated.

FIG. 9 is a block diagram showing a detailed configuration of the digital signal processing unit of the digital camera according to the third embodiment of the present invention.
The digital signal processing unit 17 illustrated in FIG. 9 includes a color image data generation unit 371, a G image data generation unit 372, a contour image data generation unit 373, an image synthesis unit 374, and a saturation level determination unit 375. ing.

  The color image data generation unit 371 converts R, G signals into one pixel data from the R, B signals obtained from the photodiodes 214 of the respective pixels of the solid-state imaging device 200 and the G signals obtained from the photoelectric conversion film 226. , B color signals having color signals are generated. Specifically, the color image data generation unit 371 performs a synchronization process for interpolating other color signals that cannot be obtained from the photodiode 214 of each pixel at the pixel position corresponding to each pixel of the solid-state imaging device 200, Three color signals of R, G, and B are generated at the pixel position, and the three color signals are made up of pixel data of the same number as the total number of pixels of the solid-state imaging device 200 as pixel data corresponding to the pixel position. Generate color image data. For example, although the R signal and the G signal exist at the pixel position corresponding to the pixel 202R, but the B signal does not exist, the B signal at the surrounding pixel positions is used for this pixel position. The signals are interpolated to generate three color signals R, G, and B at the pixel position corresponding to the pixel 202R.

  The G image data generation unit 372 generates G image data in which one pixel data has a G signal from the G signal obtained from the organic photoelectric conversion element of each pixel of the solid-state imaging element 200. Specifically, the G image data generation unit 372 arranges the G signal obtained from the organic photoelectric conversion element of each pixel at the pixel position corresponding to each pixel of the solid-state imaging device 200, and uses the G signal as the pixel position. G image data composed of the same number of pixel data as the total number of pixels of the solid-state imaging device 200 is generated as the pixel data corresponding to.

  The contour image data generation unit 373 generates contour image data obtained by extracting only the contour information from the G image data generated by the G image data generation unit 372. The method for extracting the contour information is well known and will not be described.

  The saturation level determination unit 375 determines whether or not any of the R and B signals obtained from the photoelectric conversion element 214 has reached the saturation level, and notifies the image synthesis unit 374 of the determination result.

  When the R and B signals obtained from the photoelectric conversion element 214 do not reach the saturation level, that is, the subject illuminance is at a level at which the output from the photoelectric conversion element 214 is not saturated. If there is, the color image data generated by the color image data generation unit 371 is output as it is as recording image data. On the other hand, when there is a signal that has reached the saturation level in the R and B signals obtained from the photoelectric conversion element 214, that is, when the subject illuminance is at a level at which the output from the photoelectric conversion element 214 is saturated, The color image data generated by the color image data generation unit 371 and the contour image data generated by the contour image data generation unit 373 are combined to generate recording image data, which is output.

FIG. 10 is a diagram showing the relationship between the output signal (PD signal) from the photoelectric conversion element 214 and the output signal (film signal) from the organic photoelectric conversion element with respect to the subject illuminance.
As shown in FIG. 10, since the PD is not saturated within the PD dynamic range, color image data faithfully reproducing the contour can be obtained without synthesizing the contour image data. On the other hand, since the PD is saturated outside the PD dynamic range, color image data that faithfully reproduces the contour can be obtained by synthesizing the contour image data.

Next, the operation of the digital camera will be described.
When a photographing instruction is given, the electronic shutter of the photoelectric conversion element 214 is “opened” by the control of the image pickup element driving unit 10, and a predetermined bias voltage is applied between the pixel electrode 225 and the common electrode 227, and the photoelectric conversion element The imaging of the subject is started simultaneously with 214 and the organic photoelectric conversion element. When the exposure period ends, color image data is generated from the R and B signals obtained from the photoelectric conversion element 214 and the G signal obtained from the organic photoelectric conversion element.

  Next, it is determined whether or not the R and B signals obtained from the photoelectric conversion element 214 have reached the saturation level. If the saturation level has not been reached, the generated color image data is output as recording image data. The On the other hand, when the saturation level is reached, G image data is generated from the G signal obtained from the organic photoelectric conversion element, and contour image data is generated therefrom. The contour image data and the color image data are combined and output as recording image data. The recording image data is compressed and then recorded on the recording medium 21.

  As described above, according to the digital camera of the present embodiment, when the illuminance of the subject is at a level where the output from the photoelectric conversion element 214 is not saturated, a color image is obtained using the R, B signal, and G signal. Data can be generated and output as recording image data. When the illuminance of the subject is at a level at which the output from the photoelectric conversion element 214 is saturated, the color image data and the contour image data are combined. The processed image data can be output as recording image data. For this reason, color image data can be obtained without performing contour extraction when the subject illuminance is low and there is no whitening. If the subject illuminance is high and there is whitening, the contour of the subject is faithfully reproduced. Color image data can be obtained.

  Note that the saturation level determination unit 375 shown in FIG. 9 may be omitted, and the image composition unit 374 may always output image data obtained by combining color image data and contour image data as recording image data. .

(Fourth embodiment)
As described in the first to third embodiments, even if the output of the photoelectric conversion element 214 is saturated, the effect of obtaining color image data that faithfully reproduces the contour is obtained from the photoelectric conversion film 226. It cannot be obtained if the output signal is saturated. Therefore, in the present embodiment, the image sensor driving unit 10 of the digital camera shown in FIG. 3 applies the output signal from the photoelectric conversion film 226 between the pixel electrode 225 and the common electrode 227 so as not to reach a saturation level. The solid-state imaging device 200 is driven so as to image a subject after adjusting the bias voltage.

FIG. 11 is a diagram showing the relationship between the bias voltage applied to the photoelectric conversion film 226 and the output signal from the photoelectric conversion film 226.
As shown in FIG. 11, it can be seen that the level of the output signal from the photoelectric conversion film 226 increases as the bias voltage increases even with the same subject illuminance. In other words, it can be seen that the sensitivity of the photoelectric conversion film 226 is proportional to the bias voltage. For this reason, if the subject has a high illuminance area and the output signal has reached a saturation level in this high illuminance area (when the dynamic range is insufficient), the voltage applied to the photoelectric conversion film 226 is set to If the output signal in the high illuminance region has a sufficient margin to the saturation level (when the dynamic range is sufficient), the voltage applied to the photoelectric conversion film 226 is increased to increase the dynamic range. It is effective to improve the sensitivity by narrowing the range.
In the digital camera of the present embodiment, in the digital camera described in the first to third embodiments, the system control unit 11 determines whether or not the output signal from the photoelectric conversion film 226 has reached a saturation level, In a case where the saturation level has been reached, the output signal of the photoelectric conversion film 226 is not saturated by controlling the image sensor driving unit 10 to reduce the applied voltage to the photoelectric conversion film 226 to widen the dynamic range. So you can start shooting.

Hereinafter, the operation of the digital camera of this embodiment will be described.
FIG. 12 is a flowchart for explaining the photographing operation of the digital camera according to the fourth embodiment of the present invention. In FIG. 12, the same processes as those in FIG. FIG. 13 is a diagram showing a detailed flow of the process of step S13 shown in FIG.
When the shutter button is half-pressed, the system control unit 11 determines the exposure conditions (lens aperture value, shutter speed, etc.) of the photoelectric conversion element 214 based on the photometric data obtained from the light receiving unit 13 (step). S11). Next, the system control unit 11 performs photoelectric detection via the image sensor driving unit 10 so that no output signal from the photoelectric conversion film 226 reaches a saturation level in imaging under the determined exposure conditions. The voltage applied to the conversion film 226 is adjusted (step S12).

  Specifically, first, the system control unit 11 controls the image sensor driving unit 10 to perform provisional imaging under the determined exposure condition, and from the signal obtained from the photoelectric conversion film 226 by the provisional imaging, the photoelectric conversion film It is determined whether or not there is a region (pixel) where the output signal from 226 has reached the saturation level (step S121). If there is a saturated region, the system control unit 11 decreases the voltage applied to the photoelectric conversion film 226 via the image sensor driving unit 10 (step S122). Next, the system control unit 11 performs temporary imaging again, and there is a region (pixel) in which the output signal from the photoelectric conversion film 226 has reached a saturation level from the signal obtained from the photoelectric conversion film 226 by the temporary imaging. It is determined whether or not (step S123). When there is a saturated region, the process proceeds to step S122, and when there is no saturated region, the adjustment of the voltage applied to the photoelectric conversion film 226 is finished (step S127).

  If there is no saturated region in the determination of step S121, the system control unit 11 increases the voltage applied to the photoelectric conversion film 226 via the image sensor driving unit 10 (step S124). Next, the system control unit 11 performs temporary imaging again, and there is a region (pixel) in which the output signal from the photoelectric conversion film 226 has reached a saturation level from the signal obtained from the photoelectric conversion film 226 by the temporary imaging. Whether or not (step S125). If there is no saturated region, the process proceeds to step S124. If there is a saturated region, the voltage applied to the photoelectric conversion film 226 is set to the voltage set in the immediately preceding step S124. The setting is made (step S126), and the adjustment is finished (step S127).

  After adjusting the voltage to be applied to the photoelectric conversion film 226, the system control unit 11 is driven based on the determined exposure condition and the adjusted voltage is applied to the photoelectric conversion film 226 when the shutter button is fully pressed and a shooting instruction is given. The application is performed via the image sensor driving unit 10, and the operations after Steps S1 and S3 are performed.

  As described above, according to the present embodiment, since the main imaging can be started in a state where the output signal from the photoelectric conversion film 226 is not saturated, the contour is faithful even if the output of the photoelectric conversion element 214 is saturated. Thus, it is possible to reliably obtain the effect of obtaining color image data reproduced in the same manner.

1 is a schematic plan view showing a schematic configuration of a solid-state imaging device according to a first embodiment of the present invention. FIG. 1 is a diagram illustrating a cross section of four pixels surrounded by a broken line (A) in FIG. The figure which shows schematic structure of the digital camera which is an example of the imaging device carrying the solid-state image sensor shown in FIG. The block diagram which showed the detailed structure of the digital signal processing part shown in FIG. The figure which showed the relationship between the output signal (PD signal) from a photoelectric conversion element with respect to object illumination intensity, and the output signal (film | membrane signal) from an organic photoelectric conversion element with respect to object illumination intensity in the solid-state image sensor of the digital camera of 1st embodiment. Diagram for explaining image composition processing The flowchart for demonstrating imaging | photography operation | movement of the digital camera of 1st embodiment. The block diagram which showed the detailed structure of the digital signal processing part of the digital camera of 2nd embodiment of this invention. The block diagram which showed the detailed structure of the digital signal processing part of the digital camera of 3rd embodiment of this invention. The figure which showed the relationship between the output signal (PD signal) from a photoelectric conversion element with respect to object illumination intensity, and the output signal (film | membrane signal) from an organic photoelectric conversion element with respect to to-be-photographed object in the solid-state image sensor of the digital camera of 3rd embodiment. The figure which showed the relationship between the bias voltage applied to a photoelectric converting film, and the output signal from this photoelectric converting film Flowchart for explaining the photographing operation of the digital camera according to the fourth embodiment of the present invention. The figure which showed the detailed flow of the process of step S13 shown in FIG.

Explanation of symbols

200 Solid-state imaging device 202R, G, B Pixel 214 Photoelectric conversion device 226 Photoelectric conversion film

Claims (2)

An imaging device including a solid-state imaging device having a large number of pixels,
Each of the plurality of pixels is formed in the semiconductor substrate below the photoelectric conversion film, which absorbs light in a specific wavelength region formed above the semiconductor substrate and generates a charge corresponding thereto. A photoelectric conversion element,
Exposure condition determining means for determining an exposure condition of the photoelectric conversion element;
In imaging under the exposure condition determined by the exposure condition determining means, the voltage applied to the photoelectric conversion film so that there is no signal exceeding the saturation level in the signals from the photoelectric conversion film included in the multiple pixels And an applied voltage adjusting means for adjusting
An imaging apparatus that performs imaging based on the exposure condition in a state where a voltage adjusted by the applied voltage adjusting unit is applied to the photoelectric conversion film. The imaging apparatus according to claim 1,
An imaging apparatus in which the photoelectric conversion film absorbs light in an infrared wavelength region and generates a charge corresponding thereto.

Images