CN109997352B - Imaging device, camera, and imaging method - Google Patents

Imaging device, camera, and imaging method Download PDF

Info

Publication number
CN109997352B
CN109997352B CN201780072782.1A CN201780072782A CN109997352B CN 109997352 B CN109997352 B CN 109997352B CN 201780072782 A CN201780072782 A CN 201780072782A CN 109997352 B CN109997352 B CN 109997352B
Authority
CN
China
Prior art keywords
light
image
frame image
correction
frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201780072782.1A
Other languages
Chinese (zh)
Other versions
CN109997352A (en
Inventor
关东弘明
吉田典巧
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Management Co Ltd
Original Assignee
Panasonic Intellectual Property Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Intellectual Property Management Co Ltd filed Critical Panasonic Intellectual Property Management Co Ltd
Publication of CN109997352A publication Critical patent/CN109997352A/en
Application granted granted Critical
Publication of CN109997352B publication Critical patent/CN109997352B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
    • H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
    • H01L27/144—Devices controlled by radiation
    • H01L27/146—Imager structures
    • H01L27/14601—Structural or functional details thereof
    • H01L27/14638—Structures specially adapted for transferring the charges across the imager perpendicular to the imaging plane
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/63—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to dark current
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
    • H01L27/02—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components specially adapted for rectifying, oscillating, amplifying or switching and having at least one potential-jump barrier or surface barrier; including integrated passive circuit elements with at least one potential-jump barrier or surface barrier
    • H01L27/04—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components specially adapted for rectifying, oscillating, amplifying or switching and having at least one potential-jump barrier or surface barrier; including integrated passive circuit elements with at least one potential-jump barrier or surface barrier the substrate being a semiconductor body
    • H01L27/10—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components specially adapted for rectifying, oscillating, amplifying or switching and having at least one potential-jump barrier or surface barrier; including integrated passive circuit elements with at least one potential-jump barrier or surface barrier the substrate being a semiconductor body including a plurality of individual components in a repetitive configuration
    • H01L27/118—Masterslice integrated circuits
    • H01L27/11803—Masterslice integrated circuits using field effect technology
    • H01L27/11807—CMOS gate arrays
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
    • H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
    • H01L27/142—Energy conversion devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
    • H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
    • H01L27/144—Devices controlled by radiation
    • H01L27/146—Imager structures
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02E—REDUCTION OF GREENHOUSE GAS [GHG] EMISSIONS, RELATED TO ENERGY GENERATION, TRANSMISSION OR DISTRIBUTION
    • Y02E10/00—Energy generation through renewable energy sources
    • Y02E10/50—Photovoltaic [PV] energy

Abstract

The disclosed device is provided with: an imaging element (10) including a photoelectric conversion unit (111) and a plurality of pixel circuits (21), the photoelectric conversion unit (111) generating charges based on an internal photoelectric effect by receiving light in a state where a voltage of a 1 st predetermined range is applied, and the plurality of pixel circuits (21) accumulating the charges generated by the photoelectric conversion unit (111) in units of pixels without generating the charges based on the internal photoelectric effect even if receiving light in a state where a voltage of a 2 nd predetermined range is applied, the imaging element (10) outputting a frame image based on the amount of charges accumulated by the plurality of pixel circuits; a voltage control unit (20) that controls the voltage applied to the photoelectric conversion member (111); and a correction unit (15) that performs correction to reduce the dark current signal component for at least a part of the 1 or more frame images output from the image pickup device (10).

Description

Imaging device, camera, and imaging method
Technical Field
The present disclosure relates to an imaging apparatus, a camera, and an imaging method for imaging an image.
Background
Conventionally, an imaging device is known which uses an imaging element to capture a video image composed of a plurality of consecutive frame images.
In such an imaging device, a technique for reducing a dark current signal component included in a frame image is proposed.
For example, patent document 1 describes an imaging device in which a lens diaphragm is contracted so as to be in a light-shielded state, a dark current signal component in the light-shielded state is acquired and stored, and the stored dark current signal component is used to perform correction so as to reduce the dark current signal component included in an imaged frame image.
Prior art documents
Patent document
Patent document 1: japanese patent application laid-open No. 4292751
Disclosure of Invention
Problems to be solved by the invention
In the conventional imaging apparatus, in order to capture consecutive frames, the lens diaphragm needs to be opened during the imaging period. In contrast, in the imaging device, in order to obtain the dark current signal component in the light-shielded state, it is necessary to contract the lens diaphragm to be in the light-shielded state for a certain period of time. That is, in this imaging apparatus, the dark current signal component in the light-shielded state cannot be acquired in the imaging period of the continuous frame image.
Therefore, in this imaging apparatus, it is impossible to perform correction so as to reduce the dark current signal component included in the frame image imaged in the imaging period based on the dark current signal component in the light-shielded state acquired in the imaging period of the continuous frame image.
Accordingly, an object of the present disclosure is to provide an imaging apparatus, a camera, and an imaging method, which can perform correction to reduce a dark current signal component included in a frame image imaged in an imaging period of consecutive frame images, based on the dark current signal component in a light-shielded state acquired in the imaging period.
Means for solving the problems
An imaging device according to an aspect of the present disclosure includes: an image pickup element including a photoelectric conversion unit that generates charge based on an internal photoelectric effect by receiving light in a state where a voltage of a 1 st predetermined range is applied, and that does not generate charge based on the internal photoelectric effect even if the light is received in a state where the voltage of a 2 nd predetermined range is applied, and a plurality of pixel circuits that accumulate the charge generated by the photoelectric conversion unit on a pixel-by-pixel basis, the image pickup element outputting a frame image based on an amount of charge accumulated by the plurality of pixel circuits; a voltage control section that controls a voltage applied to the photoelectric conversion member; and a correction unit that performs correction to reduce a dark current signal component for at least a part of the 1 or more frame images output from the image pickup device; the voltage control unit performs the control as follows: the image pickup device includes an image pickup device that applies a voltage of the 1 st predetermined range to the photoelectric conversion device in an exposure period that is a part of a predetermined frame period, and applies a voltage of the 2 nd predetermined range to the photoelectric conversion device in a light shielding period other than the exposure period in the frame period, and the image pickup device outputs a signal frame image based on an amount of electric charge accumulated in the plurality of pixel circuits in the exposure period and a light shielding frame image based on an amount of electric charge accumulated in the plurality of pixel circuits in the light shielding period for each of the frame periods, and the electric-charge amount correction unit performs the correction using the light shielding frame image output from the image pickup device with respect to the signal frame image output from the image pickup device.
A camera according to an aspect of the present disclosure includes the imaging device and a lens that condenses external light to the imaging element.
An image pickup method according to one aspect of the present disclosure is an image pickup method performed by an image pickup apparatus including an image pickup device including a photoelectric conversion unit that generates charge based on an internal photoelectric effect by receiving light in a state where a voltage of a 1 st predetermined range is applied, and a plurality of pixel circuits that accumulate the charge generated by the photoelectric conversion unit in units of pixels without generating the charge based on the internal photoelectric effect even when receiving light in a state where a voltage of a 2 nd predetermined range is applied; the image pickup method includes: an image pickup step in which the image pickup element outputs a frame image based on the amount of charge accumulated in the plurality of pixel circuits; a voltage control step of controlling a voltage applied to the photoelectric conversion member by the voltage control section; a correction step of performing correction to reduce a dark current signal component with respect to at least a part of 1 or more frame images output from the image pickup element, wherein the voltage control step includes the voltage control unit performing the control as follows: the image pickup device includes an image pickup device that applies a voltage of the 1 st predetermined range to the photoelectric conversion device during an exposure period that is a part of a predetermined frame period, and applies a voltage of the 2 nd predetermined range to the photoelectric conversion device during a light shielding period other than the exposure period in the frame period, and in the image pickup step, the image pickup device outputs a signal frame image based on an amount of electric charge accumulated in the plurality of pixel circuits during the exposure period and a light shielding frame image based on an amount of electric charge accumulated in the plurality of pixel circuits during the light shielding period for each of the frame periods, and in the correction step, the correction unit performs the correction using the light shielding frame image output from the image pickup device with respect to the signal frame image output from the image pickup device.
Effects of the invention
According to the imaging device, the camera, and the imaging method according to the present disclosure, it is possible to perform correction to reduce the dark current signal component included in the frame image imaged in the imaging period based on the dark current signal component in the light-shielded state acquired in the imaging period of the continuous frame image.
Drawings
Fig. 1 is a block diagram showing a configuration of a camera according to an embodiment.
Fig. 2 is a block diagram showing the structure of the image pickup device.
Fig. 3A is a plan view of the photoelectric conversion element.
Fig. 3B is a side view of the photoelectric conversion element.
Fig. 4 is a block diagram showing a configuration of a pixel circuit.
Fig. 5A is a timing chart showing the operation of the voltage control unit.
Fig. 5B is a timing chart showing the operation of the image pickup device.
Fig. 6 is a schematic diagram showing a case where the correction unit performs the correction.
Fig. 7 is a flowchart of the 1 st frame image output process.
Fig. 8 is a flowchart of the 1 st correction process.
Fig. 9 is a block diagram showing a configuration of a camera according to modification 1.
Fig. 10A is a timing chart showing the operation of the voltage control unit.
Fig. 10B is a timing chart showing the operation of the image pickup device.
Fig. 11 is a schematic diagram showing a case where the correction unit performs the correction.
Fig. 12 is a flowchart of the 2 nd frame image output processing.
Fig. 13 is a flowchart of the 2 nd correction processing.
Fig. 14 is a block diagram showing a configuration of an imaging device according to modification 2.
Fig. 15 is a schematic diagram showing a case where the addition shading frame image is generated by the addition image generating unit.
Fig. 16 is a schematic diagram showing a case where the correction unit performs the correction.
Fig. 17 is a flowchart of the 3 rd correction process.
Fig. 18 is a block diagram showing a configuration of a camera according to modification 3.
Fig. 19 is a schematic diagram showing a case where the addition average light-shielding frame image is generated by the addition average image generator.
Fig. 20 is a schematic diagram showing a case where the correction unit performs the correction.
Fig. 21 is a flowchart of the 4 th correction process.
Fig. 22 is a block diagram showing a configuration of a camera according to modification 4.
Fig. 23A is a timing chart showing the operation of the voltage control unit.
Fig. 23B is a timing chart showing the operation of the image pickup device.
Fig. 24 is a schematic diagram showing a case where the correction unit performs the correction.
Fig. 25 is a flowchart of the 3 rd frame image output processing.
Fig. 26 is a flowchart of the 5 th correction process.
Fig. 27A is a perspective view of a digital camera according to a modification.
Fig. 27B is an oblique view of a camera according to a modification.
Detailed Description
The embodiments are described in detail below. The embodiments to be described below are all embodiments showing preferred specific examples of the present disclosure. The numerical values, shapes, materials, constituent elements, arrangement positions and connection modes of the constituent elements, steps, order of the steps, and the like shown in the following embodiments are merely examples, and do not limit the present disclosure. The present disclosure is limited only by the claims. Therefore, among the components in the following embodiments, components not described in the independent claims showing the present disclosure are not necessarily required to achieve the object of the present disclosure, but are described as a more preferable configuration.
(embodiment mode)
Here, an image pickup apparatus 1 that picks up an image is explained with reference to the drawings.
[1-1. constitution ]
Fig. 1 is a block diagram showing a configuration of a camera 200 according to an embodiment.
The camera 200 includes a lens barrel 230 and an imaging device 1. The lens barrel 230 includes an optical system 210 and a lens driving unit 220.
The optical system 210 is constituted by 1 or more lenses that condense external light onto the imaging element 10 of the imaging device 1. Specifically, the optical system 210 is configured by a zoom lens 211, a camera-shake correction lens 212, a focus lens 213, and a diaphragm 214. By moving the zoom lens 211 along the optical axis 210A, the subject image can be enlarged or reduced. In addition, by moving the focus lens 213 along the optical axis 210A, the focal length of the object image can be adjusted. In addition, the handshake correction lens 212 is movable in a plane perpendicular to the optical axis 210A of the optical system 210. By moving the hand shake correction lens 212 in a direction to cancel out the shake of the camera 200, the influence of the shake of the camera 200 on the captured image can be reduced. The diaphragm 214 has an opening 214A located on the optical axis 210A, and the amount of transmitted light is adjusted by setting by a user or automatically adjusting the size of the opening 214A.
The lens driving section 220 includes a zoom actuator that drives the zoom lens 211, a handshake correction actuator that drives the handshake correction lens 212, a focus actuator that drives the focus lens 213, and a diaphragm actuator that drives the diaphragm 214. The lens driving unit 220 controls the zoom actuator, the focus actuator, the camera shake correction actuator, and the diaphragm actuator.
The imaging apparatus 1 includes an imaging device 10, a correction unit 15, a voltage control unit 20, a control unit 50, an analog-to-digital converter 250, an image processing unit 260, a memory 270, a card slot 290, an internal memory 340, an operation member 310, and a display monitor 320.
The image pickup device 10 outputs a signal frame image (described later) and a light-shielding frame image (described later) at a predetermined frame period T1 (for example, 1/60 seconds).
The analog-to-digital converter 250 performs analog gain improvement on analog image data generated by the image pickup element 10, and converts the analog image data into digital image data as a digital signal.
The correction unit 15 performs correction to reduce the dark current signal component for at least a part of 1 or more frame images (in this case, digital image data converted by the analog-to-digital converter 250) output from the image pickup device 10. The correction unit 15 is realized by a processor (not shown) executing a program stored in a memory (not shown), for example.
The voltage control unit 20 controls a voltage applied to a photoelectric conversion member 111 (described later) included in the image pickup element 10. The voltage control unit 20 is realized by a processor (not shown) executing a program stored in a memory (not shown), for example.
The image processing unit 260 performs various processes on the image data generated by the image pickup device 10 (here, the frame image corrected by the correction unit 15) to generate image data for display on the display monitor 320 or image data for storage in the memory card 300. For example, the image processing unit 260 performs various processes such as gamma correction and white balance correction on the image data generated by the image pickup device 10. The image processing unit 260 compresses the image data generated by the image pickup device 10 in a compression format based on the h.264 standard or the MPEG2 standard. The image processing unit 260 is realized by a processor (not shown) executing a program stored in a memory (not shown), for example.
The control unit 50 controls the entire camera 200. The control unit 50 is realized, for example, by a processor (not shown) in the control unit 50 expanding and executing a program recorded in the internal memory 340 in the memory 270 that temporarily stores the program.
The memory 270 functions as a work memory for the image processing unit 360 and the control unit 50. The memory 270 can be implemented by, for example, DRAM, SRAM, or the like.
Card slot 390 detachably holds memory card 300. Card slot 290 is capable of mechanically and electrically connecting with memory card 300. The memory card 300 includes a nonvolatile flash memory, a ferroelectric memory, and the like therein, and can store data such as an image file generated by the image processing unit 260.
The internal memory 340 is configured by a nonvolatile flash memory, a ferroelectric memory, or the like. The internal memory 340 stores a control program and the like for controlling the entire camera 200.
The operation section 310 is a generic term of a user interface that accepts an operation from a user. The operation member 310 includes, for example, a cross key or a confirmation button for receiving an operation from a user.
The display monitor 320 has a screen 320A capable of displaying an image represented by image data generated by the image pickup device 10 or an image represented by image data read from the memory card 300. The display monitor 320 may display various menu screens for performing various settings of the camera 200 on the screen 320A. A touch panel 320B is disposed on the screen 320A of the display monitor 320. The touch panel 320B can accept various touch operations by being touched by a user. The control unit 50 is notified of an instruction indicated by a touch operation on the touch panel 320B to perform various processes.
The imaging element 10, the correction unit 15, and the voltage control unit 20 among the above-described components constituting the imaging apparatus 1 will be described in further detail below with reference to the drawings.
Fig. 2 is a block diagram showing the structure of the image pickup device 10.
As shown in the figure, the image pickup device 10 includes a photoelectric conversion element 110, a pixel circuit array 120, a readout circuit 130, an output circuit 140, a row scanning circuit 150, a timing control circuit 160, and a voltage application circuit 170.
Fig. 3A is a plan view of the photoelectric conversion element 110, and fig. 3B is a side view of the photoelectric conversion element 110.
As shown in fig. 3A and 3B, the photoelectric conversion element 110 includes a film-like photoelectric conversion member 111, an upper transparent electrode 112 closely bonded to the upper surface of the photoelectric conversion member 111, and N × M lower pixel electrodes 113 closely bonded to the lower surface of the photoelectric conversion member 111 and arranged in a two-dimensional array of N rows and M columns (N, M is an integer of 1 or more).
The photoelectric conversion part 111 generates electric charges based on the internal photoelectric effect by receiving light in a state where a voltage of a 1 st predetermined range excluding 0V is applied, and does not generate electric charges based on the internal photoelectric effect even if receiving light in a state where a voltage of a second predetermined range including 0V is applied.
Here, the photoelectric conversion member 111 is described as an organic thin film having the above-described characteristics. That is, in this embodiment, a case where the image pickup element 10 is an organic CMOS (complementary metal oxide semiconductor) image sensor having an organic thin film as a photoelectric conversion member is taken as an example.
The upper transparent electrode 112 is a transparent electrode to which a voltage is applied so that a potential difference including "0" is generated with respect to the lower surface of the photoelectric conversion member 111 as a whole.
The lower pixel electrodes 113 are electrodes arranged in a two-dimensional array of N rows and M columns so as to cover the entire lower surface of the photoelectric conversion member 111.
In the case where a voltage that generates a positive potential difference with respect to the lower surface is applied to the upper surface of the photoelectric conversion section 111, the lower pixel electrode 113 accumulates positive charges among the charges generated in the vicinity of itself when the charges are generated by the photoelectric conversion section 111.
The photoelectric conversion element 110 configured as described above is: the lower pixel electrodes 113 each collect positive charges based on the internal photoelectric effect caused by light reception under the condition that a voltage that generates a positive potential difference in a range in which the internal photoelectric effect can occur with respect to the lower surface is applied to the upper surface of the photoelectric conversion section 111. That is, the photoelectric conversion element 110 enters an exposure state under this condition. In contrast, the upper surface of the photoelectric conversion member 111 does not generate electric charges due to the internal photoelectric effect even when receiving light under the condition that the upper surface and the lower surface have substantially the same electric potential, and therefore the lower pixel electrode 113 does not collect electric charges. That is, the photoelectric conversion element 110 enters a light-shielding state under this condition.
Hereinafter, a period in which a voltage that generates a positive potential difference in a range in which an internal photoelectric effect can occur with respect to a lower surface is applied to an upper surface of the photoelectric conversion member 111 is referred to as an "exposure period"; a period in which a voltage in a range in which no internal photoelectric effect occurs with respect to the lower surface (here, a voltage having substantially the same potential as the lower surface) is applied to the upper surface of the photoelectric conversion member 111 is referred to as a "light-shielding period".
Returning again to fig. 2, the description of the image pickup element 10 is continued.
The pixel circuit array 120 is a semiconductor device in which N × M pixel circuits 21 are arranged in a two-dimensional array of N rows and M columns, and is arranged on the lower surface side of the photoelectric conversion element 110 so as to overlap the photoelectric conversion element 110.
In the pixel circuit array 120, when the image pickup device 10 is viewed in a plan view, the pixel circuits 21 are arranged so that the positions of the pixel circuits 21 overlap the positions of the lower pixel electrodes 113 in a one-to-one correspondence.
Fig. 4 is a block diagram showing the configuration of the pixel circuit 21.
As shown in the figure, the pixel circuit 21 is configured to include a reset transistor 22, an amplification transistor 23, a selection transistor 24, and a charge storage node 25.
The charge accumulation node 25 is connected to the lower pixel electrode 113 corresponding to the pixel circuit 21 to which it belongs, the source of the reset transistor 22, and the gate of the amplifying transistor 23, respectively, and accumulates positive charges accumulated by the connected lower pixel electrode 113.
The reset transistor 22 has a gate connected to a reset signal line 51, a drain to which a reset voltage VRST is supplied, and a source to which the charge storage node 25 is connected.
The reset transistor 22 is turned on by a reset signal transmitted from a row scanning circuit 150 (described later) via a reset signal line 51, and thereby resets (initializes) the amount of charge stored in the charge storage node 25.
The amplifier transistor 23 has a gate connected to the charge storage node 25, a drain to which the power supply voltage VDD is supplied, and a source connected to the drain of the selection transistor 24.
A voltage corresponding to the charge accumulated in the charge accumulation node 25 is applied to the gate of the amplification transistor 23.
Therefore, when the selection transistor 24 is in the on state, the amplification transistor 23 functions as a current source that outputs a current corresponding to the charge stored in the charge storage node 25.
The selection transistor 24 has a gate connected to the selection signal line 52, a drain connected to the source of the amplification transistor 23, and a source connected to the vertical signal line 32.
The selection transistor 24 is turned on by a selection signal supplied from a row scanning circuit 150 (described later) via a selection signal line 52, and outputs a current flowing through the amplification transistor 23 to the vertical signal line 32.
As will be described later, the amount of current of the current output to the vertical signal line 32 is detected by the column readout circuit 31 (described later), and the amount of charge accumulated in the charge accumulation node 25 of the pixel circuit 21 including the selection transistor 24 turned on by the selection signal is read out in accordance with the detected amount.
With the above configuration, the pixel circuit 21 accumulates the charge generated by the photoelectric conversion section 111 to the charge accumulation node 25 in units of pixels. The charge amount accumulated in the charge accumulation node 25 is nondestructively read out.
The description of the image pickup device 10 is continued with reference to fig. 2.
The row scanning circuit 150 has the following accumulated charge amount reset function and the following readout pixel circuit selection function.
The accumulated charge amount reset function is: in the pixel circuit array 120, from the row (row 1) farthest from the readout circuit 130, the reset signal for resetting the positive charges accumulated in the charge accumulation nodes 25 of the pixel circuits 21 belonging to the row is sequentially transferred line by line to the row (row N) closest to the readout circuit 130 via the reset signal lines 51 connected to the pixel circuits 21 belonging to the row.
Accordingly, the reset of the electric charges stored in the electric charge storage nodes 25 of all the pixel circuits 21 included in the pixel circuit array 120 is performed in units of rows in order from the 1 st row to the N-th row.
The readout pixel circuit selection function is: in the pixel circuit array 120, the selection signal for turning on the selection transistor 24 of each of the pixel circuits 21 belonging to the row is sequentially transmitted from the 1 st row to the N th row through the selection signal line 52 connected to each of the pixel circuits 21 belonging to the row.
Accordingly, the readout of the charge amount accumulated in the charge accumulation nodes 25 of all the pixel circuits 21 included in the pixel circuit array 120 is performed sequentially in units of rows from the 1 st row to the N-th row.
The readout circuit 130 reads out the charge amounts accumulated in the respective pixel circuits 21 constituting the pixel circuit array 120.
The readout circuit 130 includes M column readout circuits 31 corresponding to M columns of the pixel circuit array 120.
The column readout circuit 31 detects the amount of current flowing through the amplification transistor 23 of the pixel circuit 21 including the selection transistor 24 turned on by the selection signal (the pixel circuit 21 is also referred to as a "readout target pixel circuit 21") via the vertical signal line 32 connected to each of the pixel circuits 21 belonging to the corresponding column, thereby reading out the amount of charge accumulated in the charge accumulation node 25 of the readout target pixel circuit 21, and outputs a digital signal of K bits (K is a positive integer, for example, 8) indicating the read-out amount of charge as the pixel value of the readout target pixel circuit 21.
The output circuit 140 outputs a frame image composed of the pixel values output from the column readout circuit 31 to the outside.
The frame image output by the output circuit 140 includes a signal frame image based on the amount of charge accumulated in the charge accumulation node 25 of each pixel circuit 21 during a period in which the photoelectric conversion element 110 is in the exposure state, and a light-shielding frame image based on the amount of charge accumulated in the charge accumulation node 25 of each pixel circuit 21 during a period in which the photoelectric conversion element 110 is in the light-shielding state.
While the photoelectric conversion element 110 is in the light-shielding state, the electric charges stored in the charge storage nodes 25 of the respective pixel circuits 21 are dark current components of the respective pixel circuits 21. Therefore, the light-shielded frame image is a frame image composed of the dark current component of each pixel circuit 21.
The voltage applying circuit 170 applies a voltage to the photoelectric conversion member 111. More specifically, the voltage application circuit 170 controls the voltage applied to the upper transparent electrode 112, and (1) applies a 1 st voltage in a 1 st predetermined range that generates a positive potential difference causing an internal photoelectric effect with respect to the lower surface to the upper surface of the photoelectric conversion portion 111, thereby causing the photoelectric conversion element 110 to be in an exposure period during the applied state; (2) the 2 nd voltage in the 2 nd predetermined range, which generates "a potential difference of a positive potential difference not generating the internal photoelectric effect" (here, the same potential as the lower surface) with respect to the lower surface, is applied, and thereby the photoelectric conversion element 110 is in the light-shielding period during the applied state.
The timing control circuit 160 controls the operation timing of the line scanning circuit 150, the operation timing of the readout circuit 130, the operation timing of the voltage application circuit 170, and the operation timing of the output circuit 140. That is, the timing control circuit 160 controls the timing at which the row scanning circuit 150 executes the stored-charge-amount reset function and the timing at which the readout pixel circuit selection function is executed, controls the timing at which the readout circuit 130 reads out the charge amount stored in the charge storage node 25 of the pixel circuit 21 selected by the selection signal, controls the timing at which the voltage application circuit 170 puts the photoelectric conversion element 110 in the exposure period and the timing at which the photoelectric conversion element 110 is put in the light-shielding period, and controls the timing at which the output circuit 140 outputs the frame image.
More specifically, upon receiving a frame start signal (described later) from the voltage control unit 20, the timing control circuit 160 controls the operation timings of the row scanning circuit 150, the readout circuit 130, the voltage application circuit 170, and the output circuit 140 to (1) read out the charge amount accumulated in the charge accumulation nodes 25 of all the pixel circuits 21 included in the pixel circuit array 120; (2) outputting a frame image based on the read-out charge amount to the outside as a light-shielded frame image; (3) resetting the amount of charge accumulated in the charge accumulation nodes 25 of all the pixel circuits 21 included in the pixel circuit array 120; (4) application of the 1 st voltage to the photoelectric conversion section 111 is started.
When receiving an exposure/light-shielding switching signal (described later) from the voltage control unit 20, the timing control circuit 160 controls the operation timings of the line scanning circuit 150, the readout circuit 130, the voltage application circuit 170, and the output circuit 140 to (1) read out the charge amount accumulated in the charge accumulation nodes 25 of all the pixel circuits 21 included in the pixel circuit array 120; (2) outputting a frame image based on the read-out amount of electric charge to the outside as a signal frame image; (3) resetting the amount of charge accumulated in the charge accumulation nodes 25 of all the pixel circuits 21 included in the pixel circuit array 120; (4) application of the 2 nd voltage to the photoelectric conversion section 111 is started. Returning again to fig. 1, the description of the image pickup apparatus 1 is continued.
The voltage control unit 20 controls the voltage applied to the photoelectric conversion part 111 so that a 1 st voltage in a 1 st predetermined range is applied to the photoelectric conversion part 111 in an exposure period which is a part of the frame period T1 and a 2 nd voltage in a 2 nd predetermined range is applied to the photoelectric conversion part 111 in a light shielding period which does not include the exposure period in the frame period T1, for each predetermined frame period T1. More specifically, the image pickup device 10 outputs a frame start signal indicating that a new frame period is started in a frame period T1, and further outputs an exposure/light-shielding switching signal indicating that the voltage applied to the photoelectric conversion portion 111 is switched from the 1 st voltage to the 2 nd voltage in a frame period T1 that is delayed by a predetermined period T2 from the output of the frame start signal, thereby controlling the voltage applied to the photoelectric conversion portion 111.
Fig. 5A is a timing chart of the frame start signal and the exposure/light-shielding switching signal outputted from the voltage control unit 20. Fig. 5B is a timing chart showing the operation of the image pickup device 10 when receiving the frame start signal and the exposure/light-shielding switching signal from the voltage control unit 20.
As shown in fig. 5A, the voltage control unit 20 outputs the frame start signal and the exposure/light-shielding switching signal to the image pickup device 10 at a timing delayed from the frame start signal by a predetermined period T2(T2< T1) for each frame period T1.
As shown in fig. 5B, when the frame start signal and the exposure/light-shielding switching signal output for each frame period T1 are received from the voltage control unit 20, the image pickup device 10 applies the 1 st voltage to the photoelectric conversion section 111 during a period from when the frame start signal is received until when the exposure/light-shielding switching signal is received next, and applies the 2 nd voltage to the photoelectric conversion section 111 during a period from when the exposure/light-shielding switching signal is received until when the frame start signal is received next.
Therefore, the photoelectric conversion element 110 is in the exposure state during a period from when the frame start signal is received until when the exposure/light-shielding switching signal is received next, and is in the light-shielding state during a period from when the exposure/light-shielding switching signal is received until when the frame start signal is received next.
The image pickup device 10 reads the charge amount accumulated in the charge accumulation node 25 of each pixel circuit 21 in the exposure period in which the photoelectric conversion element 110 is in the exposure state, outputs a signal frame image based on the read charge amount, reads the charge amount accumulated in the charge accumulation node 25 of each pixel circuit 21 in the light-shielding period in which the photoelectric conversion element 110 is in the light-shielding state, and outputs a light-shielding frame image based on the read charge amount.
Returning again to fig. 1, the description of the image pickup apparatus 1 is continued.
The correction unit 15 performs correction to reduce the dark current signal component by using the light-shielded frame image output from the image sensor 10 with respect to the signal frame image output from the image sensor 10.
Here, the correction unit 15 performs correction on a signal frame image to be corrected (hereinafter, referred to as "correction target signal frame image") using a light-shielding frame image (hereinafter, referred to as "correction light-shielding frame image") output in the same frame period as the frame period in which the correction target signal frame image is output.
Fig. 6 is a schematic diagram showing a case where the correction unit 15 performs the correction.
In fig. 6, white dots included in the correction target signal frame image and the correction light-shielding frame image schematically represent dark current signal components of the pixel. As an example, a dark current signal component sometimes called "white defect" appears as white spots on the frame image.
In general, the signal frame image to be corrected is an image in which a dark current signal component is superimposed on a subject image, and the light shielding frame image for correction is an image composed of the dark current signal component.
First, the correction unit 15 performs normalization processing on the correction light-shielded frame image to generate a normalized light-shielded frame image. Here, the normalization processing is processing for normalizing the period (light-shielding period Ts) of the light-shielded state of the correction light-shielded frame image by the period (exposure period Tr) of the exposure state of the correction target signal frame image. More specifically, the correction unit 15 multiplies each pixel value of the correction light-shielding frame image by a value (exposure period Tr/light-shielding period Ts) to generate a normalized light-shielding frame image. Accordingly, the dark current signal component of the normalized light-shielding frame image is (Tr/Ts) times the dark current signal component of the correction light-shielding frame image. Embodiment 1 is an example of a case where Tr > Ts, as shown in fig. 5B. Therefore, as shown in fig. 6, the luminance of the so-called "white defect" of the normalized shading frame image is larger than the luminance of the so-called "white defect" of the shading frame image for correction.
The correction unit 15 subtracts the pixel value of each pixel of the corresponding normalized light-shielded frame image from the pixel value of each pixel of the correction target signal frame image, thereby generating a corrected signal frame image.
By performing the above-described processing, the correction unit 15 generates a corrected signal frame image in which the dark current signal component is reduced, from the correction target signal frame image.
Hereinafter, operations performed by the imaging device 1 configured as described above will be described with reference to the drawings.
[1-2. actions ]
The imaging apparatus 1 performs 1 st frame image output processing and 1 st correction processing as characteristic operations thereof.
These processes are explained below in order.
[1-2-1. 1 st frame image output processing ]
The 1 st frame image output process is a process in which the image pickup device 10 alternately outputs a signal frame image and a light-shielded frame image at a predetermined frame period T1. Hereinafter, the description will be given with respect to the application voltage to the photoelectric conversion portion 111 as the 2 nd voltage (here, 0V, for example) in the initial state before the 1 st frame image output process is started.
The 1 st frame image output processing is started by receiving an operation to start moving image capturing by a user using the imaging apparatus 1.
Fig. 7 is a flowchart of the 1 st frame image output process.
When the 1 st frame image output process is started, the voltage control unit 20 outputs a frame start signal to the image pickup device 10 (step S5).
Upon receiving the frame start signal output from the voltage control unit 20, the image pickup device 10 reads the charge amount accumulated in the charge accumulation node 25 of each pixel circuit 21 included in the pixel circuit array 120 (step S10), and outputs a light-shielded frame image based on the read charge amount (step S15).
Then, the image pickup element 10 changes the applied voltage to the photoelectric conversion part 111 from the 2 nd voltage to the 1 st voltage (here, 10V, for example) (step S20).
Then, the state of the photoelectric conversion element 110 is switched from the light-shielding state to the exposure state (step S25).
Then, the image pickup element 10 resets the charge amount accumulated in the charge accumulation node 25 of each pixel circuit 21 included in the pixel circuit array 120 (step S30).
On the other hand, when a predetermined period T2 elapses from the previous frame start signal output (no in step S35 and yes in step S35 are repeated), the voltage control unit 20 outputs an exposure/light-shielding switching signal to the image pickup device 10 (step S40).
Upon receiving the exposure/light-shielding switching signal output from the voltage control unit 20, the imaging element 10 reads the charge amount accumulated in the charge accumulation node 25 of each pixel circuit 21 included in the pixel circuit array 120 (step S45), and outputs an exposure frame image based on the read charge amount (step S50).
Then, the image pickup device 10 changes the voltage applied to the photoelectric conversion part 111 from the 1 st voltage to the 2 nd voltage (step S55).
Then, the state of the photoelectric conversion element 110 is switched from the exposure state to the light-shielding state (step S60).
Then, the image pickup element 10 resets the charge amount accumulated in the charge accumulation node 25 of each pixel circuit 21 included in the pixel circuit array 120 (step S65).
On the other hand, when a predetermined time T1 has elapsed since the previous frame start signal was output (no in step S70 and yes in step S70), the image pickup device 10 proceeds to the process of step S5 and repeats the subsequent processes.
[1-2-2. 1 st correction processing ]
The 1 st correction process is a process in which the correction section 15 performs correction to reduce a dark current signal component to be corrected, using the light-shielded frame image output from the image pickup device 10, with respect to the signal frame image output from the image pickup device 10.
The 1 st correction process is started by outputting the first light-shielded frame image from the image pickup device 10 in the first frame image output process described above.
Fig. 8 is a flowchart of the 1 st correction process.
When the 1 st correction process is started, the correction unit 15 waits until a signal frame image is output from the image pickup device 10. When the signal frame image is output while waiting for the signal frame image to be output (step S100 is repeated and yes is obtained after no in step S100), the correction unit 15 acquires the signal frame image (step S110).
When the signal frame image is acquired, the correction unit 15 waits until the light-shielded frame image is output from the image sensor 10. When the light-shielded frame image is output while waiting for the light-shielded frame image to be output (step S120 is repeated and yes is obtained after no in step S120), the correcting unit 15 acquires the light-shielded frame image (step S130).
Then, the correcting unit 15 performs normalization processing on the acquired light-shielded frame image to generate a normalized light-shielded frame image (step S140). That is, the correction unit 15 multiplies each pixel value of the acquired light-shielded frame image by a value of (Tr/Ts) to generate a normalized light-shielded frame image.
When the normalized light-shielded frame image is generated, the correction unit 15 subtracts the pixel value of each pixel of the corresponding normalized light-shielded frame image from the pixel value of each pixel of the acquired signal frame image (step S150), thereby generating a corrected signal frame image (step S160).
When the process of step S160 is completed, the correction unit 15 proceeds to the process of step S100, and repeats the subsequent processes.
[1-3. Effect, etc. ]
As described above, the image pickup apparatus 1 performs correction for reducing the dark current signal component using the light-shielded frame image output from the image pickup device 10 in the same frame period with respect to the signal frame image output from the image pickup device 10 in the predetermined frame period T1.
Generally, the dark current signal component varies depending on the temperature of the image pickup element or the like. Therefore, if the temperature of the image pickup device changes during the image pickup period of the continuous frame image, the dark current signal component changes. Therefore, according to the imaging device 1 of the present embodiment, the dark current signal component included in the frame image captured in the imaging period can be reduced more accurately than in the conventional imaging device in which the dark current signal component in the imaging period of the continuous signal frame image cannot be obtained.
(modification 1)
Here, an imaging apparatus according to modification 1 in which a part of functions of the imaging apparatus 1 according to the embodiment are changed will be described with reference to the drawings.
The example of the imaging apparatus 1 according to the embodiment is configured such that the correction unit 15 multiplies each pixel value of the correction light-shielding frame image by the value of (exposure period Tr)/(light-shielding period Ts) to generate a normalized light-shielding frame image, and subtracts each pixel value of the corresponding normalized light-shielding frame image from each pixel value of the correction target signal frame image to generate a corrected signal frame image.
In contrast, the imaging apparatus according to the modification 1 is configured such that the exposure period Tr is set to be equal to the light-shielding period Ts, and the correction unit directly subtracts each pixel of the corresponding correction light-shielding frame image from each pixel value of the correction target signal frame image, thereby generating a corrected signal frame image.
The following description will be made of the imaging device according to modification 1, focusing on modifications to the imaging device 1 according to the embodiment, with reference to the drawings.
[2-1. constitution ]
Fig. 9 is a block diagram showing a configuration of a camera 900 according to modification 1.
As shown in the figure, the camera 900 is modified from the camera 200 according to the embodiment in that the imaging device 1 is changed to the imaging device 2. In the imaging apparatus 2, the correction unit 15 is changed to the correction unit 915 and the voltage control unit 20 is changed to the voltage control unit 920, compared with the imaging apparatus 1 according to the embodiment.
The voltage control unit 920 is modified in comparison with the voltage control unit 20 according to the embodiment such that a part of functions thereof is changed.
The voltage control unit 20 according to the embodiment is configured to control the voltage applied to the photoelectric conversion member 111 by outputting a frame start signal to the image pickup device 10 in a frame period T1 and further outputting an exposure/light-shielding switching signal in a frame period T1 that is delayed by a predetermined period T2 from the output of the frame start signal.
On the other hand, the voltage control unit 920 outputs a frame start signal to the image pickup device 10 in the frame period T1, and further outputs an exposure/light-shielding switching signal in the frame period T1 delayed by T1/2 from the output of the frame start signal, thereby controlling the voltage applied to the photoelectric conversion portion 111.
Fig. 10A is a timing chart of the frame start signal and the exposure/light-shielding switching signal output from the voltage control unit 920. Fig. 10B is a timing chart showing the operation of the image pickup device 10 when receiving the frame start signal and the exposure/light-shielding switching signal from the voltage control unit 920.
As shown in fig. 10A, the voltage control unit 920 outputs the frame start signal and the exposure/shading switching signal to the image pickup device 10 at a timing delayed by T1/2 from the frame start signal every frame period T1.
As shown in fig. 10B, when the frame start signal and the exposure/light-shielding switching signal output for each frame period T1 are received from the voltage control unit 920, the image pickup device 10 applies the 1 st voltage to the photoelectric conversion section 111 during a period from when the frame start signal is received until when the exposure/light-shielding switching signal is received next, and applies the 2 nd voltage to the photoelectric conversion section 111 during a period from when the exposure/light-shielding switching signal is received until when the frame start signal is received next.
Therefore, the photoelectric conversion element 110 is in the exposure state during a period from the reception of the frame start signal until T1/2 elapses, and is in the light-shielding state during a period from the elapse of T1/2 until the reception of the frame start signal follows.
The image pickup device 10 reads the amount of charge accumulated in the charge accumulation node 25 of each pixel circuit 21 in the exposure period in which the photoelectric conversion element 110 is in the exposure state, outputs a signal frame image based on the read amount of charge, reads the amount of charge accumulated in the charge accumulation node 25 of each pixel circuit 21 in the light-shielding period in which the photoelectric conversion element 110 is in the light-shielding state, and outputs a signal frame image based on the read amount of charge.
Returning to fig. 9, the description of the image pickup apparatus 2 is continued.
The correction unit 915 is modified in such a manner that its partial functions are changed as compared with the correction unit 15 according to the embodiment.
The example of the correction unit 15 according to the embodiment is configured to generate a normalized light-shielded frame image by multiplying each pixel value of the correction light-shielded frame image by the value of (exposure period Tr)/(light-shielded period Ts), and to generate a corrected signal frame image by subtracting the pixel value of each pixel of the corresponding normalized light-shielded frame image from each pixel value of the correction target signal frame image.
In contrast, the correction unit 915 is configured to generate a corrected signal frame image by directly subtracting each pixel of the corresponding correction mask frame image from each pixel of the correction target signal frame image.
Fig. 11 is a schematic diagram showing a case where the correction unit 915 performs the correction.
As shown in the figure, the correction unit 915 subtracts the pixel value of the corresponding correction light-shielding frame image from the pixel value of the pixel of the correction target signal frame, thereby generating a corrected signal frame image.
In the imaging device 2, the exposure period Tr is set to be equal to the light-shielding period Ts. Therefore, the correction unit 915 does not need to perform the initialization process of initializing the light-shielding period Ts by the exposure period Tr in each frame period for the correction light-shielding frame image as in the correction unit 15 according to the embodiment.
Hereinafter, operations performed by the imaging device 2 configured as described above will be described with reference to the drawings.
[2-2. actions ]
The imaging device 2 performs the 2 nd frame image output process and the 2 nd correction process as characteristic operations thereof.
These processes are explained below in order.
[2-2-1. 2 nd frame image output processing ]
The 2 nd frame image output processing is processing in which the image pickup device 10 alternately outputs the signal frame image and the light-shielded frame image with a phase shift of T1/2 from each other in a predetermined frame period T1.
The 2 nd frame image output processing is processing in which a part of the processing is changed from the 1 st frame image output processing according to the embodiment.
Fig. 12 is a flowchart of the 2 nd frame image output processing.
In the same figure, the processing of step S1205 to the processing of step S1230, and the processing of step S1240 to the processing of step S1270 are processing of replacing the voltage controller 20 with the voltage controller 920 for the processing of step S5 to the processing of step S30, and the processing of step S40 to the processing of step S79 in the image output processing of frame 1 according to the embodiment (see fig. 7), respectively.
Therefore, the processing of step S1205 to step S1230 and the processing of step S1240 to step S1270 have already been described here, and the processing of step S1235 and the processing before and after this are described.
After the process of step S1230 is completed, the voltage control unit 920 outputs an exposure switching signal to the image pickup device 10 after T1/2 from the previous frame start signal (after no in step S1235 is repeated, yes in step S1235), and (step S1240).
[2-2-2. 2 nd correction processing ]
The 2 nd correction processing is processing in which the correction section 915 directly subtracts each pixel of the corresponding light-shielded frame image from each pixel of the signal frame image output from the image pickup device 10 to correct the signal frame image to be the target so as to reduce the dark current component.
The 2 nd correction processing is processing in which a part of the processing is changed as compared with the 1 st correction processing according to the embodiment.
Fig. 13 is a flowchart of the 2 nd correction processing.
In the same figure, the processing of step S1300 to the processing of step S1330 and the processing of step S1360 are processing in which the correction unit 15 is replaced with the correction unit 915, respectively, with respect to the processing of step S100 to the processing of step S130 in the 1 st correction processing (see fig. 8) according to the embodiment.
In addition, in the 2 nd correction processing, the processing corresponding to step S140 in the 1 st correction processing according to the embodiment is deleted.
Therefore, the processing of step S1300 to step S1330 and the processing of step S1360 have already been described here, and the processing of step S1350 and the processing before and after the processing are described below.
When the light-shielded frame image is acquired in the process of step S1330, the correction unit 915 subtracts the pixel value of the corresponding light-shielded frame image from the pixel value of each pixel of the acquired signal frame image (step S1350), and generates a corrected signal frame image (step S1360).
[2-3. Effect, etc. ]
As described above, in the imaging apparatus 2, the correction unit 915 directly subtracts each pixel value of the corresponding correction light-shielding frame image from each pixel value of the correction target signal frame image, thereby generating a corrected signal frame image.
Therefore, according to the imaging device 2 of the present modification example 1, the correction of the signal frame image can be realized with a smaller amount of calculation than the imaging device 1 of the embodiment.
(modification 2)
An imaging device according to modification 2 in which a part of functions of the imaging device 1 according to the embodiment are changed will be described with reference to the drawings.
The imaging device 1 according to the embodiment is configured to correct a correction target signal frame using a light-shielded frame image output in the same frame period as the frame period in which the correction target signal frame is output.
In contrast, the imaging apparatus according to modification 2 is configured to correct the correction target signal frame using the light-shielded frame images output in a plurality of frame periods including the frame period in which the correction target signal frame is output.
The following description will be made of an imaging device according to modification 2, focusing on modifications to the imaging device 1 according to the embodiment, with reference to the drawings.
[3-1. constitution ]
Fig. 14 is a block diagram showing the configuration of a camera 1400 according to modification 2.
As shown in the figure, the camera 1400 is modified from the camera 200 according to the embodiment such that the imaging device 1 is changed to the imaging device 3. The imaging apparatus 3 is modified from the imaging apparatus 1 according to the embodiment such that the correction unit 1415 is replaced with the correction unit 15, and an additive image generation unit 1417 is added.
The addition image generation unit 1417 generates an addition light-shielded frame image by adding pixel values of pixels corresponding to each other in each light-shielded frame image with respect to a plurality of light-shielded frame images output from the image pickup device 10.
Fig. 15 is a schematic diagram showing a case where the addition shading frame image is generated by the addition image generating unit 1417.
As shown in the figure, when a light-shielded frame image is newly output from the image sensor 10, the additive image generation unit 1417 adds pixel values of pixels corresponding to each other in each light-shielded frame image to the latest n (n is an integer of 2 or more) light-shielded frame images in time series output from the image sensor 10 including the newly output light-shielded frame image, thereby generating an additive light-shielded frame image. Accordingly, the dark current signal components at the same position (i.e., the same pixel) in the n light-shielded frame images are added as the dark current signal components at the position (i.e., the pixel) in the addition light-shielded frame image. Therefore, as illustrated in fig. 15, a so-called "white defect" appearing at the same position in the n light-shielded frame images appears in the added light-shielded frame image as a white defect of a luminance obtained by adding the luminances of the white defects in the respective light-shielded frame images for the position.
On the other hand, random noise ("random components") generated at different positions (i.e., different pixels) in the n-piece shaded frame images are added in an amount of 1 piece in the addition shaded frame image. Therefore, the luminance sufficiently lower than the "white defect" obtained by adding n times appears, and is hardly noticeable. That is, in the addition shading frame image, the "random component" is reduced as compared with the "white defect". Even if random noise occurs at the same position in a plurality of light-shielded frame images among the n light-shielded frame images, since it is small compared with the number of "white defects" generated, it does not pose a problem.
Returning again to fig. 14, the description of the image pickup apparatus 3 is continued.
The correction unit 1415 is modified in such a manner that a part of the functions thereof is changed in addition to the correction unit 15 according to the embodiment.
The correction unit 15 according to the embodiment is configured to correct a signal frame to be corrected using a light-shielded frame image output in the same frame period as the frame period in which the signal frame to be corrected is output.
In contrast, the correction unit 1415 is configured to correct the correction target signal frame by using the addition light-shielded frame image output from the addition image generation unit 1417, which includes the light-shielded frame image output in the same frame period as the frame period in which the correction target signal frame is output, as the addition target.
Fig. 16 is a schematic diagram showing a case where the correction unit 1415 performs the correction.
First, the correction unit 1415 performs normalization processing on the addition shading frame image to generate a normalized addition shading frame image. Here, the normalization processing is processing for normalizing the sum (n × Ts) of the light shielding periods in the addition light shielding frame image by the exposure period (Tr) of the correction target signal frame image. More specifically, the correction unit 1415 generates a normalized addition light-shielded frame image by multiplying each pixel value of the addition light-shielded frame image by a value of (Tr/(n × Ts)). Accordingly, the dark current signal component of the normalized addition light-shielding frame image is (Tr/(n × Ts)) times the dark current signal component of the addition light-shielding frame image. Modification 2 is an example of Tr < (n × Ts). Therefore, as illustrated in fig. 16, the luminance of the so-called "white defect" and the "random component" in the normalized addition light-shielding frame image is smaller than the luminance of the so-called "white defect" and the "random component" in the addition light-shielding frame image. Here, since the luminance of the "random component" is sufficiently lower than that of the "white defect", the absolute value can be reduced by normalization. For example, if a threshold value such that the "random component" is not included in the luminance level subjected to the normalization processing is set, and the luminance equal to or lower than the threshold value is set to "0", the "random component" can be further reduced.
The correction unit 1415 subtracts the pixel value of each pixel of the corresponding normalized addition shading frame from the pixel value of each pixel of the correction target signal frame image, thereby generating a corrected signal frame image.
Hereinafter, operations performed by the imaging device 3 configured as described above will be described with reference to the drawings.
[3-2. actions ]
The imaging device 3 performs 1 st frame image output processing and 3 rd correction processing as characteristic operations thereof.
The 1 st frame image output processing has already been described in the embodiment. Therefore, the 3 rd correction process is explained here.
[3-2-1. 3 rd correction processing ]
The 3 rd correction processing is processing in which the correction unit 1415 performs correction to reduce a dark current signal component with respect to a signal frame image output from the image pickup device 10, using the addition light-shielding frame image output from the addition image generation unit 1417.
This 3 rd correction processing is started by outputting the light-shielded frame image of the n-1 st time from the image pickup element 10 in the 1 st frame image output processing described above.
Fig. 17 is a flowchart of the 3 rd correction process.
When the 3 rd correction processing is started, the correction unit 1415 waits until the signal frame image is output from the image pickup device 10. When the signal frame image is output while waiting for the signal frame image to be output (step S1700 is repeated after no in step S1700), the correction unit 1415 acquires the signal frame image (step S1710).
When the signal frame image is acquired, the correction unit 1415 waits until the addition shading frame image is output from the addition image generation unit 1417. When the addition light-shielding frame image is output while waiting for the output of the addition light-shielding frame image (step S1720 is repeated, and yes is obtained after no in step S1720), the correction unit 1415 acquires the addition light-shielding frame image (step S1730).
Then, the correcting unit 1415 performs normalization processing on the acquired addition light-shielded frame image to generate a normalized addition light-shielded frame image (step S1740). That is, the correction unit 15 multiplies each pixel value of the acquired addition light shielding frame image by a value of (Tr/(n × Ts)) to generate a normalized addition light shielding frame image.
When the normalized addition light-shielding frame image is generated, the correction unit 1415 subtracts the pixel value of each pixel of the corresponding normalized addition light-shielding frame image from the pixel value of each pixel of the acquired signal frame image (step S1750), thereby generating a corrected signal frame image (step S1760).
When the process of step S1760 is completed, the correction unit 1415 proceeds to the process of step S1700, and repeats the subsequent processes.
[3-3. Effect, etc. ]
As described above, in the imaging device 3, the addition light-shielding frame image generated by the addition image generation unit 1417 is an image reduced by averaging random components of dark current components of the n light-shielding frame images.
Therefore, according to the imaging device 3 according to the modification 2, the correction of the signal frame image can be realized with higher accuracy than the imaging device 1 according to the embodiment.
(modification 3)
Here, an imaging apparatus according to modification 3 in which a partial function of the imaging apparatus 2 according to modification 1 is changed will be described with reference to the drawings.
The imaging device 2 according to modification 1 is configured such that the exposure period Tr and the light-shielding period Ts are set to be equal to each other, and the pixel values of the pixels of the light-shielding frame image output in the same frame period as the frame period in which the signal frame to be corrected is output are directly subtracted from each pixel of the signal frame image to be corrected, thereby generating a corrected signal frame image.
In contrast, the imaging apparatus according to the modification 3 is configured in the same manner in the case where the set exposure period Tr is equal to the light-shielding period Ts, but is different in that the light-shielding frame image to be subtracted from the correction target signal frame is changed to an addition-average light-shielding frame image obtained by performing addition-averaging on each pixel of the light-shielding frame images output in a plurality of frame periods including the frame period in which the correction target signal frame is output.
The following description will be made of the imaging device according to modification 3, focusing on changes to the imaging device 2 according to modification 1, with reference to the drawings.
[4-1. constitution ]
Fig. 18 is a block diagram showing a configuration of a camera 1800 according to modification 3.
As shown in the figure, the camera 1800 is modified from the camera 900 according to the modification in such a manner that the imaging device 2 is changed to the imaging device 4. The imaging apparatus 4 is modified from the imaging apparatus 2 according to modified example 1 in such a manner that the correction unit 915 is changed to a correction unit 1815, and an addition average image generation unit 1817 is added.
The addition average image generation unit 1817 generates an addition average light-shielding frame image by adding and averaging pixel values of pixels corresponding to each other in each light-shielding frame image with respect to a plurality of light-shielding frame images output from the image pickup element 10.
Fig. 19 is a schematic diagram showing a case where the addition average light-shielding frame image is generated by the addition average image generator 1817.
As shown in the figure, when a light-shielded frame image is newly output from the image sensor 10, the addition average image generation unit 1817 performs addition average on the pixel values of pixels corresponding to each other in each light-shielded frame image with respect to the chronologically newest n (n is an integer of 2 or more) light-shielded frame images output from the image sensor 10 including the newly output light-shielded frame image, and generates an addition average light-shielded frame image. Accordingly, the dark current signal components at the same position (i.e., the same pixel) in the n light-shielded frame images are added and averaged as the dark current signal component at the position (i.e., the pixel) in the addition-averaging light-shielded frame image. Therefore, as shown in fig. 19, a so-called "white defect" appearing at the same position in the n light-shielded frame images appears as a "white defect" of a luminance obtained by averaging the luminance of the white defect of each light-shielded frame image for that position in the addition-averaging light-shielded frame image. On the other hand, random noise ("random component") generated at different positions (i.e., different pixels) in the n light-shielded frame images is added by 1 piece in the addition-averaging light-shielded frame image, and further is averaged to 1/n times. Therefore, the luminance sufficiently lower than the "white defect" obtained by adding n times appears, and is hardly noticeable. For example, if a threshold value for which the "random component" is not included is set in the luminance level for which the addition averaging process is performed, and the luminance equal to or lower than the threshold value is set to "0", the "random component" can be further reduced. As described above, the addition shading frame image generated by the addition averaging processing is an image in which the random component of the dark current component of each of the n shading frame images is reduced.
As described above, the addition average light-shielding frame image generated by the addition average processing section 1817 is an image reduced by averaging random components of dark current components of the n light-shielding frame images.
Returning to fig. 18, the description of the image pickup device 4 is continued.
The correction unit 1815 is modified from the correction unit 915 according to modified example 1 in such a manner that its partial functions are changed.
The example of the correction unit 915 according to modification 1 is configured to generate a corrected signal frame image by directly subtracting, for each pixel of the correction target signal frame image, a pixel value of each pixel of the shading frame image output in the same frame period as the frame period in which the correction target signal frame is output.
In contrast, the configuration of the example of the correction unit 1815 is modified such that the shading frame image to be subtracted from the correction target signal frame is changed to: the addition averaging shading frame image output from the addition averaging image generating section 1817 containing the shading frame image output in the same frame period as that of the output correction target signal frame as the addition averaging target.
Fig. 20 is a schematic diagram showing a case where the correction is performed by the correction unit 1815.
As shown in the figure, the correction unit 1815 subtracts the pixel value of the corresponding addition average light shielding frame image from the pixel value of the pixel of the signal frame to be corrected, thereby generating a corrected signal frame image.
Hereinafter, the operation of the imaging device 4 configured as described above will be described with reference to the drawings.
[4-2. actions ]
The imaging device 4 performs the 2 nd frame image output process and the 4 th correction process as characteristic operations thereof.
The 2 nd frame image output processing has already been described in modification 1. Therefore, the 4 th correction process is explained here.
[4-2-1. 4 th correction processing ]
The 4 th correction processing is processing in which the correction unit 1815 performs correction so as to reduce a dark current signal component with respect to a signal frame image to be a target by subtracting a pixel value of a pixel constituting the addition average light shielding frame image generated by the addition average image generation unit 1817 from a pixel value of each pixel constituting the signal frame image output from the image pickup element 10.
The 4 th correction process is started by outputting the light-shielded frame image of the n-1 st time from the image pickup element 10 in the 1 st frame image output process described above.
Fig. 21 is a flowchart of the 4 th correction process.
If the 4 th correction process is started, the correction section 1815 waits until a signal frame image is output from the image pickup element 10. When the signal frame image is output while waiting for the signal frame image to be output (step S2100 is repeated and yes is obtained after no in step S2100), the correction unit 1815 acquires the signal frame image (step S2110).
If the signal frame image is acquired, the correcting section 1815 waits until the addition average light-shielding frame image is output from the addition average image generating section 1817. When the addition average light-shielding frame image is output while waiting for the output of the addition average light-shielding frame image (no in step S2120 is repeated and yes in step S2120), the correcting unit 1815 acquires the addition average light-shielding frame image (step S2130).
The correction unit 1815 subtracts the pixel value of each pixel of the corresponding obtained addition average light shielding frame image from the pixel value of each pixel of the obtained signal frame image (step S2150), thereby generating a corrected signal frame image (step S2160).
When the process of step S2160 is finished, the correction unit 1815 proceeds to the process of step S2100, and repeats the following processes.
[4-3. Effect, etc. ]
As described above, in the imaging device 4, the addition average light-shielding frame image generated by the addition average image generation unit 1817 is an image reduced by averaging random components of dark current components of the n light-shielding frame images.
Therefore, according to the imaging device 4 according to modification 3, the correction of the signal frame image can be realized with higher accuracy than the imaging device 2 according to modification 1.
(modification 4)
Here, an imaging apparatus according to modification 4 in which a partial function of the imaging apparatus 3 according to modification 2 is changed will be described with reference to the drawings.
The image pickup apparatus 3 according to modification 2 is configured such that the correction unit 1415 multiplies each pixel value of the addition light-shielding frame image obtained by adding n light-shielding frame images by a value of (exposure period Tr)/(nx (light-shielding period Ts)) to generate a normalized addition light-shielding frame image, and subtracts each pixel value of the corresponding normalized addition light-shielding frame image from each pixel value of the correction target signal frame image to generate a corrected signal frame image.
In contrast, the imaging apparatus according to the modification 4 is configured such that the exposure period Tr is set to be n times the light-shielding period Ts, and the correction unit directly subtracts each pixel value of the corresponding addition light-shielding frame image from each pixel value of the correction target signal frame image, thereby generating a corrected signal frame image.
The following description will be given of the imaging device according to modification 4, focusing on changes to the imaging device 3 according to modification 2, with reference to the drawings.
[5-1. constitution ]
Fig. 22 is a block diagram showing a configuration of a camera 2200 according to modification 4.
As shown in the drawing, the camera 2200 is modified from the camera 1400 according to modification 2 in such a manner that the imaging device 3 is changed to the imaging device 5. The imaging apparatus 5 is modified in such a manner that the correction unit 1415 is changed to the correction unit 2215 and the voltage control unit 20 is changed to the voltage control unit 2220 in addition to the imaging apparatus 3 according to modified example 2.
The voltage control unit 2220 is modified from the voltage control unit 20 according to modification example 2 in such a manner that a part of functions thereof is changed.
The voltage control unit 20 according to modification 2 outputs a frame start signal to the image pickup device 10 in a frame period T1, and further outputs an exposure/light-shielding switching signal in a frame period T1 that is delayed by a predetermined period T2 from the frame start signal, thereby controlling the voltage applied to the photoelectric conversion means 111.
In contrast, the voltage control unit 2220 outputs a frame start signal for the image pickup device 10 in the frame period T1, and further outputs an exposure/light-shielding switching signal in the frame period T1 delayed by (n/(n +1)) × T1 from the output of the frame start signal, thereby controlling the voltage applied to the photoelectric conversion section 111.
Fig. 23A is a timing chart of the frame start signal and the exposure/light-shielding switching signal output from the voltage control unit 2220. Fig. 23B is a timing chart showing the operation of the image pickup device 10 when receiving the frame start signal and the exposure/light-shielding switching signal from the voltage control unit 2220.
As shown in fig. 23A, the voltage control unit 2220 outputs the frame start signal and the exposure/light-shielding switching signal so that the exposure/light-shielding switching signal is delayed from the frame start signal by (n/(n +1)) × T1 for each frame period T1 of the image pickup device 10.
As shown in fig. 23B, when the frame start signal and the exposure/light-shielding switching signal output for each frame period T1 are received from the voltage control unit 2220, the image pickup device 10 applies the 1 st voltage to the photoelectric conversion section 111 during a period from when the frame start signal is received until when the exposure/light-shielding switching signal is received next, and applies the 2 nd voltage to the photoelectric conversion section 111 during a period from when the exposure/light-shielding switching signal is received until when the frame start signal is received next.
Therefore, the photoelectric conversion element 110 is in the exposure state during a period from when a frame start signal is received until (n/(n +1)) × T1 is passed, and is in the light-shielding state during a period from when (n/(n +1)) × T1 is passed until a frame start signal is received next.
The image pickup device 10 reads the amount of charge accumulated in the charge accumulation node 25 of each pixel circuit 21 in the exposure period in which the photoelectric conversion element 110 is in the exposure state, outputs a signal frame image based on the read amount of charge, reads the amount of charge accumulated in the charge accumulation node 25 of each pixel circuit 21 in the light-shielding period in which the photoelectric conversion element 110 is in the light-shielding state, and outputs a signal frame image based on the read amount of charge.
Returning again to fig. 22, the description of the imaging device 5 is continued.
The correction unit 2215 is modified in addition to the correction unit 1415 according to the modification example such that a part of functions thereof is changed.
The correction unit 1415 according to modification 2 is configured to generate a normalized addition light-shielding frame image by multiplying each pixel value of the addition light-shielding frame image by a value of (Tr/(n × Ts)), and to generate a corrected signal frame image by subtracting a pixel value of each pixel of the corresponding normalized addition light-shielding frame image from each pixel value of the correction target signal frame image.
In contrast, the correction unit 2215 is configured to generate a corrected signal frame image by directly subtracting each pixel of the corresponding addition shading frame image from each pixel of the correction target signal frame image.
Fig. 24 is a schematic diagram showing a case where the correction portion 2215 performs the correction.
As shown in the figure, the correction unit 2215 generates a corrected signal frame image by subtracting the pixel value of the corresponding addition light shielding frame image from the pixel value of the pixel of the correction target signal frame.
In the imaging device 5, the exposure period Tr is set to be n times the light-shielding period Ts. Therefore, the correction unit 2215 does not need to perform normalization processing for normalizing the total sum (n × Ts) of the light-shielded period with the exposure state period (Tr) of the correction target signal frame image with respect to the addition light-shielded frame image, as in the correction unit 1415 according to modification 2.
Hereinafter, the operation of the imaging device 5 configured as described above will be described with reference to the drawings.
[5-2. actions ]
The imaging device 5 performs, as its characteristic operation, the 3 rd frame image output process and the 5 th correction process.
These processes are explained below in order.
[5-2-1. 3 rd frame image output processing ]
The 3 rd frame image output processing is processing in which the image pickup device 10 alternately outputs the signal frame image and the shading frame image at a timing delayed by (n/(n +1)) × T1 from the signal frame image at a predetermined frame period T1.
The 3 rd frame image output processing is processing in which a part of the processing is changed in addition to the 1 st frame image output processing according to the embodiment.
Fig. 25 is a flowchart of the 3 rd frame image output processing.
In the same figure, the processing of step S2505 to the processing of step S2530 and the processing of step S2540 to the processing of step S2570 are processing of step S5 to step S30 and processing of step S40 to step S79 in the 1 st frame image output processing (see fig. 7) according to the embodiment, respectively, and the voltage controller 20 is replaced with the voltage controller 920.
Therefore, here, the processing of step S2505 to the processing of step S2530 and the processing of step S2540 to the processing of step S2570 have been described, and the processing of step S2535 and the processing before and after the processing are described.
After the process of step S2530 is completed, when (n/(n +1)) × T1 elapses from the previous output frame start signal (no in step S2535 is repeated and yes in step S2535), the voltage control unit 2220 outputs an exposure switching signal to the image pickup device 10 (step S2540).
[5-2-2. 2 nd correction processing ]
The 5 th correction processing is processing in which the correction section 2215 directly subtracts each pixel of the corresponding addition light shielding frame image from each pixel of the signal frame image output from the image pickup device 10 to perform correction so as to reduce the dark current signal component with respect to the correction target signal frame image.
The 4 th correction processing is processing in which a part of the processing is changed in addition to the 3 rd correction processing according to modification example 2.
Fig. 26 is a flowchart of the 5 th correction process.
In the same figure, the processing of step S2600 to the processing of step S2630 and the processing of step S2660 are processing in which the correction unit 915 is replaced with the correction unit 2215 for the processing of step S1700 to the processing of step S1730 in the 3 rd correction processing (see fig. 17) according to modification 2.
In the 5 th correction process, the process corresponding to step S1740 in the 3 rd correction process according to modification 2 is omitted.
Therefore, the processing of step S2600 to the processing of step S2630 and the processing of step S2660 have already been described here, and the processing of step S2650 and the processing before and after the processing are described below.
When the addition light-shielding frame image is acquired in the process of step S2630, the correction unit 2215 subtracts the pixel value of the corresponding addition light-shielding frame image from the pixel value of each pixel of the acquired signal frame image (step S2650), thereby generating a corrected signal frame image (step S2660).
[5-3. Effect, etc. ]
As described above, in the imaging apparatus 5, the correction unit 2215 directly subtracts each pixel value of the corresponding addition shading frame image from each pixel value of the correction target signal frame image to generate a corrected signal frame image.
Therefore, according to the imaging device 5 of modification 4, the correction of the signal frame image can be realized with a smaller amount of calculation than the imaging device 3 of modification 2.
(supplement)
As described above, the embodiment and the modifications 1 to 4 have been described as examples of the technique disclosed in the present application. However, the technique of the present disclosure is not limited to this, and can be applied to an embodiment in which modifications, substitutions, additions, omissions, and the like are appropriately made.
(1) In the embodiment, the photoelectric conversion member 111 in the image pickup device 1 is described as an organic thin film having a function of generating electric charges by the internal photoelectric effect by receiving light in a state where a voltage of a 1 st predetermined range is applied, and generating no electric charges by the internal photoelectric effect by receiving light in a state where a voltage of a 2 nd predetermined range is applied.
However, the photoelectric conversion member 111 is not limited to the organic thin film as long as it can control generation of charges by the internal photoelectric effect by application of voltage. As an example, in the imaging device 1, an example in which the photoelectric conversion member 111 is a diode having a PN junction surface is considered.
(2) In the embodiment, the imaging device 1 is described such that the frame period T1 is, for example, 1/60 seconds.
However, the frame period T1 need not necessarily be limited to 1/60 seconds.
For example, an example in which the frame period T1 is 1/50 seconds in the imaging apparatus 1, an example in which the frame period T1 is set by a user using the imaging apparatus 1, and the like can be considered.
(3) In modification 2, the description has been given of the imaging apparatus 3 in which, when a light-shielded frame image is newly output from the imaging device 10, the additive image generation unit 1417 adds the pixel values of the pixels corresponding to each other in each light-shielded frame image to generate an additive light-shielded frame image for n light-shielded frame images output from the imaging device 10 including the newly output light-shielded frame image.
However, the method of generating the addition shading frame image is not necessarily limited to the above method.
For example, when a new shading frame image is output after an addition shading frame image has been generated, the new shading frame image is taken in by arithmetic processing such as addition averaging or IIR (Infinite Impulse Response) in the addition shading frame image that has been generated, and the addition shading frame image is updated.
In another example, an additive shading frame image may be generated by performing weighted addition of pixel values of pixels corresponding to each other in each shading frame image for n shading frame images so that a chronologically new shading frame image is given a large weight. In this way, the light-shielded frame image output at a relatively close distance can be reflected in a large amount in the reduction of the dark current signal component.
(4) The present application obviously also includes an electronic apparatus incorporating the imaging device 1 of the embodiment.
Such an electronic apparatus is implemented as, for example, a digital camera shown in fig. 27A or a video camera shown in fig. 27B.
(5) In the embodiment, as shown in fig. 1, the imaging device 1 is described as a structure separate from the optical system 210. However, the imaging device 1 is not necessarily limited to a structure separate from the optical system 210. For example, the imaging device 1 may be a camera with a lens including the optical system 210 and the lens driving unit 220.
(6) The components (functional blocks) of the imaging devices 1 to 5 may be individually formed into a single chip by a semiconductor device such as an IC (Integrated Circuit) or an LSI (Large Scale Integration), or may be formed into a single chip including a part or all of them. The method of integrating circuits is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after LSI manufacturing or a reconfigurable processor that can reconfigure connection and setting of circuit cells inside LSI can also be used. Furthermore, if the technology of the integrated circuit of the LSI is replaced by another technology derived from the progress of the semiconductor technology, the functional blocks can be integrated by the technology. The use of biotechnology and the like are also possible.
All or part of the various processes described above may be implemented by hardware such as an electronic circuit, or may be implemented by software. The software processing is realized by a processor included in the imaging apparatus 1 executing a program stored in a memory. In addition, the program can be distributed or distributed by being recorded on a recording medium. For example, by installing a distributed program to another device having a processor and causing the processor to execute the program, the device can perform the above-described respective processes.
Further, the present invention is also intended to cover embodiments realized by arbitrarily combining the constituent elements and functions described in the above embodiments.
(7) An imaging device 1 according to an aspect of the present disclosure includes: an imaging element 10 including a photoelectric conversion unit 111 and a plurality of pixel circuits 21, the photoelectric conversion unit 111 generating electric charges based on an internal photoelectric effect by receiving light in a state where a voltage of a 1 st predetermined range is applied, and generating no electric charges based on the internal photoelectric effect even if the light is received in a state where a voltage of a 2 nd predetermined range is applied, the plurality of pixel circuits 21 accumulating the electric charges generated by the photoelectric conversion unit 111 in units of pixels, the imaging element 10 outputting a frame image based on an amount of electric charges accumulated in the plurality of pixel circuits 21; a voltage control section 20 that controls a voltage applied to the photoelectric conversion section 111; and a correction unit 15 that performs correction to reduce a dark current signal component for at least a part of the 1 or more frame images output from the image pickup device 10, wherein the voltage control unit 20 performs the control as follows: the correction section 15 applies the voltage of the 1 st predetermined range to the photoelectric conversion section 111 during an exposure period which is a part of a predetermined frame period, applies the voltage of the 2 nd predetermined range to the photoelectric conversion section 111 during a light shielding period other than the exposure period in the frame period, outputs a signal frame image based on the amount of electric charges accumulated in the plurality of pixel circuits 21 during the exposure period and a light shielding frame image based on the amount of electric charges accumulated in the plurality of pixel circuits 21 during the light shielding period for each of the frame periods, and performs the correction using the light shielding frame image output from the image pickup element 10 with respect to the signal frame image output from the image pickup element 10.
In the imaging device 1, a dark current component in a light-shielded state is accumulated as electric charges in the pixel circuit 21 during a light-shielded period. The imaging element 10 outputs a light-shielded frame image based on the amount of charge accumulated in the pixel circuit 21 during the light-shielding period during the imaging period of the continuous signal frame image. On the other hand, the correction unit 15 performs correction to reduce the dark current signal component for the signal frame image based on the light-shielded frame.
Therefore, according to the imaging device 1, it is possible to perform correction so as to reduce the dark current signal component included in the frame image imaged in the imaging period based on the dark current signal component in the light-shielded state acquired in the imaging period of the continuous frame image.
Generally, the dark current signal component varies depending on the temperature of the image pickup element 10 and the like. Therefore, when the temperature of the imaging element 10 or the like changes during the imaging period of the continuous frame image, the dark current signal component changes. Therefore, according to the imaging apparatus 1, the dark current signal component included in the frame image captured in the imaging period can be reduced more accurately than in the conventional imaging apparatus which cannot acquire the dark current signal component in the imaging period of the continuous signal frame image.
In addition, for example, the image pickup device 10 may be an organic CMOS image sensor having an organic thin film as the photoelectric conversion part 111.
This enables high integration of the imaging device 10.
For example, the exposure period may be equal to the light-shielding period, and the correction unit 15 may perform the correction by subtracting, from a pixel value of each pixel constituting a signal frame image to be corrected, a pixel value of a pixel corresponding to the pixel constituting a light-shielding frame image output from the image pickup device 10 in the same frame period as the signal frame image.
Accordingly, the correction by the correction unit 15 can be realized by calculation with a small amount of processing.
For example, the correction unit 15 may perform the correction by subtracting a value obtained by multiplying a pixel value of a pixel constituting a light-shielded frame image output from the image pickup device 10 in the same frame period as the signal frame image, the pixel value corresponding to the pixel, by a ratio of the exposure period to the light-shielded period, from a pixel value of each pixel constituting the signal frame image to be corrected.
Accordingly, the correction section 15 can perform correction even when the exposure period and the light-shielding period are not equal to each other.
For example, the exposure period may be equal to the light-shielding period, the exposure apparatus may further include an addition average image generation unit 1817, the addition average image generation unit 1817 may generate an addition average light-shielding frame image by adding and averaging pixel values of pixels corresponding to each other in each light-shielding frame image for a plurality of light-shielding frame images, and the correction unit 15 may perform the correction by subtracting a pixel value of a pixel corresponding to each pixel constituting the addition average light-shielding frame image generated by the addition average image generation unit 1817 from a pixel value of each pixel constituting a signal frame image to be corrected.
This can reduce the random component included in the dark current signal component.
For example, the image processing apparatus may further include an addition image generation unit 1417, wherein the addition image generation unit 1417 generates an addition light-shielded frame image by adding pixel values of pixels corresponding to each other in each of the light-shielded frame images to the plurality of light-shielded frame images, and the correction unit 15 performs the correction based on the addition light-shielded frame image generated by the addition image generation unit 1417.
This can reduce the random component included in the dark current signal component.
For example, the ratio of the exposure period to the light-shielded period may be n (n is an integer equal to or greater than 2): 1, the addition image generation unit 1417 may generate the addition image for the n light-shielded frame images output from the image pickup device 10, and the correction unit 15 may perform the correction by subtracting the pixel value of the pixel corresponding to the pixel constituting the addition image generated by the addition image generation unit 1417 from the pixel value of each pixel constituting the signal frame image to be corrected.
Accordingly, the correction by the correction section 15 using the dark current signal component reduced in the random component can be realized by the calculation with a small amount of processing.
For example, the added image generation unit 1417 may generate the added image so as to sequentially generate the added images for n light-shielded frame images that are consecutive in time series, and the correction unit 15 may perform the correction by subtracting, from the pixel value of each pixel constituting the signal frame image to be corrected, the pixel value of the pixel constituting the added image generated by the added image generation unit 1417 for the n consecutive light-shielded frame images included in the light-shielded frame image output from the image pickup device 10 in the same frame period as the signal frame image, the pixel value corresponding to the pixel.
Accordingly, correction using a dark current signal component with a reduced random component can be realized by using a light-shielded frame image group having the shortest imaging time difference.
A camera 200 according to an embodiment of the present disclosure includes an imaging device 1 and a lens that condenses external light to an imaging element 10.
In the camera 200, a dark current component in a light-shielded state is accumulated as electric charges in the pixel circuit 21 during a light-shielded period. The imaging element 10 outputs a light-shielded frame image based on the amount of charge accumulated in the pixel circuit 21 during the light-shielding period during the imaging period of the continuous signal frame image. On the other hand, the correcting unit 15 performs correction to reduce the dark current signal component for the signal frame image based on the light-shielded frame.
Therefore, according to the camera 200, it is possible to perform correction so as to reduce the dark current signal component included in the frame image captured during the imaging period based on the dark current signal component in the light-shielded state obtained during the imaging period of the continuous frame image.
An image pickup method according to one aspect of the present disclosure is an image pickup method performed by an image pickup apparatus 1 including an image pickup device 10, a voltage control unit 20, and a correction unit 15, wherein the image pickup device 10 includes a photoelectric conversion unit 111 and a plurality of pixel circuits 21, the photoelectric conversion unit 111 generates electric charges based on an internal photoelectric effect by receiving light in a state where a voltage in a 1 st predetermined range is applied, and does not generate electric charges based on the internal photoelectric effect even if receiving light in a state where a voltage in a 2 nd predetermined range is applied, and the plurality of pixel circuits 21 accumulate electric charges generated by the photoelectric conversion unit 111 in units of pixels, the image pickup method including: an imaging step in which the imaging element 10 outputs a frame image based on the amount of charge accumulated in the plurality of pixel circuits 21; a voltage control step in which the voltage control section 20 controls the voltage applied to the photoelectric conversion section 111, and a correction step in which the correction section 15 performs correction to reduce the dark current signal component for at least a part of 1 or more frame images output from the image pickup element 10; in the voltage control step, the control is performed as follows: the voltage control unit 20 applies the voltage of the 1 st predetermined range to the photoelectric conversion member 111 during an exposure period which is a part of a predetermined frame period, and applies the voltage of the 2 nd predetermined range to the photoelectric conversion member 111 during a light shielding period other than the exposure period in the frame period, and in the image pickup step, the image pickup device 10 outputs a signal frame image based on the amount of electric charge accumulated in the plurality of pixel circuits 21 during the exposure period and a light shielding frame image based on the amount of electric charge accumulated in the plurality of pixel circuits 21 during the light shielding period for each of the frame periods, and in the correction step, the correction unit 15 performs the correction using the light shielding frame image output from the image pickup device 10 with respect to the signal frame image output from the image pickup device 10.
In this imaging method, a dark current component in a light-shielded state is accumulated as electric charges in the pixel circuit 21 during a light-shielded period. The imaging element 10 outputs a light-shielded frame image based on the amount of charge accumulated in the pixel circuit 21 during the light-shielding period during the imaging period of the continuous signal frame image. On the other hand, the correcting unit 15 performs correction to reduce the dark current signal component with respect to the signal frame image based on the light-shielded frame.
Therefore, according to this imaging method, it is possible to perform correction so as to reduce the dark current signal component included in the frame image imaged in the imaging period based on the dark current signal component in the light-shielded state acquired in the imaging period of the consecutive frame images.
Industrial applicability
The present disclosure can be widely applied to an imaging apparatus that captures an image.
Description of reference numerals:
1. 2, 3, 4, 5 image pickup device
10 image pickup element
15. 915, 1415, 1815 and 2215 correction part
20. 920, 2220 voltage control part
21 pixel circuit
30 interface part
110 photoelectric conversion element
111 photoelectric conversion member
112 upper transparent electrode
113 lower pixel electrode
120 pixel circuit array
130 readout circuit
140 output circuit
150-line scanning circuit
160 timing control circuit
170 voltage applying circuit
200. 900, 1400, 1800, 2200 camera
211 zoom lens
212 handshake correction lens
213 Focus lens
1417 additive image generating unit
1817 adding the average image generating part.

Claims (10)

1. An imaging device is provided with:
an image pickup element including a photoelectric conversion unit that generates charge based on an internal photoelectric effect by receiving light in a state where a voltage of a 1 st predetermined range is applied, and that does not generate charge based on the internal photoelectric effect even if the light is received in a state where the voltage of a 2 nd predetermined range is applied, and a plurality of pixel circuits that accumulate the charge generated by the photoelectric conversion unit on a pixel-by-pixel basis, the image pickup element outputting a frame image based on an amount of charge accumulated by the plurality of pixel circuits;
a voltage control section that controls a voltage applied to the photoelectric conversion member; and
a correction unit that performs correction to reduce a dark current signal component for at least a part of 1 or more frame images output from the image pickup device;
the voltage control unit performs the control as follows: applying the voltage of the 1 st predetermined range to the photoelectric conversion means during an exposure period which is a part of a predetermined frame period, applying the voltage of the 2 nd predetermined range to the photoelectric conversion means during a light-shielding period other than the exposure period in the frame period,
the image pickup device outputs, for each of the frame periods, a signal frame image based on the amount of charge accumulated in the plurality of pixel circuits during the exposure period and a light-shielded frame image based on the amount of charge accumulated in the plurality of pixel circuits during the light-shielded period,
the correction unit performs the correction using the light-shielded frame image output from the image pickup device for the signal frame image output from the image pickup device.
2. The image pickup apparatus according to claim 1,
the image sensor is an organic Complementary Metal Oxide Semiconductor (CMOS) image sensor, which is an organic CMOS image sensor in which an organic thin film is used as the photoelectric conversion member.
3. The image pickup apparatus according to claim 1 or 2,
the exposure period is equal to the light-shielding period,
the correction unit performs the correction by subtracting, from a pixel value of each pixel constituting a signal frame image to be corrected, a pixel value of a pixel corresponding to the pixel constituting a light-shielded frame image outputted from the image pickup device in the same frame period as the signal frame image.
4. The image pickup apparatus according to claim 1 or 2,
the correction unit performs the correction by subtracting a value obtained by multiplying a pixel value of a pixel constituting a light-shielded frame image output from the image pickup device in the same frame period as the signal frame image, by a ratio of the exposure period to the light-shielded period, from a pixel value of each pixel constituting the signal frame image to be corrected.
5. The image pickup apparatus according to claim 1 or 2,
the exposure period is equal to the light-shielding period,
the imaging device further includes an addition average image generation unit that generates an addition average light-shielding frame image by adding and averaging pixel values of pixels corresponding to each other in each of the light-shielding frame images for the plurality of light-shielding frame images,
the correction unit performs the correction by subtracting, from a pixel value of each pixel constituting the signal frame image to be corrected, a pixel value of a pixel corresponding to the pixel constituting the addition average light shielding frame image generated by the addition average image generation unit.
6. The image pickup apparatus according to claim 1 or 2,
the imaging device further includes an addition image generation unit that generates an addition light-shielded frame image by adding pixel values of pixels corresponding to each other in each of the light-shielded frame images for the plurality of light-shielded frame images,
the correction section performs the correction based on the addition light-shielding frame image generated by the addition image generation section.
7. The image pickup apparatus according to claim 6,
a ratio of the exposure period to the light-shielding period is n to 1, where n is an integer of 2 or more,
the added image generation unit generates the added image for the n light-shielded frame images output from the image pickup device,
the correction unit performs the correction by subtracting a pixel value of a pixel constituting the added image generated by the added image generation unit corresponding to each pixel from a pixel value of each pixel constituting the signal frame image to be corrected.
8. The image pickup apparatus according to claim 6,
the added image generation unit generates the added image so as to sequentially generate the added image for n light-shielded frame images that are consecutive in time series,
the correction unit performs the correction by subtracting a pixel value of a pixel constituting an added image generated by the added image generation unit for n consecutive masked frame images including a masked frame image output from the image pickup device in the same frame period as the signal frame image, from a pixel value of each pixel constituting the signal frame image to be corrected.
9. A camera is provided with:
the image pickup device according to any one of claims 1 to 8; and
and a lens that condenses external light to the image pickup element.
10. An image pickup method for an image pickup apparatus including an image pickup device, a voltage control unit, and a correction unit,
the image pickup element includes a photoelectric conversion unit that generates charge based on an internal photoelectric effect by receiving light in a state where a voltage of a 1 st predetermined range is applied and generates no charge based on the internal photoelectric effect even if the light is received in a state where the voltage of a 2 nd predetermined range is applied, and a plurality of pixel circuits that accumulate the charge generated by the photoelectric conversion unit on a pixel basis,
the image pickup method includes:
an image pickup step in which the image pickup element outputs a frame image based on the amount of charge accumulated in the plurality of pixel circuits;
a voltage control step of controlling a voltage applied to the photoelectric conversion member by the voltage control section; and
a correction step of performing correction to reduce a dark current signal component with respect to at least a part of 1 or more frame images output from the image pickup element by the correction unit,
in the voltage control step, the voltage control unit performs the control as follows: applying the voltage of the 1 st predetermined range to the photoelectric conversion means during an exposure period which is a part of a predetermined frame period, applying the voltage of the 2 nd predetermined range to the photoelectric conversion means during a light-shielding period other than the exposure period in the frame period,
in the image pickup step, the image pickup device outputs, for each of the frame periods, a signal frame image based on the amount of charge accumulated in the plurality of pixel circuits in the exposure period and a light-shielded frame image based on the amount of charge accumulated in the plurality of pixel circuits in the light-shielded period,
in the correction step, the correction unit performs the correction using the light-shielded frame image output from the image pickup device, with respect to the signal frame image output from the image pickup device.
CN201780072782.1A 2016-12-27 2017-12-26 Imaging device, camera, and imaging method Active CN109997352B (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2016-254499 2016-12-27
JP2016254499 2016-12-27
PCT/JP2017/046599 WO2018124055A1 (en) 2016-12-27 2017-12-26 Imaging device, camera and imaging method

Publications (2)

Publication Number Publication Date
CN109997352A CN109997352A (en) 2019-07-09
CN109997352B true CN109997352B (en) 2021-06-01

Family

ID=62707599

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201780072782.1A Active CN109997352B (en) 2016-12-27 2017-12-26 Imaging device, camera, and imaging method

Country Status (5)

Country Link
US (1) US10573681B2 (en)
EP (1) EP3565238B1 (en)
JP (1) JP6731626B2 (en)
CN (1) CN109997352B (en)
WO (1) WO2018124055A1 (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114270807A (en) * 2019-09-06 2022-04-01 松下知识产权经营株式会社 Image pickup apparatus
US11683568B2 (en) * 2020-02-21 2023-06-20 Hitachi Kokusai Electric Inc. Image capturing system, image processing program, and image processing method
CN112367475B (en) * 2021-01-15 2021-03-30 上海闪马智能科技有限公司 Traffic incident detection method and system and electronic equipment
US11696049B2 (en) 2021-05-28 2023-07-04 Microsoft Technology Licensing, Llc Systems and methods for dark current compensation in single photon avalanche diode imagery
US11711628B2 (en) 2021-05-28 2023-07-25 Microsoft Technology Licensing, Llc Systems and methods for obtaining color imagery using single photon avalanche diodes

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5038369A (en) * 1988-12-19 1991-08-06 Kabushiki Kaisha Toshiba X-ray fluoroscopic apparatus
CN1270706A (en) * 1997-08-15 2000-10-18 尤尼艾克斯公司 Organic diodes with switchable photosensitivity
CN101335842A (en) * 2007-06-26 2008-12-31 富士胶片株式会社 Image detection device and method for driving image detector
JP2009105246A (en) * 2007-10-24 2009-05-14 Seiko Epson Corp Photoelectric conversion element, solid-state imaging apparatus, and electronic apparatus
CN106105181A (en) * 2014-03-11 2016-11-09 E2V半导体公司 There is the dark current of reduction and the image-capturing method of low-power consumption

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4292751B2 (en) 2002-05-30 2009-07-08 日本ビクター株式会社 Imaging device
KR20060038681A (en) * 2004-11-01 2006-05-04 삼성테크윈 주식회사 Apparatus and method for removing hot pixel in digital camera
US20070041063A1 (en) 2005-08-18 2007-02-22 Matsushita Electric Industrial Co., Ltd. Image sensor
JP2007067194A (en) * 2005-08-31 2007-03-15 Fujifilm Corp Organic photoelectric conversion device and stacked photoelectric conversion device
JP2008042180A (en) * 2006-07-13 2008-02-21 Matsushita Electric Ind Co Ltd Image sensor
JP4252078B2 (en) * 2006-09-28 2009-04-08 三洋電機株式会社 Photodetector
JP5335271B2 (en) * 2008-04-09 2013-11-06 キヤノン株式会社 Photoelectric conversion device and imaging system using the same
JP2011055351A (en) * 2009-09-03 2011-03-17 Fujifilm Corp Photographing device and control method thereof
JP2011250249A (en) * 2010-05-28 2011-12-08 Panasonic Corp Video signal processing apparatus and video signal processing method
US8916947B2 (en) * 2010-06-08 2014-12-23 Invisage Technologies, Inc. Photodetector comprising a pinned photodiode that is formed by an optically sensitive layer and a silicon diode
JP5847413B2 (en) * 2011-03-24 2016-01-20 キヤノン株式会社 Imaging apparatus, imaging system, and control method thereof
GB2497571A (en) * 2011-12-15 2013-06-19 St Microelectronics Res & Dev An imaging array with high dynamic range
US9325913B2 (en) * 2011-12-28 2016-04-26 General Electric Company Radiation detector for use in sequential image acquisition
JP6223881B2 (en) * 2014-03-18 2017-11-01 株式会社東芝 Photodetector
JP2017152669A (en) * 2016-02-25 2017-08-31 パナソニックＩｐマネジメント株式会社 Imaging device
JP6904730B2 (en) * 2016-03-08 2021-07-21 株式会社半導体エネルギー研究所 Imaging device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5038369A (en) * 1988-12-19 1991-08-06 Kabushiki Kaisha Toshiba X-ray fluoroscopic apparatus
CN1270706A (en) * 1997-08-15 2000-10-18 尤尼艾克斯公司 Organic diodes with switchable photosensitivity
CN101335842A (en) * 2007-06-26 2008-12-31 富士胶片株式会社 Image detection device and method for driving image detector
JP2009105246A (en) * 2007-10-24 2009-05-14 Seiko Epson Corp Photoelectric conversion element, solid-state imaging apparatus, and electronic apparatus
CN106105181A (en) * 2014-03-11 2016-11-09 E2V半导体公司 There is the dark current of reduction and the image-capturing method of low-power consumption

Also Published As

Publication number Publication date
JP6731626B2 (en) 2020-07-29
EP3565238A4 (en) 2020-01-22
EP3565238B1 (en) 2023-07-19
US10573681B2 (en) 2020-02-25
JPWO2018124055A1 (en) 2019-07-25
WO2018124055A1 (en) 2018-07-05
CN109997352A (en) 2019-07-09
US20190280037A1 (en) 2019-09-12
EP3565238A1 (en) 2019-11-06

Similar Documents

Publication Publication Date Title
CN109997352B (en) Imaging device, camera, and imaging method
RU2609540C2 (en) Image capturing device and method of controlling image capturing device
JP5959834B2 (en) Imaging device
JP5013811B2 (en) Imaging apparatus and correction method
JP4834345B2 (en) IMAGING DEVICE, ITS CONTROL METHOD, PROGRAM, AND STORAGE MEDIUM
JP5013812B2 (en) Imaging apparatus and correction method
CN109474781B (en) Image pickup apparatus, control method for image pickup apparatus, and recording medium
US8169524B2 (en) Image sensing apparatus with pixel readout correction
JP5701942B2 (en) Imaging apparatus, camera system, and image processing method
JP2008300898A (en) Solid state imaging device and imaging system employing it
JP6463159B2 (en) IMAGING DEVICE, ITS CONTROL METHOD, PROGRAM, AND STORAGE MEDIUM
WO2018124056A1 (en) Imaging device and control method therefor
JP2008072512A (en) Photographing apparatus and its control method, and photographing system
JP2014057196A (en) Imaging apparatus
JP7271131B2 (en) IMAGING DEVICE AND METHOD OF CONTROLLING IMAGING DEVICE
JP2018093301A (en) Image pick-up device and control method of the same
JP6706850B2 (en) Imaging device, camera, and imaging method
JP2010166479A (en) Imaging device and method of correcting imaged image
JP2018019268A (en) Imaging apparatus, imaging system, and driving method of imaging apparatus
JP2008118386A (en) Imaging apparatus, its control method and imaging system
JP6706783B2 (en) Imaging device, imaging device, camera, and imaging method
JP6470589B2 (en) IMAGING DEVICE, ITS CONTROL METHOD, PROGRAM, AND STORAGE MEDIUM
JP2017188837A (en) Image processing apparatus, control method thereof, and program
JP5664010B2 (en) Imaging device
JP2012204938A (en) Imaging apparatus

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant