JP2020028081A - Imaging apparatus and control method of the same, program, and storage medium - Google Patents

Abstract

To suppress an increase in power consumption due to irradiation of high-brightness light in an imaging apparatus capable of detecting a single photon.SOLUTION: The imaging apparatus includes: a plurality of pixels each having a counting unit for counting the number of incident photons and a selection unit for selecting either of a first state for counting the number of photons or a second state in which the number of photons is not counted for the setting of the counting unit; and a generating unit for generating an image corresponding to light incident on pixels on the basis of the counting result by the counting unit in each pixel. The selection unit selects whether to set the counting unit to the first state or the second state in a second exposure period performed after a first exposure period according to the amount of light incident on the pixels during a first exposure period.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an imaging device and a control method thereof.

  In recent years, there has been proposed a photon counting type image sensor that counts the number of photons incident on a photodiode during an exposure period and outputs the counted value as a signal value.

  As a method for realizing the photon counting method, for example, Patent Document 1 discloses a method using an avalanche photodiode and a counter. When a reverse bias voltage higher than the breakdown voltage is applied to the avalanche photodiode, carriers generated by the incidence of a single photon cause avalanche multiplication, and a large current is generated.

  By counting the pulse signal generated based on this current by the counter circuit, a signal value corresponding to the number of photons incident on the avalanche photodiode can be obtained. The photon counting type image sensor treats the number of photons incident on the photodiode as a signal value as it is. Therefore, as compared with a CCD or a CMOS image sensor, there is an advantage that the influence of circuit noise on a signal is small and an image can be clearly captured even in a weak light environment.

JP-A-61-152176

  However, in such an image sensor, each time a photon is incident on a plurality of pixels, the above-described large current is generated. In particular, if high-intensity light is continuously irradiated on an image sensor having a large number of pixels, such a current continues to be repeatedly generated in many pixels, resulting in an increase in power consumption.

  The present invention has been made in view of the above-described problem, and an object of the present invention is to suppress an increase in power consumption due to irradiation of high-brightness light in an imaging device capable of detecting a single photon.

  The imaging apparatus according to the present invention includes a counting unit that counts the number of incident photons, a first state in which the counting unit counts the number of photons, and a second state in which the number of photons is not counted. Generating means for generating an image corresponding to the light incident on the pixel, based on a plurality of pixels each having a selecting means for selecting which one to set, and a counting result of the counting means in each of the pixels. Means, and wherein the selecting means controls the counting means in the second exposure period performed after the first exposure period in accordance with the amount of light incident on the pixel during the first exposure period. It is characterized by selecting whether to set to the first state or to the second state.

  ADVANTAGE OF THE INVENTION According to this invention, in the imaging device which can detect a single photon, it becomes possible to suppress the increase of the power consumption accompanying irradiation of high luminance light.

FIG. 1 is a block diagram illustrating a configuration of an imaging device according to a first embodiment of the present invention. FIG. 2 is a diagram illustrating a configuration of a solid-state imaging device according to the first embodiment. FIG. 2 is a diagram illustrating a configuration of a unit pixel according to the first embodiment. 4 is a timing chart illustrating sub-frame driving of the solid-state imaging device according to the first embodiment. 6 is a timing chart illustrating driving of the solid-state imaging device according to the first embodiment. FIG. 4 is a diagram illustrating driving in each region of a pixel unit according to the first embodiment. FIG. 2 is a diagram illustrating a semiconductor chip structure of the solid-state imaging device according to the first embodiment. FIG. 9 is a diagram illustrating a configuration of a unit pixel according to the second embodiment. 9 is a timing chart illustrating sub-frame driving of the solid-state imaging device according to the second embodiment. FIG. 7 is a diagram illustrating driving of a solid-state imaging device according to a second embodiment. FIG. 9 is a diagram illustrating a configuration of a unit pixel according to a third embodiment. 9 is a timing chart illustrating driving of the solid-state imaging device according to the third embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

<First embodiment>
FIG. 1 is a block diagram illustrating a configuration of an imaging device 150 according to the first embodiment of the present invention. In FIG. 1, a solid-state imaging device 100 converts a subject image formed by a photographing lens 101 into a signal value. The lens driving unit 102 drives a focus lens and a diaphragm of the photographing lens 101 according to an instruction from the overall control calculation unit 103.

  The overall control calculation unit 103 performs overall control of the imaging device 150, including control of the solid-state imaging device 100 and correction and development of signals output from the solid-state imaging device 100. The memory unit 104 temporarily stores image data, and the display unit 105 displays various information and captured images.

  The recording unit 106 is a removable recording medium such as a semiconductor memory in which image data and the like are recorded. The operation unit 107 is an interface for a user to input various instructions to the imaging device 150. The overall control calculation unit 103 controls each block of the imaging device 150 based on a signal output from the operation unit 107.

  FIG. 2 is a diagram illustrating a configuration of the solid-state imaging device 100. The solid-state imaging device 100 according to the present embodiment outputs a signal value corresponding to light incident on the unit pixel 200 during the exposure period. At this time, a signal value corresponding to the light incident on the subframe obtained by dividing the exposure period is internally added to generate a signal value of one frame corresponding to the entire exposure period. The details of this driving method will be described later.

  The solid-state imaging device 100 includes a pixel unit 201 in which a large number of unit pixels 200 are arranged in a matrix, a column signal line 202, an addition circuit 203, a memory 204, a signal processing circuit 205, and a timing control circuit 206. The timing control circuit 206 outputs a drive signal to the pixel unit 201 to control the pixel unit 201, and also controls the addition circuit 203, the memory 204, and the signal processing circuit 205.

  The column signal line 202 is a signal line that transmits the output of the unit pixel 200 to the addition circuit 203. Then, under the control of the timing control circuit 206, the plurality of unit pixels 200 arranged in the pixel unit 201 repeatedly output a signal value corresponding to the incident light to the column signal line 202 at a subframe interval described later.

  The addition circuit 203 sequentially adds the signal values output from the unit pixels 200 for each unit pixel 200, and outputs the result to the memory 204. This addition process corresponds to a process of mixing signal values, and the addition circuit 203 functions as a mixing unit.

  The memory 204 can hold the signal value output from the addition circuit 203 and output the held signal value to the addition circuit 203 and the signal processing circuit 205.

  Specifically, when the output of the unit pixel 200 corresponding to the first sub-frame of one frame used for generating an image is input to the addition circuit 203, the addition circuit 203 outputs this signal value to the memory 204. I do. The memory 204 holds this signal value.

  When the output of the unit pixel 200 corresponding to the subsequent subframe is input to the addition circuit 203, the signal value held in the memory 204 is output to the addition circuit 203, and the output of the unit pixel 200 and the signal value held in the memory 204 are stored. The added signal values are added and held in the memory 204 again.

  By repeating such driving, a signal obtained by sequentially accumulating the signal values for each unit pixel 200 is generated by the addition circuit 203 and stored in the memory 204. After the signal value of one frame is generated, the memory 204 outputs the signal value of one frame to the signal processing circuit 205.

  The signal processing circuit 205 performs various corrections such as black level correction on the input signal value of one frame, and outputs the corrected image signal to the overall control operation unit 103.

  Next, FIG. 3 is a diagram illustrating a configuration of the unit pixel 200. FIG. 4 is a timing chart showing sub-frame driving of the solid-state imaging device 100. The drive timing of the solid-state imaging device 100 according to the present embodiment will be described with reference to FIGS.

  The unit pixel 200 can be selectively driven in either a state where photons are detected or a state where photons are not detected. Then, either a “detection subframe” for detecting a photon or a “non-detection subframe” for not detecting a photon is selected for each subframe. By repeating a plurality of sub-frame driving, a signal value of the unit pixel 200 in one frame (a predetermined exposure period) is generated.

  FIG. 4A illustrates driving in a “detection subframe” in which a photon is detected by the unit pixel 200, and FIG. 4B illustrates driving in a “non-detection subframe” in which a photon is not detected by the unit pixel 200. ing. The driving of one frame will be described later with reference to FIG.

  As shown in FIG. 3, the unit pixel 200 includes an avalanche photodiode (hereinafter, APD) 300, a quench transistor 301, a waveform shaping circuit 302, and a counter 303. Further, it is provided with a latch 304, a control circuit 305, an AND circuit 306, and a selection switch 307.

  In FIG. 4, “SubVD”, “RST”, and “Latch” represent drive signals commonly input from the timing control circuit 206 to the unit pixels 200 in the pixel unit 201. “APD” indicates a potential of a node between the APD 300 and the quench transistor 301. The “waveform shaping circuit output” indicates the output of the waveform shaping circuit 302. “Count value” indicates the count value of the counter 303. “Latch EN” indicates a latch EN signal output from the control circuit 305. Further, “latch output” indicates the output of the latch 304.

  The APD 300 is connected to the power supply VAPD for APD and to the power supply voltage VDD via the quench transistor 301, and is configured to be able to apply a reverse bias. The APD 300 generates a large number of charges due to avalanche multiplication when a photon enters and charges are generated by photoelectric conversion in a state where the potential difference of the reverse bias is higher than the breakdown voltage.

  The generated charge is discharged through the quench transistor 301 at a certain time. The current generated by discharging the generated charges through the quench transistor 301 is called an avalanche current. Due to this operation, at the node between the APD 300 and the quench transistor 301, a fluctuation occurs in which the potential drops as a photon enters the APD. Thus, the APD 300 and the quench transistor 301 function as a light receiving unit.

  The waveform shaping circuit 302 is formed of, for example, an inverter circuit, and generates a pulse signal by performing amplification and edge detection with respect to a change in potential due to generation and discharge of electric charges according to the incidence of photons on the APD 300.

  The counter 303 functions as a counting unit, counts pulse signals output from the waveform shaping circuit 302, and outputs a count value (count result). The count value of the counter 303 is reset to 0 by the drive signal RST output from the timing control circuit 206. The latch 304 holds the count value output from the counter 303 in accordance with the timing of the pulse output from the AND circuit 306.

  The control circuit 305 outputs a latch EN signal and a voltage Vqnc according to the drive signal SubVD output from the timing control circuit 206 and the mode selection signal mode output from the overall control operation unit 103.

  The drive signal SubVD is a signal indicating the start timing of the sub-frame, and the mode selection signal mode is a signal for setting the drive mode of the unit pixel 200.

  The control circuit 305 determines whether the subframe is a “detection subframe” or a “non-detection subframe” according to the mode selection signal mode. Then, it outputs a latch EN signal and a gate voltage Vqnc according to the determination result. With this control, the control circuit 305 functions as a selection unit.

  The latch EN signal is a signal for controlling whether or not the count value of the counter 303 is held by the latch 304 or not, and for each subframe, a High level (hereinafter referred to as “H”) or a Low level (hereinafter referred to as “L”). Is output. Specifically, “H” is output in “detected subframes”, and “L” is output in “non-detected subframes”.

  The voltage Vqnc is a voltage supplied to the gate of the quench transistor 301. By controlling the gate voltage Vqnc of the quench transistor 301, the control circuit 305 selects one of the APD 300 and the avalanche current according to the photon.

  Specifically, by setting the gate voltage Vqnc to a voltage exceeding the threshold voltage of the quench transistor 301, the quench transistor 301 is turned on. In this state, the charges generated by the avalanche multiplication in the APD 300 are discharged to the power supply voltage VDD with a certain time. Then, after discharging the charges, the APD 300 is again subjected to a reverse bias equal to or higher than the breakdown voltage, and again enters an avalanche current state in accordance with photons.

  On the other hand, by controlling the gate voltage Vqnc to the same voltage as the power supply voltage VDD, the quench transistor 301 is turned off. In this state, the charge caused by the avalanche multiplication once generated in the APD 300 is not discharged, and the node between the APD 300 and the quench transistor 301 is maintained at a low potential. In this state, since the reverse bias applied to the APD 300 does not become higher than the breakdown voltage, avalanche multiplication does not occur even if a photon enters, and no avalanche current is generated.

  As described above, by controlling the gate voltage Vqnc, the control circuit 305 can select either the state in which the avalanche current is generated or the state in which the avalanche current is not generated for the APD 300 in accordance with the incidence of photons. Then, in the “detection sub-frame”, the state is controlled to generate an avalanche current, and in the “non-detection sub-frame”, the state is controlled to generate no avalanche current.

  To the AND circuit 306, a latch EN signal output from the control circuit 305 and a drive signal Latch output from the timing control circuit 206 are input. In the subframe in which the latch EN signal is “H”, the AND circuit 306 outputs “H” at the timing when the drive signal Latch becomes “H” level, and at this timing, the latch 304 holds the count value of the counter 303. .

  The drive signal SEL output from the timing control circuit 206 is input to the selection switch 307. The count value held by the latch 304 at the timing when the drive signal SEL is “H” is output to the column signal line 202 as a signal value corresponding to the light incident on the unit pixel 200.

  The drive signal SEL is input as an individual drive signal to each of the unit pixels 200 in the pixel unit 201. When the drive signal SEL sequentially becomes “H”, the signal value of the unit pixel 200 in the pixel unit 201 is changed. The signals are sequentially output to the column signal line 202.

  First, the driving of the solid-state imaging device 100 in the “detection sub-frame” will be described with reference to FIG.

  At timing t400, the drive signal SubVD becomes “H”, and the subframe starts. At the same time, the drive signal RST becomes “H”, the count value of the counter 303 is reset to 0, and the latch EN signal becomes “H”. The drive signal RST becomes “L” after the time required for resetting the count value of the counter 303 has elapsed, and the reset is released. At this timing, the output of the latch 304 is set to the count value held before this subframe.

  At timing t401, a photon enters the APD 300, and an avalanche current is generated. Specifically, the potential at the node between the APD 300 and the quench transistor 301 drops due to the charge due to the avalanche multiplication, and thereafter the charge is gradually discharged from the quench transistor 301, and the potential of the node becomes the power supply voltage VDD. The waveform shaping circuit 302 detects an edge due to a drop in the potential of the node and outputs a signal level “H”.

  Then, a transition is made to the signal level “L” as the potential of the node rises, thereby outputting a pulse corresponding to the incident photon. The counter 303 counts this pulse, and the count value increases to 1. At timings t401 to t402, the state where the counter 303 counts the pulses generated according to the incident photons is maintained.

  The above operation is repeated every time a photon enters the APD 300 until timing t403, which is the timing immediately before the end of the subframe period.

  At timing t403, the driving signal Latch becomes “H”, the count value of the counter 303 is taken in by the latch 304, and the count value is held by the latch 304 when the driving signal Latch becomes “L”. In the example of FIG. 4A, ten photons are detected, and the output of the latch 304 is ten. The timing at which the drive signal Latch becomes “H” is a predetermined time before the subframe end timing at which the next drive signal SubVD is input, taking into account the time required for the latch 304 to hold the count value. It is desirable that the timing is as follows.

  The count value held by the latch 304 is determined by the number of photons incident on the APD 300 during a period from the timing when the drive signal RST becomes “L” and the reset is released to the time when the drive signal Latch becomes “L” and the holding is completed. The corresponding count value is obtained. That is, the photon counting period does not completely coincide with the subframe period. However, the difference between this period and the period of the subframe is very small with respect to the period of the subframe, and thus can be regarded as a count value corresponding to the photons incident in this subframe.

  Subsequently, the driving of the solid-state imaging device 100 in the “non-detection sub-frame” will be described with reference to FIG.

  At timing t404, the drive signal SubVD becomes “H”, and the subframe starts. Unlike the “detection subframe”, the latch EN signal becomes “L”. Further, the gate voltage Vqnc (not shown) becomes the power supply voltage VDD, and the quench transistor 301 is turned off.

  At timing t405, photons enter the APD 300 and charge is generated by avalanche multiplication, so that the potential of the node between the APD 300 and the quench transistor 301 drops. However, since the quench transistor 301 is in a non-conductive state, this charge is not discharged, and no avalanche current is generated.

  As described above, since the potential of the node does not change from the lowered state, the reverse bias applied to the APD 300 does not exceed the breakdown voltage, and avalanche multiplication does not occur even if photons enter. In the waveform shaping circuit 302, the signal level becomes “H” in accordance with the drop in the potential of the node, and the count value of the counter 303 becomes 1.

  At timing t406, the drive signal Latch becomes “H”, but the latch EN signal is “L”, so that the output of the AND circuit 306 remains “L”, and the latch 304 is held before this subframe. The count value is kept.

  As described above, in the “non-detection subframe”, no avalanche current is generated regardless of the presence / absence of photons. As a result, in the unit pixel 200 that drives the “non-detection sub-frame”, the power consumption generated in the unit pixel 200 is significantly reduced under the condition where the photon enters, as in the “detection sub-frame” where the photon does not enter. It becomes possible.

  Subsequently, the drive timing of the solid-state imaging device 100 in the present embodiment will be further described with reference to FIG. FIG. 5 is a timing chart showing the driving of the solid-state imaging device for acquiring an image of one frame.

  In FIG. 5, “VD” is a signal indicating the start timing of one frame, and is input from the overall control operation unit 103 to the solid-state imaging device 100. “Mode: A”, “Mode: B”, and “Mode: C” represent operation examples of the unit pixel 200 in which the mode selection signal mode is A, B, and C.

  Specifically, the pixel unit 201 is divided into a plurality of divided regions in a matrix, and any one of the modes A, B, and C is set in each divided region according to the amount of incident light (imaging conditions). Assigned.

  That is, mode A is assigned to a divided area with a small amount of incident light, mode B is assigned to a divided area with an intermediate amount of incident light, and mode C is assigned to a divided area with a large amount of incident light. The setting of the divided area and the mode will be described later with reference to FIG.

  In FIG. 5, “operation” indicates whether the unit pixel 200 is driven in the “detected subframe” or the “non-detected subframe” in each subframe.

  As for “count value”, the solid line indicates the count value of the counter 303, and the broken line indicates the signal value output from the latch 304. “SEL” schematically represents driving in which signal values of the unit pixels 200 in the pixel unit 201 are sequentially output by the driving signal SEL output from the timing control circuit 206.

  “Addition processing” indicates the timing at which the addition circuit 203 adds the output of the unit pixel 200 and the signal value of the same unit pixel 200 accumulated in the memory 204 and added. Further, “sensor output” indicates the timing at which a signal of one frame held in the memory 204 is output from the solid-state imaging device 100 to the overall control operation unit 103 via the signal processing circuit 205.

  Subsequently, the operation of the solid-state imaging device 100 in accordance with the drive timing shown in FIG. 5 will be specifically described.

  At timing t500, the drive signal VD becomes “H”, and driving of one frame is started. At the same time, the drive signal SubVD also becomes “H”, and the first sub-frame of sub-frames (divided exposure periods) obtained by dividing a preset exposure period of one frame at equal intervals is started. At the start of one frame, the mode selection signal mode is input to the unit pixel 200 from the overall control calculation unit 103 for each divided region.

  In the first sub-frame of one frame, all unit pixels 200 are driven in the “detection sub-frame” regardless of the mode selection signal mode. Thereby, the count value of the counter 303 increases according to the incidence of photons.

  At timing t501, the drive signal SubVD becomes "H" again, the first subframe ends, and the second subframe starts. At this timing, the latch 304 holds a signal value corresponding to a photon incident on the unit pixel 200 in the first subframe.

  When the drive signal SEL sequentially becomes “H”, a signal value corresponding to a photon incident in the first subframe of the unit pixel 200 in the pixel unit 201 is sequentially sent to the addition circuit 203 via the column signal line 202. Is output.

  At the timing when the signal value of the first subframe is input to the addition circuit 203, the addition circuit 203 outputs the signal value of the unit pixel 200 to the memory 204 without adding the signal value, and the memory 204 holds this signal value. .

  In the second sub-frame, the unit pixel 200 to which the mode A is assigned by the mode selection signal mode is driven again as a “detection sub-frame”. On the other hand, the unit pixel 200 to which the mode B and the mode C are assigned by the mode selection signal mode is driven as a “non-detection sub-frame”.

At timing t502, the drive signal SubVD becomes "H" again, the second sub-frame ends, and the third sub-frame starts.
In the unit pixel 200 to which the mode A is assigned and which is driven as a “detection subframe” in the second subframe, a signal value corresponding to the photon incident in the second subframe in the latch 304 is held.

  On the other hand, in the unit pixel 200 in which the mode B and the mode C are allocated and the second sub-frame is driven as the “non-detection sub-frame”, the signal value corresponding to the photon incident on the latch 304 in the first sub-frame is held. Continue to be.

  Then, in the adding circuit 203, the output of the unit pixel 200 and the signal value of the unit pixel 200 held in the memory 204 are added, and the process of being held in the memory 204 again is performed. That is, the signal value corresponding to the unit pixel 200 to which the mode A is assigned, which is stored in the memory 204, is a signal value corresponding to the photon incident on the first and second subframes.

  On the other hand, the signal value corresponding to the unit pixel 200 assigned to the mode B and the mode C, which is held in the memory 204, is twice as large as the signal value corresponding to the photon incident on the first subframe.

  In the third sub-frame, the unit pixel 200 to which the mode A and the mode B are assigned by the mode selection signal mode is driven as a “detection sub-frame”. On the other hand, the unit pixel 200 to which the mode C is assigned by the mode selection signal mode is driven as a “non-detection subframe”.

  At timing t503, the third subframe ends and the fourth subframe starts. In the fourth sub-frame, the unit pixel 200 to which the mode A and the mode C are assigned by the mode selection signal mode is driven as a “detection sub-frame”. On the other hand, the unit pixel 200 to which the mode B is assigned by the mode selection signal mode is driven as a “non-detection subframe”.

  As described above, the unit pixel 200 to which the mode A is assigned by the mode selection signal mode is always driven in the “detection subframe”. Further, the unit pixel 200 to which the mode B is assigned by the mode selection signal mode is driven so as to alternately repeat the “detection sub-frame” and the “non-detection sub-frame”.

  Furthermore, the unit pixel 200 to which the mode C is assigned by the mode selection signal mode is driven once in the “detected sub-frame” and then driven twice in the “non-detected sub-frame”. In other words, the control is performed such that the greater the amount of light incident on the unit pixel 200, the longer the period (accumulated time) of the “non-detection sub-frame” in which no avalanche current occurs.

  At the timing t504, the repetition of the subframe a preset number of times ends, and the exposure period ends. After the end of the exposure period, the unit pixel 200 is driven in the “non-detection sub-frame” regardless of the mode selection signal mode. After the cumulative addition of the signal values corresponding to the photons incident in the last subframe is completed, the retained signal values are sequentially output from the memory 204 to the outside of the solid-state imaging device 100 via the signal processing circuit 205. Thus, the output of the image of one frame of the solid-state imaging device 100 is completed.

  Subsequently, the setting of the mode for each divided region in the pixel unit 201 will be described with reference to FIG. FIG. 6 is a diagram illustrating a drive mode in a divided region in the pixel unit.

  The imaging area 600 in the outer frame in FIG. 6 corresponds to the area of the pixel unit 201, and the background image shows an example of a scene to be captured. The divided area 601 indicates one area obtained by dividing the imaging area 600. In the divided area 601, a plurality of unit pixels 200 are arranged in a matrix.

  Symbols A, B, and C in the divided area 601 represent an example of a mode setting by a mode selection signal mode for controlling the unit pixels 200 arranged in the divided area 601. The mode selection signal mode for each divided region 601 is calculated by the overall control calculation unit 103 from the signal value of the previous frame output from the solid-state imaging device 100 immediately before the frame to be imaged.

  Specifically, an average value is calculated for each divided region 601 with respect to the signal value of the image of the previous frame, and the amount of incident light in that region is estimated. Then, by comparing the average value with a predetermined threshold value, a mode selection signal mode for each divided region 601 is determined according to the comparison result. The overall control operation unit 103 performs the above operation before the start of a frame in which imaging is performed, and outputs a mode selection signal mode to the solid-state imaging device 100.

  In the present embodiment, the mode selection signal mode for each divided area 601 is determined based on the signal of the previous frame. However, the present invention is not limited to this. For example, by separately providing a photometric sensor (not shown) and measuring the amount of light incident on each pixel area 601 in this photometric sensor, the pixel area 601 is determined by the mode selection signal mode. May be determined.

  Next, a semiconductor chip structure of the solid-state imaging device according to the present embodiment will be described with reference to FIG.

  FIG. 7A is a diagram illustrating that the solid-state imaging device 100 is configured by stacking two semiconductor substrates on each other. FIG. 7B is a diagram illustrating which of the two semiconductor substrates each component of the unit pixel 200 is arranged.

  In the solid-state imaging device 100, the unit pixel 200 includes many components, and particularly, since the counter 303 is configured by a multi-bit counter, the circuit scale becomes large. Therefore, it is preferable that the image sensor 100 be realized by a stacked structure in which a plurality of semiconductor substrates are connected in pixel units.

  As shown in FIG. 7A, the solid-state imaging device 100 includes an upper semiconductor substrate 700 that receives light that has passed through the imaging lens 101, and a lower substrate 701 that mainly includes digital circuits.

  The upper pixel 702 on the upper semiconductor substrate 700 side includes the APD 300 of the unit pixel 200 and the quench transistor 301 as shown in FIG. On the other hand, the lower pixel 703 on the lower semiconductor substrate 701 side includes the waveform shaping circuit 302, the counter 303, the latch 304, the control circuit 305, the AND circuit 306, and the selection switch 307 of the unit pixel 200. The addition circuit 203, the memory 204, the signal processing circuit 205, and the timing control circuit 206 are arranged on the lower semiconductor substrate 701.

  With such a configuration, the solid-state imaging device 100 can secure a sufficient opening area for the upper pixel 702 on the upper semiconductor substrate 700 side to allow photons to enter the APD 300. Further, on the lower semiconductor substrate 701 side, a circuit area for disposing the counter 303 can be secured.

  However, the semiconductor chip structure of the solid-state imaging device 100 can be freely changed according to the purpose and application. For example, a stacked structure of three or more semiconductor substrates may be used, or a single chip may be used. Further, each of the plurality of semiconductor substrates may be manufactured according to different process rules.

  In the present embodiment, the unit pixel 200 is periodically driven as a “non-detection sub-frame” in which an avalanche current does not occur in accordance with the amount of light incident on the unit pixel 200, so that the power consumption associated with the irradiation of high-illuminance light is The increase can be suppressed.

  In addition, in the unit pixel 200 having low power consumption and a small amount of incident light, photons are always detected during the exposure period. Therefore, an image generated by the signal value of the unit pixel 200 does not deteriorate image quality.

  On the other hand, in the unit pixel 200 having a large incident light amount, the number of photons to be detected is reduced by driving a part of the sub-frames in the “non-detection sub-frame”. As a result, the influence of the light shot noise increases, and the image quality generated by the signal value of the unit pixel 200 deteriorates.

  However, since the number of photons counted per unit time is large, the number of photons to be detected does not become too small as compared with the divided region 601 where the amount of incident light is small, and is hardly visually recognized as image noise. That is, it is possible to suppress an increase in power consumption due to the irradiation of the high-luminance light while suppressing the deterioration of the image quality.

  Further, as described above, in the “non-detection sub-frame”, the unit pixel 200 outputs the signal value in the previous “detection sub-frame”, and the addition circuit 203 performs the cumulative addition. Accordingly, even during a period in which a photon due to the “non-detection sub-frame” is not detected, the signal value in the immediately preceding “detection sub-frame” that is close in time and has high correlation is output, and the time for detecting the photon between the unit pixels 200 is reduced. Can be interpolated.

  Further, by driving the detection sub-frame periodically during the exposure period in the unit pixel 200 having a large amount of incident light, the correlation can be improved even if the amount of light incident on the unit pixel 200 per unit time changes. it can. As a result, it is possible to suppress the deterioration of the image quality due to the driving of the “non-detection sub-frame”.

<Second embodiment>
Hereinafter, an imaging device according to a second embodiment of the present invention will be described. In the second embodiment, the number of times of driving in the “non-detection sub-frame” is changed for each unit pixel 210 based on a signal value corresponding to a photon incident in a detection sub-frame in the same frame.

  In addition, in the second embodiment, in the still image mode for saving images and the live view mode for displaying images in real time on the display unit 105 of the imaging device without saving images, the “non- The number of “detection subframes” is made different.

  Hereinafter, points of the present embodiment different from the first embodiment will be described. The solid-state imaging device according to the second embodiment has the same configuration as the solid-state imaging device 100 according to the first embodiment except for the unit pixel 210.

  The configuration of the unit pixel 210 according to the second embodiment of the present invention will be described with reference to FIG. FIG. 8 is a diagram illustrating a configuration of a unit pixel 210 according to the second embodiment. Among the constituent elements of the unit pixel 210, constituent elements 800 to 807 correspond to the constituent elements 300 to 307 of the unit pixel of the first embodiment shown in FIG. 3, and their functions are the same as those of the first embodiment. is there.

  The unit pixel 210 is different from the unit pixel 200 of the first embodiment in that the unit pixel 210 includes a threshold value determination circuit 808. The threshold value determination circuit 808 compares the signal value held by the latch 804 in the “detection sub-frame” with a predetermined threshold value, and performs “non-detection” performed after the “detection sub-frame” according to the magnitude of the signal value. The number of “subframes” is output to the control circuit 805. The control circuit 805 has a counter (not shown) therein and performs control so as to repeat the “non-detection sub-frame” according to the number of “non-detection sub-frames”.

  As described above, the threshold value determination circuit 808 is provided in the unit pixel 210, and controls the number of times of performing the “non-detection sub-frame” according to the signal value in the “detection sub-frame”. Accordingly, power consumption generated in the unit pixel 210 can be reduced in accordance with the amount of light incident on each unit pixel 210.

  Therefore, as compared with the first embodiment in which control is performed for each of the divided areas 601, it is possible to further suppress deterioration in image quality of a subject in which the light amount in the divided area 601 is not uniform. Further, since the signal value used for determining the number of “non-detection sub-frames” is a signal corresponding to the amount of light in the same frame, temporal correlation is high. Therefore, it is possible to more desirably set the balance between the image quality and the power consumption for a subject in which the amount of light incident per unit time changes.

  Subsequently, the sub-frame driving in the second embodiment will be described with reference to FIG. FIG. 9A illustrates driving in a “detection sub-frame” for detecting photons, and FIG. 9B illustrates driving in a “non-detection sub-frame” for detecting no photons.

  9, “Cs1”, “Cs2”, and “Cs3” represent thresholds used for the determination by the threshold determination circuit 808, and will be described in detail with reference to FIG. The “output of the threshold determination circuit” indicates the output of the threshold determination circuit 808. Others are the same as the first embodiment shown in FIG.

  First, the “detection subframe” will be described. At timing t900, the drive signal SubVD becomes “H”, and the subframe starts. The latch 804 holds the signal value in the previous “detection subframe”, and the output of the threshold value determination circuit 808 also has a value corresponding to the signal value.

  After that, the first photon enters the APD 800, and an avalanche current is generated. Specifically, the potential at the node between the APD 800 and the quench transistor 801 drops due to the charge due to the avalanche multiplication, and thereafter the charge is gradually discharged from the quench transistor 801, and the potential of the node becomes the power supply voltage VDD.

  The waveform shaping circuit 802 detects an edge due to a drop in the potential of the node, and outputs a signal level “H”. Then, a transition is made to the signal level “L” as the potential of the node rises, thereby outputting a pulse corresponding to the incident photon. The counter 803 counts this pulse, and the counted value increases to 1.

  The above operation is repeated every time a photon enters the APD 800 until a timing t901, which is a timing immediately before the end of the subframe period.

  At timing t901, the drive signal Latch becomes “H”, and the latch 804 holds a signal value corresponding to the photon incident on the “detection subframe”. As a result, the threshold value determination circuit 808 compares the signal value with the threshold value. In the example shown in FIG. 9A, the signal value is larger than Cs2 and smaller than Cs3, so that the output of the threshold value determination circuit 808 is 2.

  Based on the threshold determination result, whether the next sub-frame is driven by the “detection sub-frame” or “non-detection sub-frame” is determined. Therefore, the timing when the drive signal Latch becomes “H” is This is performed at a timing before the end timing of the frame in consideration of a time required for control.

  Also in this embodiment, the time during which no photon is detected after the latch 804 holds the signal value is very small with respect to the period of the subframe. Therefore, the signal value held by the latch 804 is incident on the subframe. It can be regarded as a signal value corresponding to the photon.

  In the “non-detection sub-frame”, the latch 804 keeps holding the signal value in the previous “detection sub-frame”, so that the output of the threshold value determination circuit 808 does not change with the value corresponding to this signal value.

  Subsequently, the drive timing of the solid-state imaging device according to the present embodiment will be described with reference to FIG. FIG. 10 is a timing chart showing the driving of the solid-state imaging device for acquiring an image of one frame.

  FIG. 10A is a diagram illustrating the relationship between the signal value of the “detected subframe” and the number of “non-detected subframes” output from the threshold value determination circuit 808 and performed immediately after in the still image mode. FIG. 10B illustrates the same relationship in the live view mode. FIG. 10C is a timing chart showing driving of one frame in the unit pixels 210 having different numbers of non-detections.

  As shown in FIG. 10A, the threshold values Cs1, Cs2, and Cs3 are set so that the number of “non-detection subframes” performed immediately after the signal value in the “detection subframe” increases. You. At this time, the threshold value Cs1 is a threshold value at which the number of “non-detection subframes” becomes 1, and Cs2 and Cs3 similarly indicate a threshold value at which the number of “non-detection subframes” becomes 2, 3 respectively. .

  On the other hand, as shown in FIG. 10B, in the live view mode, even if the signal value in the “detection subframe” is the same as in the still image mode, the number of “non-detection subframes” performed immediately after Threshold values Cm1, Cm2, Cm3, Cm4, and Cm5 are set so as to increase.

  Unlike the still image mode, the live view mode does not store images, but requires a high frame rate so that changes in the subject can be reflected in real time. In addition, continuous shooting takes a long time in the live view mode. Therefore, in the live view mode, it is desirable that reduction of power consumption be prioritized over image quality as compared with the still image mode. For this reason, in the live view mode, the threshold is set so that the number of “non-detected subframes” increases.

  As shown in FIG. 10C, the unit pixel 210 where the amount of incident light is small and the number of non-detections is 0 is always driven in the “detection sub-frame” during the exposure period. On the other hand, the unit pixel 210 whose non-detection frequency is 1 is driven such that the “non-detection sub-frame” is performed once after the “detection sub-frame”. Similarly, the unit pixel 210 whose non-detection frequency is 2 is driven such that the “non-detection sub-frame” is performed twice after the “detection sub-frame” once. In addition, the unit pixel 210 whose non-detection frequency is 3 is driven so that the “non-detection sub-frame” is performed three times after the “detection sub-frame” is performed once.

  In this way, the control is performed so that the number of “non-detection subframes” in the accumulation period increases as the amount of light incident on the unit pixel 210 increases.

  FIG. 10C shows an example in which the amount of light incident on the unit pixel 210 per unit time does not substantially change within the exposure period. The determination of the number of non-detections in the threshold value determination circuit 808 is performed for each “detection sub-frame”, so if the amount of light incident on the unit pixel 210 per unit time changes within the exposure period, the “non-detection sub-frame” Also changes.

  In the imaging apparatus according to the present embodiment, similarly to the first embodiment, it is possible to suppress an increase in power consumption due to irradiation of high-luminance light while suppressing deterioration in image quality. In addition, the number of non-detection subframes is controlled using the signal value in the same frame for each unit pixel 210. Therefore, in a subject in which the amount of light incident on the divided area 601 is not uniform or in a subject in which the amount of light incident per unit time changes during the exposure period, a more desirable balance between image quality and power consumption compared to the first embodiment. Can be taken.

  Further, by setting different thresholds for judging the number of “non-detection sub-frames” in the still image mode and the live view mode, a more desirable balance between image quality and power consumption can be achieved depending on the driving mode of the imaging device. Becomes possible.

<Third embodiment>
Hereinafter, an imaging device according to a third embodiment of the present invention will be described. In the third embodiment, the driving of reading the signal value from the unit pixel 220 to the addition circuit 203 is performed once for a plurality of subframes, so that the power consumption in the addition circuit 203 and the memory 204 is reduced.

  Hereinafter, the points of this embodiment different from the first and second embodiments will be described. The solid-state imaging device according to the third embodiment has the same configuration as the solid-state imaging device 100 according to the first embodiment except for the unit pixel 220.

  The configuration of the unit pixel 220 according to the third embodiment of the present invention will be described with reference to FIG. FIG. 11 is a circuit diagram illustrating a configuration of a unit pixel 220 according to the third embodiment. Among the constituent elements of the unit pixel 220, the constituent elements 1100 to 1108 correspond to the constituent elements 800 to 808 of the unit pixel of the second embodiment shown in FIG. 8 (excluding the AND circuit 806), and the respective functions are the same as those of the second embodiment. This is the same as the second embodiment.

  The unit pixel 220 according to the second embodiment is that the drive signal Judge output from the timing control circuit 206 is input to the threshold value determination circuit 1108 and the drive signal RST is input to the control circuit 1104. Different from the pixel 210.

  The second embodiment is that the total number of non-detections, which is the number of times the control circuit 1104 drives the “non-detection sub-frame”, is output to the latch 1105, and that the latch 1105 is directly controlled by the drive signal Latch. Is different from the unit pixel 210.

  Further, the latch 1105 holds the total number of non-detections together with the count value of the counter 1104, and outputs the signal value and the number of non-detections to the addition circuit 203 via the selection switch 1107. different.

  The control circuit 1105 has a counter (not shown), counts the number of times the “non-detection sub-frame” has been driven, and outputs the result as the total number of non-detection sub-frames. The total number of non-detections is reset to 0 after the total number of non-detections is held in the latch 1105 according to the drive signal RST.

  The threshold value determination circuit 1108 determines the count value of the counter 1104 according to the drive signal Judge and holds the determination result. The determination of the count value is performed in the first subframe in one frame and in the subframe immediately after the signal value output to the addition circuit 203 is held in the latch 1105.

  The control circuit 1105 varies the number of times the “non-detection sub-frame” is driven based on the determination result. The latch 1105 holds and outputs the total number of non-detections together with the count value of the counter 1104 according to the drive signal Latch.

  When the selection switch 1107 is controlled by the drive signal SEL, the total number of non-detections is also output to the addition circuit 203 at the same timing as the signal value held by the latch 1105 is output to the addition circuit 203. The total number of non-detections output to the addition circuit 203 is cumulatively added in the same manner as the signal value, and is stored in the memory 204.

  The signal value and the cumulative result of the total number of non-detections are output to the signal processing circuit 205, respectively. The signal processing circuit 205 corrects the difference in the length of time for detecting photons between the unit pixels 220 caused by driving the “non-detection sub-frame” based on the accumulation result of the total number of non-detection.

  Subsequently, the drive timing of the solid-state imaging device according to the present embodiment will be described with reference to FIG. FIG. 12 is a timing chart illustrating the driving of the solid-state imaging device for acquiring an image of one frame. FIG. 12 illustrates an example in which driving is performed once every six subframes to read a signal value from the unit pixel 220 to the addition circuit 203.

  In FIG. 12, “Judge” is a drive signal commonly input to the unit pixels 220 in the pixel unit 201. The “total number of non-detected latch outputs” indicates the total number of non-detected latches held by the latch 1105.

  “Number of non-detections: 0” indicates a driving example of the unit pixel 220 in which the amount of light incident on the unit pixel 220 is small and the count value of the counter 1103 is smaller than the threshold Cs1 at the timing of performing the threshold determination.

  “Number of non-detections: 1” indicates a driving example of the unit pixel 220 in which the count value of the counter 1103 is equal to or larger than the threshold Cs1 and smaller than the threshold Cs2.

  Similarly, “number of non-detections: 2” indicates a driving example of the unit pixel 220 in which the count value of the counter 1103 is equal to or larger than the threshold Cs2 and smaller than the threshold Cs3.

  The above driving example is an example of driving the unit pixels 220 having different incident light amounts, and at the timing of performing the threshold determination, according to the count value of the counter 1103, as in the solid-state imaging device according to the second embodiment. The number of “non-detection subframes” is set. The “detected sub-frame” and the “non-detected sub-frame” are repeated according to the setting until the next threshold determination is performed.

  More specifically, for example, in the unit pixel 220 whose count value of the counter 1103 is equal to or more than the threshold value Cs3, the drive that repeats the “non-detection sub-frame” three times after the “detection sub-frame” performs the threshold determination next. Is repeated until.

  Subsequently, the driving of the solid-state imaging device will be further described. At timing t1200, the signal VD becomes “H”, and driving of one frame is started. At the same time, the drive signal SubVD also becomes “H”, and the first subframe among the subframes obtained by dividing the preset exposure period at equal intervals is started. In the first sub-frame, all the unit pixels 220 are driven in the “detection sub-frame”. At the start of the first sub-frame, the drive signal RST becomes “H”, and the count value of the counter 1103 is reset to 0.

  At timing t1201, the drive signal Judge becomes “H”, and the threshold value determination circuit 1108 performs threshold value determination of the count value of the counter 1103. The number of times the “non-detection sub-frame” is driven in the subsequent sub-frames is controlled according to the determination result of the threshold determination. The timing at which the drive signal Judge becomes “H” and the threshold determination is performed is performed at a timing before the end timing of the first subframe and in consideration of the time required for the threshold determination and the control change of the subsequent subframes. .

  At timing t1202, the drive signal SubVD becomes “H” again, the first subframe ends, and the second subframe starts. Either the “detection sub-frame” or the “non-detection sub-frame” is driven for each unit pixel 220 according to the determination result in the first sub-frame.

  At the start of the second sub-frame, the drive signal RST does not become “H”, and the counter 1103 of the unit pixel 220 driven in the “detection sub-frame” accumulates the count value in the first sub-frame. , And counts pulse signals generated according to the incident photons. On the other hand, since the pulse signal is not generated in the counter 1103 of the unit pixel 220 driven in the “non-detection sub-frame”, the count value counted in the first sub-frame is held.

  At a timing t1203, the drive signal Latch becomes “H”, and the count value of the counter 1103 and the total number of non-detections are held by the latch 1105. Since the total number of non-detections is obtained by counting the number of times the “non-detection sub-frame” has been driven up to the current sub-frame, the number of times of non-detection: , 0 are retained.

  On the other hand, 3 is held in the unit pixel 220 of the number of non-detections: 1, and 4 is held in the unit pixel 220 of the number of non-detections: 2. After the holding is completed, the count value and the total number of non-detections held by the latch 1105 are output as signal values to the adding circuit 203 when the drive signal SEL sequentially becomes “H”.

  Since the output to the adding circuit 203 is the first in one frame, the adding circuit 203 outputs the signal value of the unit pixel 220 and the total number of non-detections to the memory 204 without adding the signal value and the total number of non-detections. Hold.

  At timing t1204, the drive signal SubVD becomes “H” again, the sixth sub-frame ends, and the seventh sub-frame starts. In the next frame in which the signal value of the unit pixel 220 and the total number of non-detections output to the addition circuit 203 are held in the latch 1105, the same as the first sub-frame, except that the output to the addition circuit 203 is performed in parallel. Is driven.

  At a timing t1205, the repetition of the subframe a preset number of times ends, and the exposure period ends. After the end of the exposure period, all the unit pixels 220 are driven in the “non-detection sub-frame”. When the cumulative addition of the signal value and the total number of non-detections held by the latch 1105 is completed in the last subframe, the stored signal value and the total number of non-detections are sequentially output from the memory 204 to the signal processing circuit 205.

  The signal processing circuit 205 corrects the signal value that has been cumulatively added according to the cumulative addition result of the total number of non-detections. The correction is performed such that in the sub-frame driven in the “non-detection sub-frame”, the same signal value as the average of the signal values in the sub-frame driven in the “detection sub-frame” is output.

  For example, with respect to the signal value obtained by cumulatively adding the unit pixel 220 having the number of non-detections of 1 in FIG. 12, since the number of subframes obtained by dividing the exposure period is 12 and the number of “detection subframes” is 6, 12 / Multiply 6 = 2.

  Similarly, the signal value obtained by cumulatively adding the unit pixels 220 of the number of non-detections: 2 is multiplied by 12/4 = 3. The signal values corrected by the signal processing circuit 205 are sequentially output to the outside of the solid-state imaging device 100, whereby the output of the image of one frame of the solid-state imaging device 100 is completed.

  FIG. 12 shows an example in which the amount of light incident on the unit pixel 220 per unit time does not substantially change within the exposure period. The determination of the number of non-detections in the threshold value determination circuit 1108 is performed every time the signal value of the unit pixel 220 and the total number of non-detections are output to the addition circuit 203.

  Therefore, when the amount of light incident on the unit pixel 220 per unit time changes within the exposure period, the number of times the “non-detection sub-frame” continues also changes. By performing such driving, it is possible to balance more desirable image quality and power consumption for a subject in which the amount of light incident per unit time changes during the exposure period, as in the second embodiment.

  Also in the imaging device of the present embodiment, similarly to the first and second embodiments, it is possible to suppress an increase in power consumption due to irradiation of high-brightness light while suppressing deterioration in image quality. In addition, by performing driving for reading a signal value from the unit pixel 220 to the addition circuit 203 once for a plurality of subframes, power consumption generated in the addition circuit 203 and the memory 204 can be reduced.

  In the present embodiment, the drive for reading the signal value to the adder circuit 203 is performed once every six subframes, but the present invention is not limited to this. Considering the maximum value of the count value of the counter 1103 after a lapse of a plurality of subframes assumed by the control of the threshold value and the number of times of non-detection, the signal value is read in advance within a range not exceeding the maximum value determined by the bit number of the counter 1103. It is desirable to be set.

(Other embodiments)
The present invention supplies a program for realizing one or more functions of the above-described embodiments to a system or an apparatus via a network or a storage medium, and one or more processors in a computer of the system or the apparatus read and execute the program. This processing can be realized. Further, it can be realized by a circuit (for example, an ASIC) that realizes one or more functions.

100: solid-state imaging device, 101: photographing lens, 102: lens drive unit, 103: overall control operation unit, 104: memory unit, 105: display unit, 106: recording unit, 107: operation unit, 200, 210, 220: Unit pixel

Claims (15)

Counting means for counting the number of incident photons; and selecting whether the counting means is set to a first state for counting the number of photons or a second state for not counting the number of photons. A plurality of pixels each having:
Generating means for generating an image corresponding to light incident on the pixel, based on a counting result by the counting means in each of the pixels,
The selecting unit sets the counting unit to the first state in a second exposure period performed after the first exposure period according to a light amount incident on the pixel during the first exposure period. Or setting to set to the second state.   The counting means includes first generating means for generating an avalanche current in response to the incidence of photons, second generating means for generating a pulse based on the waveform of the avalanche current, and the second generating means. The imaging apparatus according to claim 1, further comprising a counter that counts pulses.   3. The imaging apparatus according to claim 2, wherein the selection unit sets the counting unit to the second state by setting the first generation unit to a state where no avalanche current is generated.   4. The imaging apparatus according to claim 1, wherein the selection unit sets the counting unit to the first state during the first exposure period. 5.   The said selection means sets the state of the said counting means so that the light quantity which injects into the said pixel in the said 1st exposure period is so long that the accumulation time of the said 2nd state may become long. Item 5. An imaging device according to Item 4.   The method according to claim 1, wherein the selection unit selects, for each of the plurality of pixels, whether to set the counting unit to the first state or the second state. An imaging device according to item 13.   Each of the first exposure period and the second exposure period is one of a plurality of divided exposure periods obtained by dividing a predetermined exposure period at equal intervals, and the selecting unit includes: The imaging apparatus according to any one of claims 1 to 6, wherein the unit selects whether to set the counting unit to the first state or the second state.   A mixing unit that mixes counting results according to the light incident on the plurality of divided exposure periods, wherein the mixing unit sets the counting result in the divided exposure period in which the counting unit is set to the second state. 8. The imaging apparatus according to claim 7, wherein the counting unit immediately before uses the counting result in the divided exposure period set in the first state.   The selecting unit sets the counting unit to the first state in the second exposure period in accordance with a counting result in the divided exposure period in which the counting unit is set to the first state. The imaging apparatus according to claim 7, wherein whether to set to the second state is selected.   The counting means further includes a comparing means for comparing a counting result in a divided exposure period set in the first state with a threshold value, and the selecting means, in accordance with a comparison result of the comparing means, The imaging apparatus according to claim 9, wherein during the exposure period, whether to set the counting unit to the first state or the second state is selected.   The imaging apparatus according to claim 10, wherein the comparing unit changes the threshold according to a photographing condition.   The imaging apparatus according to any one of claims 1 to 6, wherein the first exposure period is an exposure period performed before the predetermined exposure period. Counting means for counting the number of incident photons; and selecting whether the counting means is set to a first state for counting the number of photons or a second state for not counting the number of photons. A plurality of pixels each having a selection unit, and a generation unit that generates an image corresponding to light incident on the pixel based on a count result of the counting unit in each of the pixels. A way to
The selecting unit sets the counting unit to the first state in a second exposure period performed after the first exposure period in accordance with a light amount incident on the pixel during the first exposure period. Or a step of selecting whether to set to the second state.   A program for causing a computer to execute the control method according to claim 13.   A computer-readable storage medium storing a program for causing a computer to execute the control method according to claim 13.

Images