JP2023039400A - Photoelectric conversion device and photoelectric conversion system - Google Patents

Abstract

To provide a photoelectric conversion device having an APD whose dynamic range can be expanded.SOLUTION: A photoelectric conversion device includes: a circuit disposed between an avalanche photodiode and a power supply; and a counter for counting output signals from the avalanche photodiode. The photoelectric conversion device also includes a memory in which timing information showing that the counter has reached a threshold is written within a predetermined exposure period shorter than an exposure period. In addition, the photoelectric conversion device is configured so that clock signals are input to the circuit in the exposure period.SELECTED DRAWING: Figure 10

Description

The present invention relates to a photoelectric conversion device and a photoelectric conversion system.

A photodetector using an avalanche photodiode (APD) capable of detecting weak light at the single photon level using avalanche (electron avalanche) multiplication is known. The APD forms a high electric field region (avalanche multiplier) with a first semiconductor region of the first conductivity type having the same polarity as the signal charge and a second semiconductor region of the second conductivity type having a polarity different from that of the signal charge.

Patent Document 1 describes a photodetector that controls a standby state in which the APD can perform avalanche multiplication and a recharge state in which the APD is returned to the standby state by a clock signal having a predetermined frequency. Specifically, the clock signal controls on and off of a switch provided between the APD and a power supply that applies a reverse bias to the APD. For example, when the clock signal is at a first level, the switch is turned off and the APD is in standby. Also, when the clock signal is at the second level, the switch is turned on and the APD is recharged. Also, the clock signal is configured to take logic with the output signal from the APD. Therefore, when a photon enters the APD in the standby state, the output signal is output from the APD to the counter at the timing when the clock signal transitions from the first level to the second level.

JP 2020-123847 A

By using the configuration of Patent Document 1, it is possible to suppress the so-called pile-up phenomenon in which photons are not counted in the case of high illuminance. However, in the configuration of Patent Document 1, the count value is saturated when the illuminance becomes higher than a predetermined value, and thus the expansion of the dynamic range is insufficient.

Accordingly, an object of the present invention is to provide a photoelectric conversion device having an APD capable of expanding the dynamic range more than the configuration of Patent Document 1.

A photoelectric conversion device according to the present invention includes a photodiode that performs avalanche multiplication; a first state in which the photodiode is arranged between the photodiode and a power supply; a circuit that switches the diode to a second state in which the diode is not electrically connected to the power supply; a counter that counts the output signal from the photodiode; and a memory in which time information indicating that the count value of the counter has reached a threshold value is written, and a clock signal can be input to the circuit during the exposure period. .

According to the present invention, it is possible to provide a photoelectric conversion device having an APD capable of expanding the dynamic range more than the configuration of Patent Document 1.

A diagram showing a structure of a photoelectric conversion device Arrangement example of sensor board Circuit board layout example Block diagram including an equivalent circuit of a photoelectric conversion element FIG. 4 is a diagram showing the relationship between APD operations and output signals; Block diagrams and equivalent circuit diagrams of pixel circuits in each form The figure which shows the relationship between the illuminance and count value in a comparative example. A diagram showing the relationship between time and count value in each form A diagram showing the relationship between illuminance and count value, and between illuminance and exposure time in the present embodiment. Detailed block diagram of the photoelectric conversion element in this embodiment Timing chart of the photoelectric conversion element in this embodiment A diagram showing the relationship between the number of incident photons and the count value in each form Flow chart of image reconstruction in this embodiment Flow chart of image reconstruction in this embodiment FIG. 11 is a diagram showing clock signals described in the second embodiment; FIG. 11 is a diagram showing clock signals described in the third embodiment; FIG. 11 is a diagram showing clock signals described in the third embodiment; Block diagram of the photoelectric conversion system of the fourth embodiment Block diagram of the photoelectric conversion system of the fifth embodiment Block diagram of the photoelectric conversion system of the sixth embodiment Block diagram of the photoelectric conversion system of the seventh embodiment A diagram showing a specific example of the photoelectric conversion system of the eighth embodiment.

The embodiments shown below are for embodying the technical idea of the present invention, and are not intended to limit the present invention. The sizes and positional relationships of members shown in each drawing may be exaggerated for clarity of explanation. In the following description, the same configuration may be assigned the same number and the description thereof may be omitted. In addition, the configurations described in each embodiment can be replaced or combined with configurations described in other embodiments as long as there is no technical problem.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail based on the drawings. In the following description, terms indicating specific directions and positions (for example, "upper", "lower", "right", "left", and other terms including those terms) are used as necessary. . These terms are used to facilitate understanding of the invention with reference to the drawings, and the technical scope of the present invention is not limited by the meaning of these terms.

In the following description, the anode of the avalanche photodiode (APD) is set to a fixed potential and the signal is extracted from the cathode side. Therefore, the semiconductor region of the first conductivity type in which majority carriers are the same polarity as the signal charges is an N-type semiconductor region, and the semiconductor region of the second conductivity type in which majority carriers are charges of a different polarity from the signal charges. is a P-type semiconductor region. It should be noted that the cathode of the APD may be set at a fixed potential and the signal may be extracted from the anode side. In this case, the semiconductor region of the first conductivity type having majority carriers of the same polarity as the signal charges is a P-type semiconductor region, and the semiconductor region of the second conductivity type having majority carriers of charges having a polarity different from that of the signal charges. is an N-type semiconductor region. A case where one node of the APD is set to a fixed potential will be described below, but the potentials of both nodes may vary.

FIG. 1 is a diagram showing the configuration of a stacked photoelectric conversion device 100. As shown in FIG. The photoelectric conversion device 100 is configured by laminating and electrically connecting two substrates, a sensor substrate 11 and a circuit substrate 21 . The sensor substrate 11 has a first semiconductor layer having photoelectric conversion elements 102, which will be described later, and a first wiring structure. The circuit board 21 has a second semiconductor layer having circuits such as the signal processing unit 103, which will be described later, and a second wiring structure. The photoelectric conversion device 100 is configured by stacking a second semiconductor layer, a second wiring structure, a first wiring structure, and a first semiconductor layer in this order. The photoelectric conversion device described in each embodiment is a back-illuminated photoelectric conversion device in which light enters from the second surface and a circuit board is arranged on the first surface.

In the following description, the sensor substrate 11 and the circuit substrate 21 are diced chips, but they are not limited to chips. For example, each substrate may be a wafer. Further, each substrate may be laminated in a wafer state and then diced, or may be formed into chips from the wafer state and then laminated and bonded to each chip.

A pixel region 12 is arranged on the sensor substrate 11 , and a circuit region 22 for processing signals detected by the pixel region 12 is arranged on the circuit substrate 21 .

FIG. 2 is a diagram showing an arrangement example of the sensor substrate 11. As shown in FIG. Pixels 101 having photoelectric conversion elements 102 including APDs are arranged in a two-dimensional array in plan view to form a pixel region 12 .

The pixels 101 are typically pixels for forming an image, but when used for TOF (Time of Flight), they do not necessarily form an image. That is, the pixels 101 may be used to measure the time and amount of light that light reaches.

FIG. 3 is a configuration diagram of the circuit board 21. As shown in FIG. It has a signal processing unit 103 that processes charges photoelectrically converted by the photoelectric conversion element 102 in FIG. there is

The photoelectric conversion element 102 in FIG. 2 and the signal processing unit 103 in FIG. 3 are electrically connected via connection wiring provided for each pixel.

The vertical scanning circuit unit 110 receives the control pulse supplied from the control pulse generation unit 115 and supplies the control pulse to each pixel. Logic circuits such as shift registers and address decoders are used in the vertical scanning circuit unit 110 .

A signal output from the photoelectric conversion element 102 of the pixel is processed by the signal processing unit 103 . The signal processing unit 103 is provided with a counter, a memory, and the like, and a digital value is written and held in the memory.

The horizontal scanning circuit unit 111 inputs a control pulse for sequentially selecting each column to the signal processing unit 103 in order to read the signal from the memory of each pixel holding the digital signal.

A signal is output to the signal line 113 from the signal processing unit 103 of the pixel selected by the vertical scanning circuit unit 110 for the selected column.

The signal output to the signal line 113 is output to the external recording unit or signal processing unit of the photoelectric conversion device 100 via the output circuit 114 .

In FIG. 2, the photoelectric conversion elements may be arranged one-dimensionally in the pixel region. The function of the signal processing unit does not necessarily have to be provided for each photoelectric conversion element. For example, one signal processing unit may be shared by a plurality of photoelectric conversion elements, and signal processing may be performed sequentially.

As shown in FIGS. 2 and 3, a plurality of signal processing units 103 are arranged in a region overlapping the pixel region 12 in plan view. A vertical scanning circuit portion 110, a horizontal scanning circuit portion 111, a readout circuit 112, an output circuit 114, and a control pulse generation portion 115 are arranged so as to overlap between the edge of the sensor substrate 11 and the edge of the pixel region 12 in plan view. is distributed. In other words, the sensor substrate 11 has a pixel region 12 and non-pixel regions arranged around the pixel region 12 . A vertical scanning circuit portion 110, a horizontal scanning circuit portion 111, a readout circuit 112, an output circuit 114, and a control pulse generating portion 115 are arranged in a region overlapping the non-pixel region in plan view.

FIG. 4 is an example of a block diagram including the equivalent circuits of FIGS. 2 and 3. In FIG. FIG. 4 shows a block diagram of a photoelectric conversion device with a general APD.

In FIG. 4, the photoelectric conversion element 102 having the APD 201 is provided on the sensor substrate 11, and the other members are provided on the circuit substrate 21. FIG.

The APD 201 generates charge pairs according to incident light through photoelectric conversion. A voltage VL (first voltage) is supplied to the anode of the APD 201 . Also, the cathode of the APD 201 is supplied with a voltage VH (second voltage) higher than the voltage VL supplied to the anode. A reverse bias voltage is supplied to the anode and cathode so that the APD 201 performs an avalanche multiplication operation. By supplying such a voltage, charges generated by the incident light undergo avalanche multiplication, generating an avalanche current.

When a reverse bias voltage is supplied, the Geiger mode operates with a potential difference between the anode and cathode that is greater than the breakdown voltage, and operates with a voltage difference near or below the breakdown voltage between the anode and cathode. It has a linear mode.

An APD operated in Geiger mode is called a SPAD (single photon avalanche diode). For example, the voltage VL (first voltage) is -30V, and the voltage VH (second voltage) is 3V. The APD 201 may operate in linear mode or in Geiger mode.

The quenching element 202 is connected to the power supply supplying the voltage VH and the APD 201 . The quench element 202 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication, suppresses the voltage supplied to the APD 201, and has a function of suppressing avalanche multiplication (quench operation). Also, the quench element 202 has a function of returning the voltage supplied to the APD 201 to the voltage VH by causing a current corresponding to the voltage drop due to the quench operation (recharge operation).

The signal processing section 103 has a waveform shaping section 210 , a counter 211 and a selection circuit 212 . In this specification, the signal processing section 103 may have any one of the waveform shaping section 210 , the counter 211 and the selection circuit 212 .

A waveform shaping unit 210 shapes the potential change of the cathode of the APD 201 obtained during photon detection, and outputs a pulse signal. For example, an inverter circuit is used as the waveform shaping section 210 . Although FIG. 4 shows an example in which one inverter is used as the waveform shaping section 210, a circuit in which a plurality of inverters are connected in series may be used, or another circuit having a waveform shaping effect may be used.

The counter 211 counts the number of pulse signals output from the waveform shaping section 210 and holds the count value. Further, when the control pulse pRES is supplied via the drive line 213, the signal held in the counter 211 is reset.

The selection circuit 212 is supplied with a control pulse pSEL from the vertical scanning circuit section 110 in FIG. 3 via the drive line 214 in FIG. Switch between connection and disconnection. The selection circuit 212 includes, for example, a buffer circuit for outputting a signal.

A switch such as a transistor may be provided between the quench element 202 and the APD 201 or between the photoelectric conversion element 102 and the signal processing unit 103 to switch the electrical connection. Similarly, the voltage VH or the voltage VL supplied to the photoelectric conversion element 102 may be electrically switched using a switch such as a transistor.

In this embodiment, the configuration using the counter 211 is shown. However, instead of the counter 211, a time-to-digital converter (hereinafter referred to as TDC) and a memory may be used as the photoelectric conversion device 100 for acquiring the pulse detection timing. At this time, the generation timing of the pulse signal output from the waveform shaping section 210 is converted into a digital signal by the TDC. A control pulse pREF (reference signal) is supplied to the TDC from the vertical scanning circuit unit 110 of FIG. 1 through a drive line for measuring the timing of the pulse signal. The TDC acquires a signal as a digital signal when the input timing of the signal output from each pixel via the waveform shaping section 210 is relative to the control pulse pREF.

FIG. 5 is a diagram schematically showing the relationship between the operation of the APD and the output signal.

FIG. 5(a) is a diagram of the APD 201, the quench element 202, and the waveform shaping section 210 extracted from FIG. Here, the input side of the waveform shaping section 210 is VC, and the output side is VO. FIG. 5(b) shows the voltage of VC in FIG. 5(a), and FIG. 5(c) shows the signal of VO in FIG. 5(a).

Between time t0 and time t1, a potential difference of VH-VL is applied to the APD 201 in FIG. 5(a). When a photon enters the APD 201 at time t1, avalanche multiplication occurs in the APD 201, an avalanche multiplication current flows through the quench element 202, and the voltage of VC drops. When the amount of voltage drop increases further and the potential difference applied to the APD 201 decreases, the avalanche multiplication of the APD 201 stops as at time t2, and the voltage level of VC does not drop beyond a certain value. After that, between time t2 and time t3, a current that compensates for the voltage drop from voltage VL flows through VC, and at time t3, VC stabilizes at the original potential level. At this time, a portion of the output waveform at VC that exceeds a certain threshold is waveform-shaped by waveform shaping section 210 and output as a signal at VO.

Note that the arrangement of the signal lines 113 and the arrangement of the reading circuit 112 and the output circuit 114 are not limited to those in FIG. For example, the signal line 113 may be arranged extending in the row direction, and the readout circuit 112 may be arranged beyond the extension of the signal line 113 .

(First embodiment)
FIG. 6 shows a block diagram and an equivalent circuit diagram of the pixel circuit. 6A and 6B are comparative examples, and FIG. 6C is the present embodiment. Also, FIGS. 7A and 7B are comparative examples. Further, FIGS. 8(a) and (b) are comparative examples, and FIG. 8(c) is the present embodiment.

(Comparative example 1: passive recharge circuit)
FIG. 6A is a block diagram and an equivalent circuit of a passive recharge pixel circuit as a comparative example. The gate of the transistor which is the quench element 202 connected to the cathode side of the APD 201 and the output of the logic circuit 221 (OR circuit) are at the same node. A signal ENB and a signal STOP can be input to the logic circuit 221 . The signal ENB is at low level during the exposure period T, and is at high level during the rest of the period.

When the signal STOP is at low level and the signal ENB is at low level, the transistor is turned on to enter a recharge state, and the APD 201 enters a standby state in which avalanche multiplication is possible after a predetermined period of time. In the case of other signal patterns, the quench element is turned off and does not enter the recharge state, so the APD 201 enters a non-standby state in which avalanche multiplication cannot be performed. The waveform shaping section 210 is composed of an inverter, and the counter 211 is provided with an 11-bit flip-flop. For example, when counter 211 reaches the maximum count value of 2047, signal STOP transitions from a low level to a high level, turning off the transistor of quench element 202 . As a result, the quench element 202 continues to be turned off even when the signal ENB becomes low level, and the non-standby state of the APD 201 is maintained.

The right diagram of FIG. 7(a) shows the voltage of VC and the signal of VO of FIG. 6(a).

In the case of low illuminance, since the interval between incident photons is long, it is possible to ensure a sufficient time for the VC to be recharged to a high voltage after the state in which the voltage of VC is low after the incident photons. The right diagram of FIG. 7A shows that three pulses can be counted corresponding to three photons. On the other hand, in the case of high illuminance, since the interval between incident photons is short, the voltage of VC is maintained in a low state and does not return to a high state, so it takes time to exceed the determination threshold from bottom to top. In the case of the right diagram of FIG. 7(a), only three pulses are counted although about 20 photons are incident. That is, photon count leakage occurs.

As a result, in the case of the passive recharge mode shown in FIG. 6(a), as shown in FIG. 7(a), a situation may occur in which the count value is the same regardless of whether the illuminance is low or high. Further, when photon incidence occurs more frequently, the voltage of VC is kept low, and the decision threshold is not exceeded from bottom to top. In this case, the voltage on VO will remain high and no signal will be generated. That is, in the case of the passive recharge mode shown in FIG. 6(a), since an appropriate count value cannot be obtained at high illuminance, the dynamic range is narrowed.

(Comparative example 2: clock recharge circuit)
FIG. 6B is a block diagram and an equivalent circuit diagram of a clock recharge pixel circuit as a comparative example. A signal CLKB and a signal STOP can be input to the input of the logic circuit 221 (OR circuit). The signal CLKB is a clock signal having Nc pulse signals during the exposure period T. FIG. In FIG. 6(b), the transistor is turned on and recharged when the signal STOP is at low level and the signal CLKB is at low level. For any other signal pattern, the quenching element is turned off. For example, when the signal CLKB is at high level, the quench element is off after being recharged, which means that it is in a standby state in which avalanche multiplication is possible. In other words, a circuit (transistor) is arranged between the APD 201 and the power supply (voltage VH), and the circuit operates in a first state in which the APD 201 and the power supply are electrically connected and in a state where the APD 201 and the power supply are electrically connected. control is performed to switch between a second state in which connection is not made to the In this case, the first state is the recharge state and the second state is the standby state.

Between the cathode of the APD 201 and the counter 211, there is provided a logic circuit 222 (an AND circuit in which the logic of the input on one side is inverted), and is configured so that the signals CLKB and VC can be input. The logic circuit 222 is a logic circuit to which the inverted output from VC is input, and VC transitions from high level to low level due to incident photons. In this case, when the signal CLKB is at high level, the output of the logic circuit 222 is at high level. After that, when the signal CLKB transitions from high level to low level, the output of the logic circuit 222 becomes low level, so an output signal is generated.

The right diagram of FIG. 7(b) shows the signal CLKB, the voltage of VC, and the signal of VO in FIG. 6(b).

First, in the case of low illuminance, when the signal CLKB is at high level (standby state) and a photon is incident, the voltage of VC drops and transitions to low level. Since the signal CLKB is high level, the output from VO becomes high level, and then when the signal CLKB becomes low level, the output from VO changes from high level to low level, and a signal is generated.

On the other hand, in the case of high illuminance, when the signal CLKB is high level (standby state), even if photon incidence occurs frequently, as long as the signal CLKB is high level, VO maintain a high level. Next, the transition of signal CLKB from high level to low level causes the output from VO to transition from high level to low level, generating one signal.

Thus, in the clock recharge circuit, the situation where the count value is the same regardless of whether the illuminance is low or high, or the situation where no signal is generated, which occurs in the case of passive recharge, is resolved. In other words, the clock recharge circuit has the advantage that the low illuminance count value is not greater than the high illuminance count value. The signal output from the logic circuit 222 is configured to be output from the counter 211 via the memory 240 to the outside. It should be noted that it is possible to appropriately set whether counting is performed at the falling edge of the output pulse of the logic circuit 222 or at the rising edge of the preceding pulse.

FIG. 8 shows the timing chart of the clock recharge circuit in more detail.

At time t0, the pulse of the signal RES (not shown in FIG. 6B) is turned on, the counter 211 is reset, and the count value COUNT of the counter 211 becomes "0".

At time t1, the signal EN (not shown in FIG. 6B) is turned on, and the exposure period T starts. When the signal CLKB transitions from high level to low level at time t1, since the signal STOP is at low level, the APD starts recharging, the voltage of VC gradually rises, and the standby mode is entered. At time t1, signal CLKB transitions from high level to low level, and VO changes from high level to low level.

At time t2, when photons are incident, avalanche multiplication begins and the potential of VC drops. Since the signal input from VC to the logic circuit 222 is at low level and the signal CLKB is also at high level, the output VO from the logic circuit 222 transitions from low level to high level. This transition causes COUNT to change from "0" to "1".

At time t3, a photon is incident, but the signal CLKB remains at high level, the potential of VC does not change, and VO remains at high level.

When the signal CLKB transitions from high level to low level at time t4, recharging starts and the potential of VC changes, as at time t1. Also, since the signal CLKB transitions from high level to low level, VO transitions from high level to low level. That is, the waveform of VO that rises at time t2 falls at time t4.

Similarly, at time t5, a photon is incident, avalanche multiplication begins and VO transitions from low level to high level, as described above.

Such an operation is sequentially repeated, and at time t6, the number of transitions of the signal CLKB from the high level to the low level reaches Nc. becomes Nc. Since the maximum count value of the counter is exceeded, the signal STOP also transitions from low level to high level. As a result, even if there is a change in VO after the time when the count value reaches Nc, it is not counted. In this example, it is assumed that at least one photon is incident each time in each standby state. In this case, the number of pulses of the signal CLKB is Nc during the exposure period T, so COUNT is also "Nc". Become.

At time t8, the signal EN input to the counter 211 changes from high level to low level. Here, since VO is at high level at time t8, the signal EN is changed from high level to low level so that this high level is not counted. It is configured so that it is not counted.

When the read signal WRT (not shown in FIG. 6B) becomes high level at time t9, the memory 223 stores the count value at that time. At time t10, when the read signal READ (not shown in FIG. 6B) becomes high level, the count value stored in the memory 223 is read from the selection circuit to the outside of the photoelectric conversion device. For example, the signal WRT is configured to store information in the memory 223 in all rows at once, and the signal READ is configured to read out to the outside in row order. This makes it possible to implement a global shutter that starts the exposure period for all rows at once.

(Description of this embodiment)
FIG. 8 shows time and count values in clock recharge driving. Here, "Nsat" is a number corresponding to the maximum count value of the counter. Details of "Nsat" will be described later.

In FIG. 8A, (ii) is a case of low illuminance, indicating that the counter number does not reach Nsat when the exposure time T is completed. On the other hand, (i) is the case of high illuminance, and the count value reaches Nsat before the exposure time T is completed. In this case, if the time to reach Nsat is recorded, it is possible to calculate the count value at the completion of the exposure period T using an extrapolation method. A count value calculated using this extrapolation method is also referred to as a calculated count value. As a result, it is possible to expand the dynamic range.

Also in FIG. 8B, (i) shows the case of high illuminance, and (ii) shows the case of low illuminance. In (i) of FIG. 8B, at a predetermined timing (T/m) before the end of the exposure period, it is determined whether or not a predetermined count value (Nsat/m) has been reached. For example, FIG. 8(b) shows a case where m=2 and whether or not a predetermined count value or more is determined at a timing of 1/2 of the exposure period T is determined. Then, when the count value reaches or exceeds a predetermined count value, the drive is shown in which the exposure is stopped. In this case, a value obtained by multiplying the measured count value by "m" is treated as the calculated count value when the exposure period T is completed. For example, m=2.

FIG. 8C is a diagram showing driving according to this embodiment. Specifically, it shows a case where a plurality of predetermined determination timings (predetermined checkpoints) are provided while adopting the driving shown in FIGS. 8(b) and (i). In (i), it is determined whether or not a predetermined count value (Nsat/m) is exceeded during the first exposure period (T/m ³ ). In (ii), it is determined whether or not the second exposure period (T/m ² ) is equal to or greater than a predetermined count value (Nsat/m). In (iii), it is determined whether or not the third exposure period (T/m) is equal to or greater than a predetermined count value (Nsat/m). In this case, the value obtained by multiplying each of the measured count values by “m ³ ”, “m ² ”, and “m”, for example, by 8, 4, and 2, is the completion of the exposure period T. Treated as a calculated count value for the hour.

Here, at each determination timing, it is determined whether or not the count value is equal to or greater than a predetermined count value, and if the count value is equal to or greater than the predetermined count value, exposure is stopped and the count value is output. Therefore, when exposure is stopped, the count value is Nsat/m or more and Nsat or less.

If the determination is made at a plurality of checkpoints in this way, the dynamic range can be further expanded.

FIG. 9(a) shows the relationship between the illuminance and the count value in the format described in FIGS. 8(b) and 8(c). FIG. 9(b) shows the relationship between illuminance and exposure time in this format. In the case of the area with the lowest illuminance, counting continues until the exposure period T is completed, so the exposure time is the longest as shown in FIG. 9(b). In the area where the illuminance is the lowest, the counter continues to count until it saturates, so as shown in FIG. 9A, the count is from 0 to Nsat. Next, in the case of an area where the illuminance is slightly high, it is determined that the count value (Nsat/m) is equal to or greater than the predetermined count value (Nsat/m) during the third exposure period (T/m), and the exposure is stopped. Since the exposure is stopped halfway, the exposure time is shortened as shown in FIG. 9(b). Also, as shown in FIG. 9A, the count value output in this case is Nsat/m or more and Nsat or less. The same applies when exposure is stopped during the second exposure period (T/m ² ) and when exposure is stopped during the first exposure period (T/m ³ ).

In case the exposure is stopped at the first exposure period (T/m ³ ), the maximum Nsat can be counted. In this case, the calculated count value is a value multiplied by ^m3 , so it is Nsat× ^m3 . That is, the count value that can be used for image formation can be expanded from Nsat to Nsat× ^m3 , and the dynamic range can be expanded. It is also possible to control whether to stop counting at different timings for each pixel. That is, the exposure time can be controlled for each pixel, and the dynamic range can be expanded for any pixel. Examples of circuits and drives for realizing the above-described method will be described below.

(Embodiment: Pixel Circuit Block Diagram)
FIG. 6C is a block diagram of a pixel circuit that controls the exposure time for each pixel. The APD 201, the quench element 202, the logic circuit 221, the logic circuit 222, and the counter 211 are the same as in FIG.

The signal CLKB is the same as in FIG. 6B in that a clock signal, which is Nc pulse signals, is input during the exposure period T (hereinafter also referred to as the maximum exposure period). However, as will be described later, the pulse period of the clock signal is shorter in FIG. 6(c) than in FIG. 6(b). 6C, a signal is input from the counter 211 to the exposure control circuit 230, and a signal is input from the exposure control circuit 230 to the memory 223 as well. The number of bits of the memory that the exposure control circuit has is smaller than the number of bits of the flip-flop that the counter 211 has.

FIG. 10 is a detailed block diagram of the pixel circuit of this embodiment. The description of the parts using the same reference numerals as in FIG. 6(c) is omitted. Since the counter 211 has, for example, an 11-bit flip-flop, the saturation value of the counter 211 is 2047 counts. Also, a signal EN is input to the counter 211, and the signal EN is a signal that defines the exposure period T. FIG. That is, when the signal EN transitions from low level to high level, the exposure period T starts, and when the signal EN transitions from high level to low level, the exposure period T ends and the counter 211 stops.

The counter 211 is configured to input a signal to the exposure control circuit 230 . The exposure control circuit 230 has multiple latches 231 . In FIG. 10, first to fourth latches 231 are provided from left to right. These four latches 231 and multiplexer 232 determine a predetermined count value (threshold) at predetermined determination timing.

The first latch 231 (leftmost) is configured to receive a signal Sn _/8 when the value is ⅛ of the maximum count value of the counter, that is, when m=8. there is As mentioned above, the maximum count value of the counter is 2047, but if this is 1/8, it will not be an integer. Therefore, for the sake of convenience, 2048, which is a number close to the maximum count value and is easy to handle in calculation, is assumed to correspond to the maximum count value, and is used as a reference for S _n/8 , S _n/4 , and S _n/2 . Here, the signal output when 2048/8=256 or more is assumed to be signal Sn _/8 .

Similarly, the second latch 231 (second from the left) receives a signal S _n/4 when the value is 1/4 of the maximum count value of the counter, that is, when m=4. is configured as follows. Here, for convenience, the signal output when 2048/4=512 or more is assumed to be signal _Sn/4 .

Similarly, the third latch 231 (second from the right) receives a signal Sn _{/ 2} is configured to be input. Here, for the sake of convenience, the signal output when 2048/2=1024 or more is assumed to be signal Sn _/2 .

Similarly, a signal _Sn is input to the fourth latch 231 (rightmost) when the maximum count value of the counter is reached. Here, the signal output when the maximum count value of the 11-bit count reaches 2047 is signal _Sn .

Here, when the signal _Sn is input to the fourth latch 231, the signal STOP is output through the logic circuit 234 (OR circuit) because the counter 211 has reached the maximum count value. The signal STOP is input to the counter 211 and the logic circuit 221 to stop the operation of the counter 211 and turn off the transistor which is the quench element 202 .

A control signal (not shown) is input to the multiplexer 232 , and the signals stored in the first latch 231 to the third latch 231 are selected and input to the memory 233 . For example, to increase the dynamic range, the signal of the first latch 231 is used. As described above, the count value that can be used for image formation can be expanded from Nsat to Nsat× ^m3 , so the larger m is, the wider the dynamic range can be. Therefore, the signal of the first latch 231 with m=8 is used. On the other hand, if the dynamic range is expanded too much, the reconstructed image may appear unnatural. For example, when there is a step in the image, the signal stored in the third latch 231 where m=2 is used. In this manner, one of the first latch 231 to the third latch 231 can be appropriately selected according to the application.

In FIG. 10, as the memory 233, first to third memories 233 are provided from top to bottom. As will be described later, at timings T0, T1, and T2, it is checked whether the count value is equal to or greater than the threshold. ” is recorded. Similarly, when the count is greater than or equal to the threshold value at timing T1, "1" is recorded in the second memory 233 . Similarly, when the count is equal to or greater than the threshold at timing T2, "1" is recorded in the third memory 233. FIG.

Time codes (time information) TC<0>, TC<1>, and TC<2> are output from the memory 233 and stored in the memory 223 . Since an 11-bit signal is output from the counter 211 and a 3-bit signal is output from the memory 233, the memory 223 has a total of 14 bits.

If any of TC<0>, TC<1>, and TC<2> is "1", it means that the count number is greater than or equal to the threshold (Nsat/m or greater), so the logic circuit 234 ( A signal STOP is output through an OR circuit).

When the read signal WRT is input to the memory 223, the 11-bit signal of the counter 211 at that time and the 3-bit signal of T0<2:0> are recorded in the memory 223. FIG. Further, when the read signal READ is input to the selection circuit 212, the signal stored in the memory 223 is read from the selection circuit 212 to the outside of the photoelectric conversion device. For example, the signal WRT is configured to store information in the memory 223 in all rows at once, and the signal READ is configured to read out to the outside in row order. This makes it possible to implement a global shutter that starts the exposure period for all rows at once.

(Embodiment: timing chart)
FIG. 11 is a timing chart of this embodiment.

At time t0, the pulse of signal RES is turned on. As shown in FIG. 10, signal RES is input to latch 231 and counter 211 . This allows the previous frame's information stored in latch 231 and counter 211 to be reset. As a result, the COUNT of the counter 211 becomes "0". Further, although not shown in FIG. 10, the signal RES may be input to the memory 223 and the memory 233 to reset the information of the previous frame.

At time t1, signal EN turns on and exposure period T begins. Here, the exposure period T is called the maximum exposure period. If few photons are incident, the counter will not saturate, so the counter will continue to count during this maximum exposure period. However, as will be described later, when the counter saturates or when the count value exceeds a predetermined threshold value within a predetermined exposure period, the STOP signal becomes high level, and the effective exposure period becomes the maximum exposure period. shorter than the period.

When the signal CLKB transitions from high level to low level at time t1, since the signal STOP is at low level, the APD starts recharging, the voltage of VC gradually rises, and the standby mode is entered. Although the signal CLKB transitions from high level to low level at time t1, VO changes from high level to low level because the photon has not yet entered.

At time t2, when photons are incident, avalanche multiplication begins and the potential of VC drops. Since the signal input from VC to the logic circuit 222 is at low level and the signal CLKB is at high level, the output VO from the logic circuit 222 transitions from low level to high level. This transition causes COUNT to change from "0" to "1". That is, in the case of this embodiment, counting is performed using the rise of VO.

When the signal CLKB transitions from high level to low level at time t3, recharging starts and the potential of VC changes, as at time t1. Also, since the signal CLKB transitions from high level to low level, VO transitions from high level to low level. That is, the waveform of VO that rises at time t2 falls at time t3.

The signal T0 transitions from low level to high level at time t4 when T/ ^m3 has elapsed from the start of the exposure period T. FIG. Here, m is an arbitrary number, for example, "8" when the signal of the leftmost latch 231 that receives the signal of Sn _/8 is selected using the multiplexer 232 . Therefore, the time t4 is the time when T/512 has passed since the start of the exposure period T.

Focusing on the signal CLKB, since the total number of pulses in the exposure period T is Nc, at time t4 after T/512 from the start of the exposure period T, the total number of pulses is Nc/ ^m3 .

Here, Nc/ ^m3 , which is the total number of pulses from the start of the exposure period T until T/ ^m3 has passed, is set to be equal to or greater than the maximum counter number of the counter. This is because if the number is smaller than the maximum number of counters, expansion of the dynamic range is restricted. If the dynamic range may be suppressed to some extent, Nc/ ^m3 may be made smaller than the maximum number of counters. For example, Nc/ ^m3 can be greater than or equal to 3/4 of the maximum number of counters.

In the present embodiment, it is determined whether or not the count value is equal to or greater than the threshold at the timing when three exposure periods that are included in the exposure period (maximum exposure period) and shorter than the exposure period are completed.

Here, if the exposure period is shorter, the first exposure period (first determination timing), the second exposure period (second determination timing), and the third exposure period (third determination timing) , the first exposure period is the period in which the exposure period is set to be the shortest. In this embodiment, the first exposure period corresponds to T/ ^m3 , the second exposure period corresponds to the exposure period T/ ^m2 , and the third exposure period corresponds to the exposure period T/m. . That is, m is the ratio of the length of the first exposure period to the length of the second exposure period, and the ratio of the length of the second exposure period to the length of the third exposure period.

As mentioned above, since m=8 here, the length of the second exposure period is eight times the length of the first exposure period. Also, the length of the third exposure period is eight times the length of the second exposure period. 8 times is a specific example, and these multiples may be 2 times or more, or 4 times or more.

Of these, in order to make the total number of pulses Nc/ ^m3 from the start of the exposure period T to the time T/ ^m3 elapsed be equal to or greater than the maximum count value of the counter, for example, when the maximum count value is 2047, Nc/ ^m3 is 2048. That is, Nc/ ^m3 is set to a value equal to or greater than the maximum count value of the counter. Therefore, Nc, which is the total number of pulses during the exposure period T, is about 1,000,000 when m=8. In normal clock recharge driving, which is a comparative example, Nc, which is the number of pulses in the exposure period, is 2048 when Nc is the number corresponding to the maximum count value of the counter. Therefore, compared with the comparative example, the number of pulses Nc in the present embodiment is very large and the pulse cycle of the clock signal is very short. This is the reason why the periods of the pulse signals are shown to be different in FIGS. 6(b) and (c).

At time t4, COUNT is "X1", which is a value smaller than the threshold "Nsat/m" (that is, X1<Nsat/m). Here, "Nsat" is 2048, for example. As described above, "Nsat" is a number corresponding to the maximum count value of the counter and is a number for ease of calculation. In this specification, "the number corresponding to the maximum count value of the counter" may also be treated as "the maximum count value of the counter".

In addition, as described above, m is the ratio of the exposure period lengths, for example m=8. Therefore, "Nsat/m" is 256, for example. In the example shown in FIG. 11, the signal Sn _/8 is not latched in the leftmost first latch 231 because the count value is less than the threshold. Therefore, the signal VC<0> serving as the time code remains at the low level, and "0" is input to the first memory 233 at the top.

Between time t4 and time t5, COUNT becomes a value equal to or greater than the threshold "Nsat/m", and the leftmost first latch 231 latches the signal Sn _/8 . The output from multiplexer 232 then transitions from low to high.

The signal T1 transitions from low level to high level at time t5 when T/ ^m2 has elapsed from the start of the exposure period T. For example, the time t5 is the time T/64 has elapsed from the time t1 at which the exposure period T starts.

Since the output from the multiplexer 232 is high level between time t4 and time t5, when the signal T1 is input at time t5, the signal TC<1> serving as the time code changes from low level to high level. Transition. As a result, “1” is input to the second memory 233 . Also, since TC<1> transitions from the low level to the high level, the signal STOP is applied to the counter 211 and the logic circuit 221 via the logic circuit 234 as shown in FIG. is turned off.

Focusing on the signal CLKB, since the number of pulses in the exposure period T is Nc, at time t5 after T/64 from the start of the exposure period T, the total number of pulses is Nc/ ^m2 .

FIG. 11 shows an example in which photons are incident after time t5 has elapsed. In this case, photon incidence causes avalanche multiplication and the potential of VC decreases, but since the transistor that is the quench element 202 is kept off, avalanche multiplication does not occur again. Therefore, VO maintains the high level after changing from the low level to the high level.

The signal T2 transitions from low level to high level at time t6 after T/m from the start of the exposure period T. For example, the time T/8 has passed since the start of the exposure period T. Focusing on the signal CLKB, since the number of pulses is Nc during the exposure period T, the total number of pulses at time t6, which is T/8 after the start of the exposure period T, is Nc/m.

Here, the output from the multiplexer 232 is maintained at high level, and when the signal T2 is input at time t6, the signal TC<2> serving as the time code transitions from low level to high level. As a result, “1” is input to the third memory 233 .

By the way, since VO is at high level due to the photon incidence after time t5, the signal EN is controlled to transition from high level to low level so as not to count this high level. As mentioned above, the signal EN also has the function of defining the start and end of the exposure period T (maximum exposure period). A signal other than the signal EN may be used to define the start and end of the maximum exposure period.

When the read signal WRT becomes high level at time t7, the signal is read from the memory 223 to the selection circuit 212 . At time t8, when the readout signal READ becomes high level, the signal is read from the selection circuit 212 to the outside of the photoelectric conversion device. For example, the signal WRT is configured to store information in the memory 223 in all rows at once, and the signal READ is configured to read out to the outside in row order. This makes it possible to implement a global shutter that starts the exposure period for all rows at once.

In the present embodiment, the threshold value for determination timing is set to "Nsat/m" with respect to m, which is the exposure period ratio. Here, the count threshold is generalized to be "Nsat/n" (n is a number of 2 or more). "Nsat/n" is, in other words, "the maximum count value of the counter/n". Here, the "maximum counter number of the counter" is a number including the maximum count value of the counter (eg 2147) and a number corresponding to the maximum count value of the counter (eg 2148).

In this case, if n is smaller than m, which is the exposure period ratio, for example, the count value in the first exposure period does not exceed the threshold value, and the count value reaches the saturation value in the second exposure period. can increase. As a result, gradation is lost under a specific amount of light. On the other hand, by setting n of "Nsat/n" to m or more, which is the exposure period ratio, pixels that reach the saturation value are reduced, and gradation is improved under a wide range of light intensity conditions from low luminance to high luminance. has the advantage of being able to ensure

FIG. 12 shows the effect of the photoelectric conversion device in each of the forms of FIGS. 6(a), 6(b) and 6(c). In other words, the concept explained in FIGS. 7(a), 7(b), and 9(a) is indicated by specific values. The horizontal axis of FIG. 12 is the number of incident photons, and the vertical axis is the median count value. Dotted line (a) corresponds to the form of passive recharge described with reference to FIG. 6(a). The unplotted solid line (b) corresponds to the form of clock recharge described with reference to FIG. 6(b). A solid line (c) with a plot corresponds to the form of controlling the exposure time for each pixel described with reference to FIG. 6(c).

The dotted line (a) in FIG. 12 saturates at 2047 counts, after which photons cannot be counted and the numerical value drops sharply. This is because, as described above, when the number of incident photons is very large, the voltage of VC is maintained at a low state, and the judgment threshold is not exceeded from bottom to top, and no signal is generated. be.

The unplotted solid line (b) in FIG. 12 is saturated at 2047 counts because the recharge clock in one frame is set to 2048 (Nc=2048) and the maximum count value of the counter is 2047. However, compared to FIG. 12(a), the number of incident photons before reaching saturation is large. When the number of incident photons is extremely large, a situation where no signal is generated does not occur, but since only the number of clock frequencies can be counted, the upper limit of the count value is determined by the clock frequency.

A solid line (c) with circle plots in FIG. 12 is a combination of clock recharge and exposure time control for each pixel, which is one of the embodiments of the present invention. The upper limit of the count value is 2047 counts, which is the same as (a) and (b). However, in (c), there are three times when the count value bottoms out. Since "Nsat/m" is set as the threshold as described above, the base count is 256 when m=8. In the case of this method, it is possible to continue counting for incident light more than the incident light corresponding to 2047 counts, and it is possible to widen the dynamic range. Specifically, the shortest exposure period during which the time code can be output is set to T/m ³ , and m=8, so the dynamic range is 512 times greater than in the case of FIG. 12(b). There is an advantage.

(Embodiment: Explanation of arithmetic processing)
FIG. 13 is a diagram for explaining a circuit for performing arithmetic processing and the arithmetic processing.

As shown in FIG. 13( a ), the signal output from the photoelectric conversion device 100 is configured to be input to the arithmetic circuit 300 . The arithmetic circuit 300 may be inside the photoelectric conversion device 100 . Signals may be input to the arithmetic circuit 300 via a recording medium regardless of whether they are wired or wireless.

FIG. 13(b) shows the operation flow in the arithmetic circuit 300. As shown in FIG.

In S<b>301 , 14-bit Raw data is read from the memory 223 .

In S302, 11-bit light count value information and 3-bit time code information are separated from 14-bit Raw data.

In S303, logical shift (bit shift) is performed using the count value information and the time code information.

FIG. 14 specifically shows the logical shift processing. At S310, a logical shift is initiated. At S312, it is determined whether or not the time code TC<0> is 1. If TC<0> is 1, in S314, the light count value for 11 bits is multiplied by ^m3 . If the time code TC<0> is not 1 in S312, it is determined whether or not the time code TC<1> is 1 in S316. If TC<1> is 1, in S318, the light count value for 11 bits is multiplied by ^m2 . If the time code TC<1> is not 1 in S316, it is determined whether the time code TC<2> is 1 in S320. If TC<2> is 1, the 11-bit light count value is multiplied by m in S322. If the time code TC<2> is not 1 in S320, the counter is not saturated, so logical shift processing is unnecessary, and the logical shift operation ends in S324. Thus, in the logical shift process, the multiplication factor of the count value of the counter is changed based on the time information.

Returning to FIG. 13B, nonlinear correction is performed in S304. For example, referring to the plot in (c) of FIG. 12, there is a region where the slope is not constant around the number of incident photons of 1×10 ⁶ to 1×10 ⁷ . Therefore, in S304, a step of correcting the region where the inclination is not constant is performed.

Specifically, where X is the count value after correction and Y is the count value before correction, Y=Nc×(1−exp (−X/Nc), where Nc is as described above. This is the number of pulses entered during the exposure period T. In S305, demosaic processing (interpolation processing) is performed, and in S306, since the light transmittance and reflectance of the filter are different for each of RGB, an appropriate signal amplification factor is determined. Tuning is performed.In S307, HDR tone mapping is performed, and in S308, a color image that has undergone such arithmetic processing is output.Although FIG. In this case, the processes such as S305 and S306 can be omitted as appropriate.

(Modification)
In the above-described first embodiment, it is determined whether or not the count value reaches the threshold value at a predetermined checkpoint, and the threshold value is set to a value smaller than the maximum count value (saturation value) of the counter. I explained mainly. However, it is also possible to employ a method in which the maximum count value of the counter is set as a threshold and counting is performed up to the threshold. In this case, information on the time when the threshold value is reached may be stored in a memory, and a calculated count value may be obtained from the time information using a method such as an extrapolation method.

Also in this method, the number of pulses of the clock signal input during the exposure period T is set to be at least twice the maximum count value of the counter. For example, assuming that the exposure period T includes the first exposure period and the second exposure period, the number of pulses of the clock signal in the first exposure period is set to be equal to or greater than the maximum count value of the counter. In addition, if it is assumed that the first exposure period is shorter than the second exposure period, the number of pulses of the clock signal within the first exposure period may be equal to or greater than the maximum count value of the counter. Furthermore, as in the above embodiment, assuming that the number of pulses equal to or greater than the maximum count value of the counter is required until passing T/m ³ , when m is 2, the number of pulses to be put in the exposure period T is Eight times or more the maximum count value of the counter. Also, when m is 8, the number of pulses put in the exposure period T is 512 times or more the maximum count value of the counter.

(Second embodiment)
This embodiment is an embodiment for explaining variation of the frequency of the pulse signal using FIG. FIGS. 15A to 15E are timing charts of the signal CLKB input to the logic circuit 221 described in FIG. 6C.

(first form)
FIG. 15(a) shows, as a first mode, a mode in which the exposure period is operated with a signal CLKB having a constant frequency from the beginning to the end of the exposure period. At this time, as in the first embodiment, if the shortest exposure period for counting is T/m ³ , the number of pulses of the signal CLKB entering the period from the start of the exposure period to T/m ³ is For example, Nsat. In this case, the number of pulses of the signal CLKB in the exposure period T is Nsat× ^m3 . That is, by inserting Nsat recharges within the shortest exposure period, it is possible to expand the dynamic range by utilizing the upper count limit. Also, the shortest exposure period for determining the count does not need to include exactly Nsat CLKB pulses, and may include Nsat or more pulses. That is, any clock signal having the number of pulses equal to or greater than the maximum count value of the counter may be used.

(Second form)
FIG. 15(b) is a timing chart of the signal CLKB in the second mode. The difference from the first mode is that the frequency of the signal CLKB changes within the exposure period. In the second form, the frequency of the clock signal is switched at determination points such as T/m ³ , T/m ² , and T/m at which the frequency of the signal CLKB determines the count value. That is, two or more frequencies of the clock signal are provided within the exposure period.

For example, the frequency of signal CLKB is frequency f1 from the start of the exposure period to T/ ^m3 , frequency f2 from T/m3 to T/ ^m2 , frequency f3 from T/ ^m2 to T/m, and T/m to T is frequency f4. In this case, the relationship is f1>f2>f3>f4. At this time, if m=8, the frequency may be decreased according to the ratio of the exposure period, such as f1=f2×8=f3×64=f4×512. By applying this ratio, the minimum frequency is set when photons enter at a constant cycle until each count determination timing. The minimum frequency is a frequency calculated by dividing the assumed maximum number of incident photons Nsat by the period up to each count determination timing such as T/m ³ , T/m ² , T/m. If the frequency is equal to or higher than this minimum frequency, the dynamic range can be fully utilized up to the count upper limit value Nsat at each count determination timing. For example, if we assume a maximum Nsat number of incident photons up to T/m ³ , the minimum frequency f1=Nsat/(T/m ³ ). In the period from T/m ³ to T/m ² , the maximum photon incidence frequency is the case where the count Nsat is reached at ^the next T/m ² , since the threshold is not exceeded at T/m 3. . That is, the period from T/ ^m3 to T/ ^m2 should be f2=Nsat/(T/ ^m2 ). Similarly calculated, f3=Nsat/(T/m), f4=Nsat/T. In this way, by reducing the frequency of the signal CLKB to a required number within the exposure period, power consumption by the signal CLKB can be reduced while ensuring the effect of expanding the dynamic range.

A frequency dividing circuit may be provided to change the frequency of the clock signal. The frequency dividing circuit can be provided in the vertical scanning circuit section, the pixel circuit section, and the control pulse generating section. When provided in the pixel circuit portion, it is possible to apply a clock signal with a different frequency to each pixel.

Note that the frequency of the clock signal from the start of the exposure period to T/ ^m3 (first exposure period) may not be constant. Similarly, the frequency of the clock signal between T/m3 and T/ ^m2 (second exposure period) may not be constant. In this case, the frequency can be considered as an average frequency. For example, the first frequency, which is the average frequency for the first exposure period, is greater than the second frequency, which is the average frequency for the second exposure period.

(Third form)
FIG. 15(c) is a diagram showing a timing chart of signal CLKB in the third mode.

The difference from the second mode is that the frequency is gradually lowered from the start to the end of the exposure period T instead of switching the frequency before and after the timing for determining the count value. In this way, by gradually adjusting the frequency instead of switching the frequency before and after the determination timing, it is possible to reduce the step in the number of counts caused by the switching of the frequency. In addition, as a method for changing the frequency, it is possible to expand options for circuit configuration such as using a frequency modulation circuit in addition to the frequency dividing circuit.

In this embodiment, it can be expressed that the average frequency of the clock signal gradually decreases from the start to the end of the exposure period T. FIG. It is also possible to express that the average frequency for a predetermined period before the end of the exposure period T is lower than the average frequency for a predetermined period after the start of the exposure period T.

(Fourth form)
FIG. 15(d) is a diagram showing a timing chart of signal CLKB in the fourth mode. In the second mode shown in FIG. 15(b), the frequency of the signal CLKB is always lowered toward the latter half of the exposure period ^. The frequency increases once between T/m. For example, from the start of the exposure period to the determination timing of T/ ^m2 , not many photons arrived and the illumination was low, but from the determination timing of T/ ^m2 to the time T at the end of the exposure period, photons did not arrive. Assume a case where a large number of people arrive and a high illuminance condition occurs. In this case, if the drive shown in FIG. 15B is adopted, a count loss will occur when the illuminance condition becomes high in the second half. According to the drive shown in FIG. 15(d), even under such conditions, the photon count loss in the latter half can be reduced. Note that the rate at which the frequency of the signal CLKB is increased and the timing at which it is increased may be determined arbitrarily, as long as the number of pulses is equal to or greater than the number of pulses necessary for expanding the dynamic range.

(Fifth form)
FIG. 15(e) is a diagram showing a timing chart of signal CLKB in the fifth mode. The difference from the second mode shown in FIG. 15(b) is that the intervals between the pulses forming one group are equal, but the intervals between the pulse groups gradually increase. However, FIG. 15B and FIG. 15E have the same number of pulses in each exposure period (period defined by determination timings). According to the form of FIG. 15(e), when the signals CLKB approach each other in each exposure period, it becomes possible to count even when the timing of photon incidence becomes closer. In other words, the sensitivity can be maintained up to high illuminance, so the dynamic range can be expanded.

In this embodiment, the number of pulses of the clock signal per unit time in the first exposure period can be expressed as being greater than the number of pulses of the clock signal per unit time in the second exposure period. be. That is, the pulse distribution ratio of the clock signal in the first exposure period is denser than the pulse distribution ratio of the clock signal in the second exposure period.

(Third embodiment)
16 and 17, this embodiment is an embodiment for explaining a further variation of the method of inputting the pulse signal. 16 and 17 are timing charts of the signal CLKB input to the logic circuit 221 described in FIG. 6(c).

In FIG. 16, CLKB<0>, CLKB<1>, . >, the signal CLKB is input to each. In this way, even in a plurality of rows, it is possible to control the frequency for each exposure period for each vertical scanning address in global shutter driving. At this time, the method of controlling the frequency for each vertical address is arbitrary. For example, a configuration is conceivable in which a plurality of frequencies are input from the control pulse generator 115 to the vertical scanning circuit 110 and the frequencies are selected for each address in the vertical scanning circuit 110 . Alternatively, a single frequency may be input to the vertical scanning circuit 110 and divided in the middle of the exposure period for each vertical scanning address. Alternatively, a constant frequency may be input from the vertical scanning circuit 110 and divided inside the signal processing unit 103 of the pixel.

FIG. 17 shows an example of another drive, which is different from FIG. 16 in that rolling shutter drive is used instead of global shutter drive. By performing frequency control for each address in this way, it is possible to cope with either type of driving.

(Fourth embodiment)
A photoelectric conversion system according to this embodiment will be described with reference to FIG. FIG. 18 is a block diagram showing a schematic configuration of a photoelectric conversion system according to this embodiment.

The photoelectric conversion devices described in the above embodiments are applicable to various photoelectric conversion systems. Examples of applicable photoelectric conversion systems include digital still cameras, digital camcorders, surveillance cameras, copiers, facsimiles, mobile phones, vehicle-mounted cameras, and observation satellites. A camera module including an optical system such as a lens and an imaging device is also included in the photoelectric conversion system. FIG. 18 illustrates a block diagram of a digital still camera as an example of these.

The photoelectric conversion system illustrated in FIG. 18 includes an imaging device 1004 that is an example of a photoelectric conversion device, and a lens 1002 that forms an optical image of a subject on the imaging device 1004 . The near-electric conversion system further has an aperture 1003 for varying the amount of light passing through the lens 1002 and a barrier 1001 for protecting the lens 1002 . A lens 1002 and a diaphragm 1003 are an optical system for condensing light onto an imaging device 1004 . The imaging device 1004 is a photoelectric conversion device according to any of the above embodiments, and converts an optical image formed by the lens 1002 into an electrical signal.

The photoelectric conversion system also includes a signal processing unit 1007 that is an image generation unit that processes an output signal output from the imaging device 1004 to generate an image. A signal processing unit 1007 performs an operation of performing various corrections and compressions as necessary and outputting image data. The signal processing unit 1007 may be formed in the semiconductor layer in which the imaging device 1004 is provided, or may be formed in a semiconductor layer separate from the imaging device 1004 . Also, the imaging device 1004 and the signal processing unit 1007 may be formed in the same semiconductor layer.

The photoelectric conversion system further includes a memory unit 1010 for temporarily storing image data, and an external interface unit (external I/F unit) 1013 for communicating with an external computer or the like. Further, the photoelectric conversion system includes a recording medium 1012 such as a semiconductor memory for recording or reading image data, and a recording medium control interface section (recording medium control I/F section) 1011 for recording or reading from the recording medium 1012. have Note that the recording medium 1012 may be built in the photoelectric conversion system or may be detachable.

The photoelectric conversion system further includes an overall control/calculation unit 1009 for controlling various calculations and the entire digital still camera, and a timing generation unit 1008 for outputting various timing signals to the imaging device 1004 and signal processing unit 1007 . Here, the timing signal and the like may be input from the outside, and the photoelectric conversion system may have at least the imaging device 1004 and the signal processing unit 1007 that processes the output signal output from the imaging device 1004 .

The imaging device 1004 outputs the imaging signal to the signal processing unit 1007 . A signal processing unit 1007 performs predetermined signal processing on the imaging signal output from the imaging device 1004 and outputs image data. A signal processing unit 1007 generates an image using the imaging signal.

As described above, according to the present embodiment, a photoelectric conversion system to which the photoelectric conversion device (imaging device) of any one of the above embodiments is applied can be realized.

(Fifth embodiment)
A photoelectric conversion system and a moving object according to this embodiment will be described with reference to FIG. 19 . FIG. 19 is a diagram showing the configuration of the photoelectric conversion system and moving object of this embodiment.

FIG. 19(a) shows an example of a photoelectric conversion system for a vehicle-mounted camera. The photoelectric conversion system 2300 has an imaging device 2310 . The imaging device 2310 is the photoelectric conversion device described in any of the above embodiments. The photoelectric conversion system 2300 has an image processing unit 2312 that performs image processing on a plurality of image data acquired by the imaging device 2310 . The photoelectric conversion system 2300 also has a parallax acquisition unit 2314 that calculates parallax (phase difference of parallax images) from a plurality of image data acquired by the photoelectric conversion system 2300 . Furthermore, the photoelectric conversion system 2300 includes a distance acquisition unit 2316 that calculates the distance to the object based on the calculated parallax, and a collision determination unit that determines whether there is a possibility of collision based on the calculated distance. 2318 and . Here, the parallax acquisition unit 2314 and the distance acquisition unit 2316 are examples of distance information acquisition means for acquiring distance information to the object. That is, the distance information is information related to parallax, defocus amount, distance to the object, and the like. The collision determination unit 2318 may use any of these distance information to determine the possibility of collision. The distance information acquisition means may be implemented by specially designed hardware, or may be implemented by a software module. Moreover, it may be implemented by FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit), or the like, or by a combination thereof.

The photoelectric conversion system 2300 is connected to a vehicle information acquisition device 2320, and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. The photoelectric conversion system 2300 is also connected to a control ECU 2330 which is a control device (control unit) that outputs a control signal for generating a braking force to the vehicle based on the determination result of the collision determination unit 2318 . The photoelectric conversion system 2300 is also connected to an alarm device 2340 that issues an alarm to the driver based on the determination result of the collision determination section 2318 . For example, if the collision determination unit 2318 determines that there is a high probability of collision, the control ECU 2330 performs vehicle control to avoid collisions and reduce damage by braking, releasing the accelerator, or suppressing engine output. The alarm device 2340 warns the user by sounding an alarm such as sound, displaying alarm information on a screen of a car navigation system, or vibrating a seat belt or steering wheel.

In this embodiment, the photoelectric conversion system 2300 captures an image of the surroundings of the vehicle, for example, the front or rear. FIG. 19B shows a photoelectric conversion system for capturing an image in front of the vehicle (imaging range 2350). A vehicle information acquisition device 2320 sends an instruction to the photoelectric conversion system 2300 or imaging device 2310 . With such a configuration, the accuracy of distance measurement can be further improved.

In the above, an example of controlling so as not to collide with another vehicle was explained, but it can also be applied to control to automatically drive following another vehicle or control to automatically drive so as not to stray from the lane. . Furthermore, the photoelectric conversion system can be applied not only to vehicles such as own vehicles but also to moving bodies (moving devices) such as ships, aircraft, and industrial robots. In addition, the present invention can be applied not only to mobile objects but also to devices that widely use object recognition, such as intelligent transportation systems (ITS).

(Sixth embodiment)
A photoelectric conversion system according to this embodiment will be described with reference to FIG. FIG. 20 is a block diagram showing a configuration example of a distance image sensor, which is a photoelectric conversion system.

As shown in FIG. 20, the distance image sensor 401 comprises an optical system 402, a photoelectric conversion device 403, an image processing circuit 404, a monitor 405, and a memory 406. The distance image sensor 401 receives the light (modulated light or pulsed light) projected from the light source device 411 toward the subject and reflected by the surface of the subject, thereby producing a distance image corresponding to the distance to the subject. can be obtained.

The optical system 402 includes one or more lenses, guides image light (incident light) from a subject to the photoelectric conversion device 403, and forms an image on the light receiving surface (sensor section) of the photoelectric conversion device 403. Let

As the photoelectric conversion device 403 , the photoelectric conversion device described in the above embodiment is applied, and the distance signal indicating the distance obtained from the received light signal output from the photoelectric conversion device 403 is supplied to the image processing circuit 404 .

The image processing circuit 404 performs image processing to construct a distance image based on the distance signal supplied from the photoelectric conversion device 403 . A distance image (image data) obtained by the image processing is supplied to the monitor 405 to be displayed, or supplied to the memory 406 to be stored (recorded).

In the range image sensor 401 configured in this way, by applying the above-described photoelectric conversion device, it is possible to obtain, for example, a more accurate range image as the characteristics of the pixels are improved.

(Seventh embodiment)
A photoelectric conversion system of this embodiment will be described with reference to FIG. FIG. 21 is a diagram showing an example of a schematic configuration of an endoscopic surgery system, which is the photoelectric conversion system of this embodiment.

FIG. 21 shows an operator (doctor) 1131 performing an operation on a patient 1132 on a patient bed 1133 using an endoscopic surgery system 1103 . As illustrated, the endoscopic surgery system 1103 is composed of an endoscope 1100, a surgical tool 1110, and a cart 1134 on which various devices for endoscopic surgery are mounted.

An endoscope 1100 is composed of a lens barrel 1101 whose distal end is inserted into a body cavity of a patient 1132 and a camera head 1102 connected to the proximal end of the lens barrel 1101 . The illustrated example shows an endoscope 1100 configured as a so-called rigid scope having a rigid barrel 1101, but the endoscope 1100 may be configured as a so-called flexible scope having a flexible barrel. good.

The tip of the lens barrel 1101 is provided with an opening into which an objective lens is fitted. A light source device 1203 is connected to the endoscope 1100, and light generated by the light source device 1203 is guided to the tip of the lens barrel 1101 by a light guide extending inside the lens barrel 1101, whereupon the objective lens through the body cavity of the patient 1132 toward the object to be observed. Note that the endoscope 1100 may be a straight scope, a perspective scope, or a side scope.

An optical system and a photoelectric conversion device are provided inside the camera head 1102, and reflected light (observation light) from an observation target is collected by the optical system on the photoelectric conversion device. The photoelectric conversion device photoelectrically converts the observation light to generate an electrical signal corresponding to the observation light, that is, an image signal corresponding to the observation image. As the photoelectric conversion device, the photoelectric conversion devices described in the above embodiments can be used. The image signal is transmitted to a camera control unit (CCU: Camera Control Unit) 1135 as RAW data.

The CCU 1135 includes a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and the like, and controls the operations of the endoscope 1100 and the display device 1136 in an integrated manner. Further, the CCU 1135 receives an image signal from the camera head 1102 and performs various image processing such as development processing (demosaicing) for displaying an image based on the image signal.

The display device 1136 displays an image based on an image signal subjected to image processing by the CCU 1135 under the control of the CCU 1135 .

The light source device 1203 is composed of, for example, a light source such as an LED (Light Emitting Diode), and supplies the endoscope 1100 with irradiation light for imaging a surgical site or the like.

Input device 1137 is an input interface for endoscopic surgery system 1103 . The user can input various information and instructions to the endoscopic surgery system 1103 via the input device 1137 .

The treatment instrument control device 1138 controls driving of the energy treatment instrument 1112 for tissue cauterization, incision, blood vessel sealing, or the like.

The light source device 1203 that supplies irradiation light to the endoscope 1100 for imaging the surgical site can be composed of, for example, a white light source composed of an LED, a laser light source, or a combination thereof. When a white light source is configured by a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high accuracy. It can be carried out. In this case, the observation target is irradiated with laser light from each of the RGB laser light sources in a time-sharing manner, and by controlling the drive of the imaging device of the camera head 1102 in synchronization with the irradiation timing, each of the RGB can be handled. It is also possible to pick up images by time division. According to this method, a color image can be obtained without providing a color filter in the imaging device.

Further, the driving of the light source device 1203 may be controlled so as to change the intensity of the output light every predetermined time. By controlling the driving of the imaging device of the camera head 1102 in synchronism with the timing of the change in the intensity of the light to acquire images in a time-division manner and synthesizing the images, a high dynamic A range of images can be generated.

Also, the light source device 1203 may be configured to be capable of supplying light in a predetermined wavelength band corresponding to special light observation. Special light observation, for example, utilizes the wavelength dependence of light absorption in body tissues. Specifically, a predetermined tissue such as a blood vessel on the surface of the mucous membrane is imaged with high contrast by irradiating light with a narrower band than the irradiation light (that is, white light) used during normal observation. Alternatively, in special light observation, fluorescence observation may be performed in which an image is obtained from fluorescence generated by irradiation with excitation light. In fluorescence observation, body tissue is irradiated with excitation light and fluorescence from the body tissue is observed, or a reagent such as indocyanine green (ICG) is locally injected into the body tissue and the fluorescence wavelength of the reagent is observed in the body tissue. It is possible to obtain a fluorescent image by irradiating excitation light corresponding to . The light source device 1203 can be configured to supply narrowband light and/or excitation light corresponding to such special light observation.

(Eighth embodiment)
The photoelectric conversion system of this embodiment will be described with reference to FIG. 22 . FIG. 22A is a diagram showing an example of the configuration of spectacles 1600 (smart glasses), which is a photoelectric conversion system. Glasses 1600 have a photoelectric conversion device 1602 . The photoelectric conversion device 1602 is the photoelectric conversion device described in the above twelfth embodiment. A display device including a light emitting device such as an OLED or an LED may be provided on the rear surface side of the lens 1601 . One or more photoelectric conversion devices 1602 may be provided. Further, a plurality of types of photoelectric conversion devices may be used in combination. The arrangement position of the photoelectric conversion device 1602 is not limited to that shown in FIG.

Glasses 1600 further comprise a controller 1603 . The control device 1603 functions as a power source that supplies power to the photoelectric conversion device 1602 and the display device. Further, the control device 1603 controls operations of the photoelectric conversion device 1602 and the display device. An optical system for condensing light onto the photoelectric conversion device 1602 is formed in the lens 1601 .

FIG. 22(b) illustrates glasses 1610 (smart glasses) according to one application. The glasses 1610 have a control device 1612, and the control device 1612 is equipped with a photoelectric conversion device corresponding to the photoelectric conversion device 1602 and a display device. A photoelectric conversion device in the control device 1612 and an optical system for projecting light emitted from the display device are formed in the lens 1611 , and an image is projected onto the lens 1611 . The control device 1612 functions as a power source that supplies power to the photoelectric conversion device and the display device, and controls the operation of the photoelectric conversion device and the display device. The control device may have a line-of-sight detection unit that detects the line of sight of the wearer. Infrared rays may be used for line-of-sight detection. The infrared light emitting section emits infrared light to the eyeballs of the user who is gazing at the display image. A captured image of the eyeball is obtained by detecting reflected light of the emitted infrared light from the eyeball by an imaging unit having a light receiving element. By having a reduction means for reducing light from the infrared light emitting section to the display section in plan view, deterioration in image quality is reduced.

The line of sight of the user with respect to the display image is detected from the captured image of the eye obtained by imaging the infrared light. Any known method can be applied to line-of-sight detection using captured images of eyeballs. As an example, it is possible to use a line-of-sight detection method based on a Purkinje image obtained by reflection of irradiation light on the cornea.

More specifically, line-of-sight detection processing based on the pupillary corneal reflection method is performed. The user's line of sight is detected by calculating a line-of-sight vector representing the orientation (rotational angle) of the eyeball based on the pupil image and the Purkinje image included in the captured image of the eyeball using the pupillary corneal reflection method. be.

The display device of the present embodiment may have a photoelectric conversion device having a light receiving element, and may control a display image of the display device based on the user's line-of-sight information from the photoelectric conversion device.

Specifically, the display device determines a first visual field area that the user gazes at and a second visual field area other than the first visual field area, based on the line-of-sight information. The first viewing area and the second viewing area may be determined by the control device of the display device, or may be determined by an external control device. In the display area of the display device, the display resolution of the first viewing area may be controlled to be higher than the display resolution of the second viewing area. That is, the resolution of the second viewing area may be lower than that of the first viewing area.

Further, the display area has a first display area and a second display area different from the first display area. may be determined. The first viewing area and the second viewing area may be determined by the control device of the display device, or may be determined by an external control device. The resolution of areas with high priority may be controlled to be higher than the resolution of areas other than areas with high priority. That is, the resolution of areas with relatively low priority may be lowered.

AI may be used to determine the first field of view area and the areas with high priority. The AI is a model configured to estimate the angle of the line of sight from the eyeball image and the distance to the object ahead of the line of sight, using the image of the eyeball and the direction in which the eyeball of the image was actually viewed as training data. It's okay. The AI program may be owned by the display device, the photoelectric conversion device, or the external device. If the external device has it, it is communicated to the display device via communication.

In the case of performing display control based on visual recognition detection, it can be preferably applied to smart glasses that further have a photoelectric conversion device that captures an image of the outside. Smart glasses can display captured external information in real time.

The embodiments described above can be modified as appropriate without departing from the technical concept. Further, examples in which a part of the configuration of any one embodiment is added to another embodiment, and examples in which a part of the configuration of another embodiment is replaced are also included in the embodiments of the present invention.

Further, the disclosure of this embodiment includes the following configurations and methods.

(Configuration 1)
a photodiode for avalanche multiplication;
a circuit disposed between the photodiode and a power supply for switching between a first state in which the photodiode is electrically connected to the power supply and a second state in which the photodiode is not electrically connected to the power supply; ,
a counter that counts the output signal from the photodiode;
a memory in which time information indicating that the count value of the counter has reached a threshold within a predetermined exposure period included in the exposure period and shorter than the exposure period is written;
A photoelectric conversion device, wherein a clock signal can be input to the circuit during the exposure period.

(Configuration 2)
The photoelectric conversion device according to Structure 1, wherein the predetermined exposure period has a first exposure period and a second exposure period.

(Composition 3)
Configuration 2, wherein the first exposure period is shorter than the second exposure period, and the number of pulses of the clock signal during the first exposure period is equal to or greater than the maximum count value of the counter. 3. The photoelectric conversion device according to .

(Composition 4)
The photoelectric conversion device according to any one of Structures 1 to 3, wherein the threshold is a maximum count value of the counter.

(Composition 5)
4. The photoelectric conversion device according to any one of Structures 1 to 3, wherein the threshold value is smaller than a maximum count value of the counter.

(Composition 6)
4. The photoelectric conversion device according to any one of Structures 1 to 3, wherein the threshold is a maximum count value/n of the counter.

(Composition 7)
The predetermined exposure period has a first exposure period and a second exposure period that is a longer exposure period than the first exposure period,
The photoelectric conversion device according to structure 6, wherein the n is equal to or greater than the length of the second exposure period/the length of the first exposure period.

(Composition 8)
3. The photoelectric conversion device according to claim 2, wherein the length of said second exposure period is at least twice the length of said first exposure period.

(Composition 9)
The predetermined exposure period has a third exposure period that is longer than the second exposure period, and the length of the third exposure period is the length of the second exposure period. 9. The photoelectric conversion device according to configuration 2 or 8, characterized in that it is two times or more.

(Configuration 10)
The predetermined exposure period has a third exposure period that is longer than the second exposure period, and the length of the third exposure period is the length of the second exposure period. four times more,
The photoelectric conversion device according to any one of Structures 2, 8, and 9, wherein the length of the second exposure period is four times or more the length of the first exposure period.

(Composition 11)
11. The control circuit according to any one of claims 1 to 10, further comprising a control circuit that changes a potential supplied to a gate of a transistor included in the circuit when the counter reaches the threshold within the predetermined exposure period. 3. The photoelectric conversion device according to .

(Composition 12)
12. The photoelectric conversion device according to any one of configurations 1 to 11, further comprising a control circuit that stops the counter when the counter reaches the threshold within the predetermined exposure period.

(Composition 13)
13. The photoelectric conversion device according to any one of configurations 1 to 12, wherein the number of bits of the memory is smaller than the number of bits of the counter.

(Composition 14)
14. The photoelectric conversion according to any one of configurations 1 to 13, further comprising a logic circuit between the photodiode and the counter, wherein the clock signal is input to the logic circuit during the exposure period. Device.

(Composition 15)
15. The photoelectric conversion device according to any one of Structures 1 to 14, wherein the clock signal has two or more frequencies within the exposure period.

(Composition 16)
A first frequency that is an average frequency of the clock signal within the first exposure period and a second frequency that is an average frequency of the clock signal during the second exposure period are different. The photoelectric conversion device according to Structure 2.

(Composition 17)
17. The photoelectric conversion device according to configuration 16, wherein the first frequency is higher than the second frequency.

(Composition 18)
The optoelectronic device according to configuration 2, wherein the number of pulses of the clock signal per unit time during the first exposure period is greater than the number of pulses of the clock signal per unit time during the second exposure period. conversion device.

(Composition 19)
An arithmetic circuit for calculating a signal output from the photoelectric conversion device according to any one of claims 1 to 18,
An arithmetic circuit, wherein the arithmetic circuit changes a multiplication factor of the count value of the counter based on the time information.

(Configuration 20)
The photoelectric conversion device according to Structure 1, wherein the photoelectric conversion device according to Structure 1 has the arithmetic circuit according to Structure 19.

(Composition 21)
A photoelectric conversion system comprising the photoelectric conversion device according to claim 1 and the arithmetic circuit according to configuration 19.

(Composition 22)
a photoelectric conversion device according to configuration 1;
and a signal processing unit that generates an image using a signal output from the photoelectric conversion device.

(Composition 23)
A moving body comprising the photoelectric conversion device according to Configuration 1,
A moving object, comprising: a control unit that controls movement of the moving object using a signal output from the photoelectric conversion device.

201 avalanche photodiode 202 quench element 210 waveform shaping unit 211 counter circuit 212 selection circuit 221 logic circuit 222 logic circuit 223 memory 230 exposure control circuit 231 latch 232 multiplexer 233 memory 234 logic circuit

Claims (23)

a photodiode for avalanche multiplication;
a circuit disposed between the photodiode and a power supply for switching between a first state in which the photodiode is electrically connected to the power supply and a second state in which the photodiode is not electrically connected to the power supply; ,
a counter that counts the output signal from the photodiode;
a memory in which time information indicating that the count value of the counter has reached a threshold within a predetermined exposure period included in the exposure period and shorter than the exposure period is written;
A photoelectric conversion device, wherein a clock signal can be input to the circuit during the exposure period. 2. The photoelectric conversion device according to claim 1, wherein the predetermined exposure period has a first exposure period and a second exposure period. 3. The first exposure period is shorter than the second exposure period, and the number of pulses of the clock signal during the first exposure period is equal to or greater than the maximum count value of the counter. 2. The photoelectric conversion device according to 2 above. 4. The photoelectric conversion device according to claim 1, wherein the threshold is a maximum count value of the counter. 4. The photoelectric conversion device according to claim 1, wherein the threshold is smaller than the maximum count value of the counter. 4. The photoelectric conversion device according to claim 1, wherein the threshold is a maximum count value/n of the counter. The predetermined exposure period has a first exposure period and a second exposure period that is a longer exposure period than the first exposure period,
7. The photoelectric conversion device according to claim 6, wherein said n is equal to or greater than the length of said second exposure period/length of said first exposure period. 3. The photoelectric conversion device according to claim 2, wherein the length of said second exposure period is at least twice the length of said first exposure period. The predetermined exposure period has a third exposure period that is longer than the second exposure period, and the length of the third exposure period is the length of the second exposure period. 9. The photoelectric conversion device according to claim 2 or 8, which is two times or more. The predetermined exposure period has a third exposure period that is longer than the second exposure period, and the length of the third exposure period is the length of the second exposure period. four times more,
10. The photoelectric conversion device according to claim 2, wherein the length of said second exposure period is four times or more the length of said first exposure period. 2. The photoelectric conversion device according to claim 1, further comprising a control circuit that changes a potential supplied to a gate of a transistor included in said circuit when said counter reaches said threshold value within said predetermined exposure period. 2. The photoelectric conversion device according to claim 1, further comprising a control circuit for stopping said counter when said counter reaches said threshold within said predetermined exposure period. 2. The photoelectric conversion device according to claim 1, wherein the number of bits of said memory is smaller than the number of bits of said counter. 2. The photoelectric conversion device according to claim 1, further comprising a logic circuit between said photodiode and said counter, wherein said clock signal is input to said logic circuit during said exposure period. 2. The photoelectric conversion device according to claim 1, wherein said clock signal has two or more frequencies within said exposure period. A first frequency that is an average frequency of the clock signal within the first exposure period and a second frequency that is an average frequency of the clock signal during the second exposure period are different. The photoelectric conversion device according to claim 2. 17. The photoelectric conversion device according to claim 16, wherein said first frequency is higher than said second frequency. 3. The method according to claim 2, wherein the number of pulses of said clock signal per unit time during said first exposure period is greater than the number of pulses of said clock signal per unit time during said second exposure period. Photoelectric conversion device. An arithmetic circuit for calculating a signal output from the photoelectric conversion device according to claim 1,
An arithmetic circuit, wherein the arithmetic circuit changes a multiplication factor of the count value of the counter based on the time information. 19. The photoelectric conversion device according to claim 1, wherein the photoelectric conversion device according to claim 1 has the arithmetic circuit according to claim 19. 20. A photoelectric conversion system comprising the photoelectric conversion device according to claim 1 and the arithmetic circuit according to claim 19. a photoelectric conversion device according to claim 1;
and a signal processing unit that generates an image using a signal output from the photoelectric conversion device. A moving object comprising the photoelectric conversion device according to claim 1,
A moving object, comprising: a control unit that controls movement of the moving object using a signal output from the photoelectric conversion device.

Images