WO2025202149A1

WO2025202149A1 - Image sensor with single photon detectors and image sensor system

Info

Publication number: WO2025202149A1
Application number: PCT/EP2025/058028
Authority: WO
Inventors: Christian BRÄNDLI
Original assignee: Sony Advanced Visual Sensing AG; Sony Semiconductor Solutions Corp
Current assignee: Sony Advanced Visual Sensing AG; Sony Semiconductor Solutions Corp
Priority date: 2024-03-28
Filing date: 2025-03-25
Publication date: 2025-10-02
Anticipated expiration: 2026-09-28

Abstract

An image sensor system includes a pixel array with a plurality of single photon detectors, wherein each single photon detector is configured to output an inactive voltage level in a state after reset and before detecting a photon and output an active voltage level in response to receiving a photon. A signal interface is configured to output binary detector information about active and inactive voltage levels of at least the predefined subset of the single photo avalanche detectors. A flux estimator is configured to estimate photon flux information from the binary detector information by a statistical analysis of a sequence of consecutively captured binary detector information.

Description

IMAGE SENSOR WITH SINGLE PHOTON DETECTORS AND IMAGE SENSOR SYSTEM

The present disclosure relates to an image sensor which includes a pixel array of single photon detectors, and an image sensor system, which includes an image sensor with single photon detectors.

BACKGROUND

Single photon pixel arrays include a plurality of single photon detectors (SPDs), e.g., single photon avalanche diodes (SPADs) formed in a semiconductor body along pixel rows and pixel columns. Each SPAD includes a photosensitive region. In the operating mode, two opposite electrodes generate an electric field across the photosensitive region, wherein the electric field is approximately as high as the avalanche breakdown electric field. Each photon of sufficient energy entering the photosensitive region generates an electron-hole pair. The electric field separates the generated charge carriers according to their polarity. Since the bias voltage is approximately as high as or higher than the avalanche breakdown voltage, the generated charge carriers generate further charge carriers that induce an avalanche current through the SPAD.

When the electric field is strong enough to drive the avalanche breakdown into its saturation range, the SPAD operates in Geiger mode. After the avalanche breakdown, a quench mechanism reduces the electric field in the photosensitive region to stop the avalanche breakdown. The quenching resets the SPAD.

In Geiger mode, the multiplication effect of the avalanche breakdown is great enough such that it is no longer possible to deduce the radiation intensity from the photocurrent of the SPAD. Instead, the SPAD is reset after each avalanche breakdown and the information about the intensity of detected radiation is derived from the number of avalanche breakdowns per time unit. Other types of SPDs include highest-sensitive photodiodes.

The object of the present application is to provide an image sensor based on SPDs that makes it possible to extract essential image information at low circuit complexity in the image sensor.

SUMMARY

An SPD image sensor typically requires counters that count the number of photon detections during an exposure time for each SPD separately. The number of avalanche breakdowns corresponds to the number of detected photons and increases with the detected radiation intensity. A counter provided as part of the SPD image sensor requires a substantial amount of silicon area on the SPD image sensor.

The present disclosure mitigates such deficiencies of the prior art. In particular, the present disclosure relates to an image sensor with SPDs that detect light intensity based on information about whether or not a photon is detected in predefined time periods. Accordingly, an image sensor system includes a pixel array with a plurality of single photon detectors, wherein each single photon detector is configured to output an inactive voltage level in a state after reset and before detecting a photon, and an active voltage level in response to receiving a photon. A signal interface is configured to output binary detector information about active and inactive voltage levels of at least the predefined subset of the single photo avalanche detectors. A flux estimator is configured to estimate photon flux information from the binary detector information by a statistical analysis of a sequence of consecutively captured binary detector information.

Instead of predefining a suitable exposure time and counting the number of avalanche breakdowns for each SPD during the exposure time, the image sensor outputs a sequence of binary detector information (“binary images”). For all SPDs or for all SPDs of a predefined subset of SPDs, each binary detector information indicates whether or not an SPD has detected a photon. The binary detector information in combination with information about the predefined time periods can be processed in a processor outside the image sensor. The greater computational resources that can be made available outside the image sensor improve performance and open up additional possibilities for controlling the image sensor. The additional information can be used estimate to photon flux information from the binary detector information by a statistical analysis. As the analogue-to-digital conversion (ADC) in the form of the one-bit pixel already takes place in the single photon detector, the read noise can be removed.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of the disclosure and many of the advantages associated therewith will be obtained by reference to the following detailed description in conjunction with the accompanying drawings, in which:

FIG. 1 is a schematic block diagram illustrating an image sensor with a pixel array with single photon detectors, and a flux estimator in accordance with an embodiment.

FIG. 2 is a simplified circuit diagram illustrating a configuration example of an image sensor with an exposure controller controlling a passive quenching of the single photon detectors in accordance with an embodiment.

FIG. 3 is a simplified circuit diagram illustrating a configuration example of an image sensor with active quenching single photon detectors in accordance with another embodiment.

FIG. 4 is a simplified circuit diagram illustrating a configuration example of an image sensor with high-Z quenching single photon detectors in accordance with another embodiment.

FIG. 5 is a simplified circuit diagram illustrating a configuration example of a signal interface with pixel line reset and row buffers in accordance with an embodiment. FIG. 6 is a circuit diagram illustrating a configuration example of a single photon detector for a pixel array in combination with a signal interface with exposure control signal and row select signal for global shutter in accordance with an embodiment.

FIG. 7 is a simplified block diagram illustrating a configuration example of a pixel array and a signal interface for the single photon detector of FIG. 6.

FIG. 8 is a circuit diagram illustrating a configuration example of a single photon detector for a pixel array in combination with a signal interface with exposure control signal and row select signal for ROI global shutter in accordance with an embodiment.

FIG. 9 is a simplified block diagram illustrating a configuration example of a pixel array and a signal interface for the single photon detector of FIG. 8.

FIG. 10 is a schematic block diagram of an image sensor with an entropy encoder in accordance with an embodiment.

FIG. 11 is a schematic block diagram of an image sensor with a photon statistics estimator configured to control the exposure controller in accordance with an embodiment.

FIG. 12 is a schematic block diagram of an image sensor system including an image sensor with single photon detectors and a flux estimator in accordance with a further embodiment.

FIG. 13 is a schematic block diagram of an image sensor system including an image sensor with single photon detectors and a flux estimator controlling an exposure controller of the image sensor in accordance with an embodiment.

FIG. 14 is a schematic block diagram of an image sensor system including an image sensor with single photon detectors and a flux estimator separately controlling single photon detectors assigned to pixel subgroups of a pixel array in accordance with an embodiment.

FIG. 15 is a schematic block diagram illustrating an image sensor sensor with a pixel array with single photon detectors, an exposure controller, and a signal interface in accordance with the embodiments.

DETAILED DESCRIPTION

Embodiments for implementing techniques of the present disclosure will be described below in detail using the drawings. The techniques of the present disclosure are not limited to the described embodiments, and various numerical values and the like in the embodiments are illustrative only. The same elements and elements with the same functions are denoted by the same reference signs. Duplicate descriptions are omitted. Connected electronic elements may be directly electrically connected through a direct and permanent low- resistive connection, e.g., through a conductive line. The terms “connected”, “electrically connected” and “signal-connected” may also include a connection through other electronic elements provided and suitable for permanent and/or temporary signal transmission and/or transmission of energy. For example, electronic elements may be electrically connected or signal-connected through resistors, capacitors, and electronic switches such as transistors or transistor circuits, e.g., field effect transistors (FETs), transmission gates, complementary switches, an FET and a dummy switch electrically connected in series, and others.

The load path of a transistor is the controlled current path through a transistor. For example, a voltage applied to the gate of a FET controls the current flow through the load path (controlled path) between source and drain of the FET by field effect. When it is described that a transistor is connected in series with another element or is connected in parallel with another element, this connection refers to the load path of the transistor.

A digital signal alternates between at least one active level and at least one passive level. A digital signal having an active level is active. A digital signal having an inactive level is inactive. The active level can be a digital high level or a digital low level. The inactive level can be a digital low level or a digital high level.

FIG. 1 shows an image sensor system 90 that includes a pixel array 10 with a plurality of single photon detectors 100, wherein each single photon detector 100 is configured to output an inactive voltage level in a state after reset and before detecting a photon, and output an active voltage level in response to receiving a photon. A signal interface 30 is configured to output binary detector information about active and inactive voltage levels of at least the predefined subset of the single photo avalanche detectors 100. A flux estimator 60 is configured to estimate photon flux information from the binary detector information by a statistical analysis of a sequence of consecutively captured binary detector information.

The single photon detectors are arranged in pixel rows and pixel columns, wherein each single photon detector can be identified by a row address and a column address. Each single photon detector includes a single photon avalanche diode (SPAD) and an output stage. The SPAD includes a photosensitive region to which an electric field is applied. The magnitude of the electric field is selected such that an electron-hole pair generated by an incoming photon triggers an avalanche breakdown. The avalanche breakdown induces an avalanche current through the photosensitive region. A quench element is configured to interrupt the avalanche current in response to a quench control signal. Interrupting the avalanche current resets the SPAD and the output stage.

The output stage of the single photo avalanche detector outputs a digital signal changing between an inactive voltage level and an active voltage level. After reset and before an avalanche breakdown, the output stage outputs the inactive voltage level. An avalanche breakdown triggers the output stage to output the active voltage level. The active voltage level may be a high voltage level or a low voltage level. The inactive voltage level is the low voltage level when the active voltage level is the high voltage level. The inactive voltage level is the high voltage level when the active voltage level is the low voltage level.

The exposure controller 20 simultaneously resets at least the output stage of all or a predefined subset of the single photon detectors 100 in predefined time periods. The reset can take place regardless of whether or not an avalanche breakdown has actually occurred in the SPAD since the preceding reset.

The single photon detectors 100 may include a passive quenching circuit (PQC), an active quenching circuit (AQC) or a circuit for high-Z quenching. For example, for PQC for single photon detectors 100, the exposure controller 20 may reset the single photon detectors 100 by activating a quenching element that interrupts the avalanche current.

The signal interface 30 outputs the binary detector information about active and inactive voltage levels of all or predefined subsets of the single photo avalanche detectors 100 for each of the predefined time periods at the time of reset. For example, the binary detector information may include “l”s for active voltage levels and “0”s for inactive voltage level. The binary detector information is decoded in a form that allows to assign the “l”s and “0”s to the row and column addresses of the respective single photon detector. Each binary detector information represents a binary image for an exposure time equal to the length of the applied predefined time period.

The image sensor system 90 further includes a receiving device 80. The receiving device 80 receives the multitude of binary detector information and can integrate the number of “l”s for each single photon detector 100 to obtain information about the light intensity received by the single photon detector 100 in the various time intervals. By analyzing the binary detector information over a cycle period that includes various predefined time periods, the receiving device 80 may generate light intensity information and/or control signals for controlling the exposure controller 20. The image sensor system 90 may be a camera or a computer-vision apparatus.

Instead of integrating the photon counts on the sensor, the photon counts can be “integrated” in the receiving device 80 where a larger amount of computational resources is available that allows to achieve better performance as well as calculating additional control functions. Some meta-statistics may be performed on the overall photon count, wherein the array readout is binary (or compressed binary). The predefined time periods may have the same length.

According to an embodiment, the flux estimator 60 is configured to estimate photon flux information by fitting a Poison distribution over time distances between detections of photons in a single photon detector 100, wherein the time distances are obtained from a sequence of binary detector information.

The photon flux is the number of photons received per time. Typically, the number of photons counted per exposure is divided by the exposure time. The flux estimator 80 may make use of the observation that the photon flux follows a Poisson distribution with the number of photons representing the events k and the exposure representing the interval t as given in equation #1, and may estimate the Poisson distribution based on a statistical analysis of individual instances:

#1 P(k events in interval t)

If the number of photons per time unit follows a poison distribution, the waiting time for the first photon is 1/ q. For estimating the flux based on binary observations with variable exposure time, the Poisson distribution is fit over the observations. The fitting can be done explicitly. According to an embodiment, neural networks are trained to perform this inference.

The image sensor system 90 with the flux estimator 60 combines the capture of multiple single bit images with a statistical analysis of the time distances between successive detections. The image sensor system makes use of the fact that the sequence of single bit images contains a lot of statistical information that is lost when the count is averaged across an exposure period. The inference of the flux can be used to update the exposure parameters to maximize time- inference performance. The network can learn which exposures will provide it with the most information for a reliable inference on the underlying Poisson distribution.

The flux estimator 60 may be integrated in a processor outside of the image sensor 40. The binary detector information is received from the image sensor 40 and includes information about one or more binary images, wherein each binary image is defined by the active and inactive voltage levels of the single photon detector output signals. In addition, the flux estimator 60 may consider information about the lengths of the time intervals in which the photons for the binary detector information have been detected. The information about the lengths of the time periods may be received from the image sensor or may be inherently available in the flux estimator 60.

The image sensor system 90 determines the intensity of incoming radiation as photon flux by detecting single photons over a variable amount of time instead of estimating the intensity by integrating the photon count over a fixed exposure time.

Instead of integrating the photon counts in the image sensor 40 for each single photon detector 100 separately, the flux estimator 60 estimates the photon flux based on the binary detector information to obtain information about the intensity of the incoming radiation. The greater computational resources available outside of the image sensor 40 improve the processing performance.

In addition, the flux estimator 60 can provide some meta-statistics for the overall photon count.

According to an example without feedback from the flux estimator 60 to the image sensor 40, the exposure controller 20 controls the capture of binary images using predefined time periods, wherein a length of the predefined time periods increases according to a pre-defined pattern (e.g. Ins, 2ns, 4ns, 8ns, 16ns, ...). A cycle, which includes a predefined number of consecutive predefined time periods of different length, can be run repeatedly to improve the signal -to-noise ratio (SNR) by removing variance in the flux estimate. Alternatively, the cycle can be repeated to estimate an underlying motion in the captured scene and/or to estimate a change in brightness in the captured scene over time.

The estimated photon flux (flux estimate) output by the flux estimator 60 can represent an intensity-encoded static image. Alternatively, the flux estimator 60 estimates the photon flux as a function of time F_x>y(t). If the image sensor 40 is moving, the photon flux can also be estimated along a third spatial dimension z: F_x,y,z(t). The flux estimate may also be represented as an implicit representation similar to a neural radiance fields (NeRF) concept. The flux estimate can be used for photo, video or 3D rendering.

According to an embodiment, the single photo detector 100 includes a single photo avalanche diode 101. Alternatively, the single photo detector 100 includes a high-sensitive photodiode or JOT.

According to an embodiment, the predefined time periods may have different lengths.

The exposure time can be set by the flux estimator such that probability a single photon detector 100 of the pixel array 10 detects a photon in a single exposure is 50%.

The length of the predefined time periods may increase with each time interval applied. The length of the predefined time periods may increase strictly monotonically, e.g., linearly, quadratically or exponentially. For example, the length of the nth time interval may be 2^{(n l)}. For example, the length of a first time interval is Ins, the length of a second time interval is 2ns, the length of a third time interval is 4ns, the length of a fourth time interval is 8ns and the length of a fifth time interval is 16ns. The number of samples in a sweep depends on signal -to-noise (SNR) requirements. The actual exposure times depend on light and pixel parameters.

According to an embodiment, a cycle period includes a plurality of consecutive predefined time periods of different lengths.

A single cycle period is shorter by several orders of magnitude than an entire exposure period for capturing a static image of a scene Alternatively, the image sensor operates continuously and continuously repeats the cycle period. Successive cycle periods may have the same sequence of time intervals, wherein for each sequence of cycle periods, the predefined time periods and the order of the predefined time periods remains the same. Alternatively, the predefined time periods or the sequence of the time intervals may change, e.g., in response to an exposure control signal.

FIG. 2 shows an image sensor 40 with an exposure controller 20 configured to simultaneously reset all single photon detectors 100 of the pixel array 10 using PQC.

The pixel array 10 includes m x n single photon detectors 100, wherein each single photon detector 100 is unambiguously assigned to one of n pixel rows and one of m pixel columns and identifiable by a unique combination of row address and column address. Each single photon detector 100 may include a single photon avalanche diode (SPAD) 101, a controllable quench element 102, and an output stage 104. In the illustrated pixel array 10, the controllable quench element 102 is an n-channel field effect transistor (nFET) electrically connected between an anode of the SPAD 101 and a negative potential VL. The cathode of the SPAD 101 is electrically connected to a positive potential VH. The anode of the SPAD 101 is electrically connected to an input of the output stage 104.

When the controllable quench element 102 is on, a bias voltage applied between the high potential VH and the low potential VL is effective across a photosensitive area of the SPAD 101. The bias voltage is close to an avalanche breakdown voltage of the SPAD 101 and biases the SPAD 101 in the blocking direction. In a reset state after reset and before detecting a photon, the potential at the cathode of the SPAD is low and the output stage 104 outputs an inactive voltage level.

A photon crossing the electric field in the photosensitive area of an SPAD 101 triggers an avalanche breakdown through the SPAD 101 and an avalanche current flows through the SPAD 101. The avalanche current changes the potential at the anode of the SPAD 101. The output of the output stage 104 changes to an active voltage level.

The outputs Al 1 to Anm of all single photon detectors 100 that receive photons during the predefined time period change to the active voltage level. The outputs Al l to Anm of the single photon detectors 100 receiving no photons in the predefined time period output the inactive voltage level. The outputs Al l to Anm represent a binary image for an exposure lasting the respective predefined time period.

The inputs of the signal interface 30 receive the binary image. After the first predefined time period, the exposure controller 20 outputs an active global reset signal GRES that resets the single photon detectors 100. After reset, each single photon detector 100 outputs the inactive voltage levels until the next detection of a photon.

The output stage 104 includes a CMOS inverter with a p-channel field effect transistor (pFET) 105 and an nFET 106 electrically connected in series between a positive supply voltage VDD and a negative supply voltage VSS. The cathode of the SPAD 101 is electrically connected to the gates of the pFET 105 and the nFET 106. The quench elements 102 are nFETs that receive the global reset signal GRES at the gate electrodes. The exposure controller 20 outputs the global reset signal GRES at the end of each of the predefined time periods.

FIG. 3 shows an image sensor 40 using AQC. A charging transistor 103 is electrically connected between an off-voltage Voff and the SPAD anode. The output stage 104 includes a CMOS inverter 104 and an auxiliary circuit 107. An output of the auxiliary circuit 107 controls the quench element 102 and the charging transistor 103 in response to a detected avalanche to control a forced recharge of the SPAD 101.

In FIG. 4, instead of the auxiliary circuit 107 of FIG. 3, a first control signal XSOFF controls the charging transistor 103 and second control signal SON controls the quench element 102. The exposure controller 20 controls the first and second control signals XSOFF, SON to precharge the SPAD 101 and to leave the SPAD 101 in the high-Z mode. The avalanche stops when the voltage across the SPAD 101 falls below the breakdown voltage.

In each of FIG. 2, FIG. 3, and FIG. 4, the signal interface 30 receives the binary image from the outputs Al 1 to Anm of the single photon detectors 100 and transmits information containing the binary image to a receiving device outside the image sensor 40.

In addition, the exposure controller 20 may output an active clock signal CLK having a predefined temporal relationship with the reset of the pixel array 10. In particular, a change of the clock signal CLK to an active voltage level and a change of the global reset signal GRES to an active voltage level may have a predefined temporal relationship to each other.

The signal interface 30 may output the clock signal CLK or a signal derived from the clock signal CLK to enable the receiving device 80 to learn the validity of the binary detector information for each predefined time period.

FIG. 5 shows a pixel array 10 with single photon detectors 100 that include an SPAD 101 and a row select nFET 109. The single photon detectors 100 of a same pixel line 109 are connected in parallel between the positive supply voltage VDD and the pixel line 109, wherein the SPAD 101 and the row select nFET 109 are electrically connected in series between the positive supply voltage VDD and the column signal line. The exposure controller 20 controls pixel line reset transistors 301 for quenching the SPADs 101 and includes a row controller 25 for selecting pixel rows. The signal interface 30 includes one sense amplifier 302 and one row buffer circuit 303 per pixel line 309, and a shift register that reads in the outputs ouf the row buffer circuits 303 in parallel.

FIG. 6 shows a single photon detector 100 for a global shutter operation. The SPAD 101 and an nFET as quenching element 102 are electrically connected in series between a higher voltage and a lower voltage. An active global reset signal GRST switches on the quenching element 102. The controlled load path of an exposure transistor 108 is electrically connected between the anode of the SPAD 101 and the input of the output stage 104. An active exposure control signal EXP switches on the exposure transistor 108. The output stage 104 may include a latch function to hold information about a detected avalanche event until the next readout. The controlled load path of a select transistor 109 is electrically connected between the output of the output stage 104 and the pixel line 309. An active select signal SEL switches on the select transistor 109 once after each exposure. The exposure control signal EXP determines the length of the exposure. The select signal SEL selects one of a plurality of single photon detectors 100 to be connected to the pixel line 309.

FIG. 7 shows a pixel array 10 including the single photon detectors 100 of FIG. 6. The exposure controller 20 includes a row shift register 26. The number of register latches of the row shift register 26 and the number of pixel rows of the pixel array 10 are equal. Each register latch of the shift register 26 outputs the select signal SEL for one pixel row. A single logic “1” is shifted through the shift register to consecutively address the pixel rows. In addition, the exposure controller 20 controls the global reset signal GRST and the global exposure signal EXP according to a timing obtained from the flux controller.

In FIG. 8, the quenching element 102 is controlled by the output signal of a select gate 110. The output signal of the select gate 110 is active, when both the select signal SEL is active and a subgroup reset signal ROI C is active.

FIG. 9 shows an image sensor 40 with the exposure controller 20 being configured to simultaneously reset the single photon detectors 100-1, ..., 100-m, assigned to a subgroup 15 of the single photon detectors 100 of the pixel array 10.

The single photon detectors 100 of the pixel array 10 can be addressed in subgroups (regions-of-interest, ROI) 15 defined by a range of row addresses and a range of line addresses. Each subgroup 15 includes a plurality m of single photon detectors 100. The single photon detectors 100 assigned to the same subgroup 15 may be neighboring ones, both along the line direction and along the row direction. All single photon detectors 100 of the same subgroup 15 are controlled to execute a reset at the same time.

The exposure controller 20 includes a row shift register 26 as in FIG. 7 and a line shift register 27. The line shift register 27 includes as many register latches 271 as the pixel array 10 comprises pixel lines 309. For each register latch 271, an AND gate 272 gates the register latch output with the global reset signal GRST to obtain the subgroup reset signal ROI C for the single photon detector of FIG. 8. The addresses of the single photon detectors 200 of the subgroup 15 are shifted into the row shift register 26 and the line shift register 27.

If binary detector information obtained from preceding time intervals indicate that for the following time intervals certain regions-of-interest do not include significant additional information, the single photon detectors in the subgroups 15 assigned to the certain regions-of-interest can be shut down or switched into an idle state, e.g., by turning off the quenching elements 102. Alternatively, the outputs of the single photon detectors 100 in a region-of-interest can be suppressed in the pixel array 10 or in the signal interface 30.

The signal interfaces 30 illustrated in FIG. 2, FIG. 3 and FIG. X is configured to output the binary detector information from all single photo avalanche detectors 100 in parallel and simultaneously.

The signal interface 30 may include a purely passive wiring connection between the single photon detectors 100 and the receiving device. Alternatively, the signal interface 30 may include a driver circuit for each output signal Al 1 to Anm, and/or a one-bit latch circuit for temporarily storing each output signal Al 1 to Anm. The signal interface 30 can be provided without multiple-bit counters assigned to each single photon detector 100 such that the image sensor can be provided with a high fill factor and a high portion of active photosensitive area. FIG. 10 shows an image sensor 40 with a signal interface 30 that includes an entropy encoder 35 adapted for data compression of output signals of the single photon detectors 100.

The entropy encoder 35 compresses the output data of the pixel array 10. The binary detector information contains data-compressed binary images. Since for any given exposure, large parts of the image are either underexposed or overexposed, the each image can be efficiently compressed using entropy encoding schemes. For example, the entropy encoder 35 is configured to compress the binary images output by the single photon detectors 100 of the pixel array 10 using Huffman coding or run-length encoding (RLE). The entropy encoder 35 outputs the encoded binary image to the receiving device using a serial or parallel data bus.

In FIG. 11, the exposure controller 20 includes a programmable data register 25 that sets the predefined time periods. The content of the data register 25 defines the length of at least one predefined time period. The data register 25 facilitates the predefined time periods for capturing subsequent binary images being adapted in response to information obtained from previously captured binary images.

The image sensor 40 further includes a photon statistics estimator 50 configured to control the exposure controller 20 in response to statistic properties of previously obtained active and inactive voltage levels.

The photon statistics estimator 50 performs statistical analysis on the overall photon count or its variance (e.g. histograms). The result of the statistical analysis is used to optimize the exposure control policy in real-time and facilitates single-photon auto exposure by programming the data register 25 with time periods that promise a large amount of new information in the subsequently obtained binary images.

Compared to counters and memories for each individual single photon detector 100, the photon statistics estimator 50 requires little silicon area and can help to significantly reduce the amount of information to be transmitted from the image sensor 40 to the receiving device 80.

For example, the duration of a predefined time period is increased when the portion of “l”s in the binary image is low, e.g. smaller than 50%, and may be decreased when the total number of “l”s in the binary image is high, e.g. greater 50%.

In the embodiment of FIG. 12, the flux estimate is directly fed into a block for image processing and/or machine learning (ML) inference 70.

In FIG. 13 the flux estimator 60 is configured to control the exposure controller 20 in response to information obtained from the binary detector information.

The flux estimate is obtained from one or more previous binary images and from information about the length of the time periods in which the binary images are detected. The flux estimator 60 processes the flux estimate to obtain exposure control information. The exposure control information may include suitable time periods for the exposure of the binary images. Information about the suitable time periods is fed back from the flux estimator 60 to the image sensor 40 to select suitable time periods for subsequent exposure periods.

According to FIG. 14, the flux estimator 60 may include a pixel-level statistics estimator 65 configured to control the exposure controller 20 in response to statistic properties of the binary detector information.

The binary detector information may include one or more binary images and information about the length of the time periods for detecting the binary images. Since the binary photon count information is integrated in a processor outside of the image sensor 40 and not on the image sensor 40, the flux estimator 60 can perform elaborate pixel-level statistics and steer the exposure controller 20 in such a way that the flux estimation accuracy is maximized.

The pixel-level statistics estimator 65 can be configured to track a variance on a flux estimate and to control the exposure controller 20 to apply a time period for detection with a length (exposure time) such that the binary detector information has a maximal amount of new information.

This allows to choose the exposure time such that the binary detection information obtained for the chosen exposure time contains a maximal amount of new information compared to what the flux estimator 60 is already certain of (i.e., low variance).

The pixel-level statistics estimator may track the variance on the flux estimate by using a gradient decent algorithm. For example, the flux estimator 60 is configured to estimate an exposure time that corresponds to a photon detection likelihood of 50%.

The flux estimator 60 may include an artificial neural network (aNN). After training, the flux estimator 60 performs an inference on the underlying flux. In other words, the flux estimator 60 tries to estimate the exact exposure time that corresponds to a photon detection likelihood of exactly 50%. The simplest version of a flux estimation output is just for a discrete point in time i.e. an “image”. But the flux can also be estimated as a function of time F_x>y(t). If the sensor is moving, the flux may also be estimated along the third spatial dimension z: F_x>y>z(t). The flux function may also be represented as an implicit representation similar to a “NeRF”. The flux estimate can be used for photo, video or 3D rendering but it can also be directly fed into a neural network or other machine learning (ML) inference algorithm.

FIG. 14 shows a flux estimator 60 that includes a region-of-interest identifier 67 configured to identify areas where the flux estimate has converged and to control the exposure controller 20 and/or signal interface (30) to suppress output of information about voltage levels outside the identified areas.

To avoid unnecessary sampling and processing in the flux estimator for areas where the flux estimate has converged, the exposure controller 20 may define and use one or multiple regions-of-interest (ROI). The pixels n outside of these regions are not reset, not read, flagged or a combination thereof. The exposure controller 20 may use group reset signals GRST as described with reference to FIG. 8 and FIG. 9.

FIG. 15 shows an image sensor 40 includes a pixel array 10 with a plurality of single photon avalanche detectors 100. Each single photon avalanche detector 100 outputs an inactive voltage level in a state after reset and before detecting a photon, and outputs an active voltage level in response to receiving a photon. An exposure controller 20 simultaneously resets at least a predefined subset of the single photon avalanche detectors lOOin predefined time periods. A signal interface 30 outputs binary detector information about active and inactive voltage levels of at least the predefined subset of the single photo avalanche detectors 100.

The present technology can also be configured as described below:

[1] An image sensor system (90), comprising: a pixel array (10) comprising a plurality of single photon detectors (100), wherein each single photon detector (100) is configured to output an inactive voltage level in a state after reset and before detecting a photon, and outputs an active voltage level in response to receiving a photon; a signal interface (30) configured to output binary detector information about active and inactive voltage levels of at least the predefined subset of the single photo avalanche detectors (100); and a flux estimator (60) configured to estimate photon flux information from the binary detector information by a statistical analysis of a sequence of consecutively captured binary detector information.

[2] The image sensor system according to [1], wherein the flux estimator (60) is configured to estimate photon flux information by fitting a Poison distribution over time distances between detections of photons in a single photon detector (100), wherein the time distances are obtained from a sequence of binary detector information.

[3] The image sensor system according to any of [1] to [2], wherein the single photo detector (100) comprises a single photo avalanche diode (101).

[4] The image sensor system according to any of [1] to [3], wherein the exposure controller (20) is configured to simultaneously reset all single photon detectors (100) of the pixel array (10).

[5] The image sensor system according to any of [1] to [4], wherein the exposure controller (20) is configured to simultaneously reset the single photon detectors (100-1, 100-2, ...) assigned to a subgroup (15) of the single photon detectors (100) of the pixel array (10).

[6] The image sensor system according to any of [1] to [5], wherein the signal interface (30) comprises an entropy encoder (35) adapted for data compression of output signals of the single photon detectors (100).

[7] The image sensor system according to any of [1] to [6], wherein the exposure controller (20) comprises a programmable data register (22) configured to set the predefined time periods. [8] The image sensor system according to any of [1] to [7], further comprising: a photon statistics estimator (50) configured to control the exposure controller (20) in response to statistic properties of previously obtained active and inactive voltage levels.

[9] The image sensor system according to any of [1] to [8], wherein the flux estimator (60) is configured to control the exposure controller (20) in response to information obtained from the binary detector information.

[10] The image sensor system according to any of [1] to [9], wherein the flux estimator (60) comprises a pixel-level statistics estimator (65) configured to control the exposure controller (20) in response to statistic properties of the binary detector information.

[11] The image sensor system according to [10], wherein the pixel-level statistics estimator (65) is configured to track a variance on a flux estimate and to control the exposure controller (20) to apply a time period for detection with a length such that the binary detector information has a maximal amount of new information.

[12] The image sensor system according to any of [1] to [11], wherein the flux estimator (60) comprises an artificial neural network.

[13] The image sensor system according to any of [1] to [12], wherein the flux estimator (60) comprises a region-of-interest identifier (67) configured to identify areas where the flux estimate has converged and to control the exposure controller (20) and/or the signal interface (30) to suppress output of information about voltage levels outside the identified areas.

Claims

1. An image sensor system, comprising: a pixel array comprising a plurality of single photon detectors, wherein each single photon detector is configured to output an inactive voltage level in a state after reset and before detecting a photon, and outputs an active voltage level in response to receiving a photon; a signal interface configured to output binary detector information about active and inactive voltage levels of at least the predefined subset of the single photo avalanche detectors; and a flux estimator configured to estimate photon flux information from the binary detector information by a statistical analysis of a sequence of consecutively captured binary detector information.

2. The image sensor system according to claim 1, wherein the flux estimator is configured to estimate photon flux information by fitting a Poison distribution over time distances between detections of photons in a single photon detector, wherein the time distances are obtained from a sequence of binary detector information.

3. The image sensor system according to claim 1, wherein the single photo detector comprises a single photo avalanche diode.

4. The image sensor system according to claim 1, wherein the exposure controller is configured to simultaneously reset all single photon detectors of the pixel array.

5. The image sensor system according to claim 1, wherein the exposure controller is configured to simultaneously reset the single photon detectors assigned to a subgroup of the single photon detectors of the pixel array.

6. The image sensor system according to claim 1, wherein the signal interface comprises an entropy encoder adapted for data compression of output signals of the single photon detectors.

7. The image sensor system according to claim 1, wherein the exposure controller comprises a programmable data register configured to set the predefined time periods.

8. The image sensor system according to claim 1, further comprising: a photon statistics estimator configured to control the exposure controller in response to statistic properties of previously obtained active and inactive voltage levels.

9. The image sensor system according to claim 1, wherein the flux estimator is configured to control the exposure controller in response to information obtained from the binary detector information.

10. The image sensor system according to claim 1, wherein the flux estimator comprises a pixel-level statistics estimator configured to control the exposure controller in response to statistic properties of the binary detector information.

11. The image sensor system according to claim 10, wherein the pixel-level statistics estimator is configured to track a variance on a flux estimate and to control the exposure controller to apply a time period for detection with a length such that the binary detector information has a maximal amount of new information.

12. The image sensor system according to claim 1, wherein the flux estimator comprises an artificial neural network.

13. The image sensor system according to claim 1, wherein the flux estimator comprises a region-of-interest identifier configured to identify areas where the flux estimate has converged and to control the exposure controller and/or the signal interface to suppress output of information about voltage levels outside the identified areas.