73342 1 IMAGE SENSOR ASSEMBLY WITH ANALOG COMPUTE MODULE The present disclosure relates to an image sensor assembly with pixel circuits generating pixel signal pulses having a pulse width that is a function of radiation intensity incident on the pixel circuit. BACKGROUND Image sensors for solid-state imaging devices have pixel arrays with pixel circuits organized in pixel rows and pixel columns. Each pixel circuit includes one or more photoelectric conversion elements and pixel transistors. The photoelectric conversion elements generate photocurrents that are proportional to the received radiation intensity. The pixel transistors control an initialization of the photoelectric conversion elements in an initialization period, the integration of the photocurrents in an exposure period, the conversion of the integrated photocurrents into analog pixel signals, and the output of the analog pixel signals in a readout period. A voltage amplitude of the analog pixel signal is a function of the received radiation intensity. Analog-to-digital converters convert the analog pixel signals into digital pixel values. A digital unit compiles the digital pixel values into frames and scales down the frames. A compute module may perform simple types of image processing, for example spatial and/or temporal filtering. SUMMARY Computation of even the scaled down frames in the digital sphere can consume a considerable amount of the total energy consumed by the image sensor assembly. The present disclosure mitigates such and other shortcomings of the prior art. For this purpose, an image sensor assembly includes an image sensor array having a pixel kernel. The pixel kernel includes pixel circuits, wherein each pixel circuit generates a pixel signal pulse having a signal pulse width. The signal pulse width is a function of radiation intensity incident on the pixel circuit. The image sensor assembly further includes a plurality of analog compute modules. Each analog compute module receives the pixel signal pulse from a first one of the pixel circuits of the pixel kernel, outputs weighted pixel signal pulses to each of the other pixel circuits of the pixel kernel with separate weighting coefficients, receives the weighted pixel signal pulses from the other analog compute modules assigned to the pixel kernel, and generates a computed pulse having a computed pulse width. The computed pulse width is a function of the signal pulse widths of the received weighted pixel signal pulses and the weighting coefficients applied to the received weighted pixel signal pulses.
The computed pulse width may be also a function of the signal pulse width of the pixel signal pulse of the first one of the pixel circuits and a pixel-internal weighting coefficient applied to the pixel signal pulse of the first one of the pixel circuits. The image sensor assembly performs a convolution of the neighborhood of each pixel according to an analog convolution function f(x), whereby the received radiation intensity information x is encoded in the pulse width and the effect of the neighboring pixels is encoded in the pulse amplitude. The exposure can be performed with a global shutter. Convolution is performed in the analog domain before analog-to-digital conversion of the radiation intensity information takes place and can be performed in parallel for all pixels simultaneously. The total power consumption can be significantly reduced. BRIEF DESCRIPTION OF THE DRAWINGS A more complete appreciation of the disclosure and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein: FIG. 1A is a simplified block diagram of an image sensor assembly with an image sensor array encoding different radiation intensity into pixel signal pulses with different pulses widths and with analog compute modules processing the pixel signal pulses in accordance with an embodiment of the present technology. FIG. 1B is a simplified block diagram showing components of the image sensor array and the analog compute modules of the image sensor assembly of FIG. 1A in more detail. FIG. 2 shows a schematic block diagram of a pixel circuit and a corresponding analog compute module of the image sensor assembly illustrated in FIG. 1A. FIG. 3A and FIG. 3B are circuit diagrams of programmable current sources based on charge trapping transistors for an analog compute module in accordance with embodiments directed to one and two convolution kernels. FIG. 4 and FIG. 5 are conceptional illustrations schematically showing the lateral arrangement of pixel circuits of a pixel kernel and the weighted input and output signal pulses for an analog compute module of the main pixel circuit of a 3x3 pixel kernel in accordance with an embodiment. FIG. 6 shows a simplified circuit diagram of a pixel circuit and an analog compute module assigned to the pixel circuit in accordance with an embodiment.
FIG. 7A shows a simplified circuit diagram of a pixel circuit and an analog compute module assigned to the pixel circuit in accordance with an embodiment with charge trapped transistors as programmable current sources and using a single convolution kernel. FIG. 7B shows a configuration of outputs of four analog compute modules of a pixel field with four pixels with analog compute modules as illustrated in FIG. 7A. FIG. 7C shows a time diagram for signals applied to and generated in the analog compute module of FIG. 7A. FIG. 8A shows a simplified circuit diagram of a pixel circuit and an analog compute module assigned to the pixel circuit in accordance with embodiments using more than one convolution kernels. FIG. 8B shows a block diagram for a driver circuit for control signals of the pixel circuit and the analog compute module illustrated in FIG. 8A. FIG. 9 shows a time diagram for signals applied to and generated in the analog compute module of FIG. 8 according to an operation method providing one exposure per pixel kernel and one pixel signal pulse per pixel kernel according to an embodiment. FIG. 10 shows a time diagram for signals applied to and generated in the analog compute module of FIG. 8 according to an operation method providing one exposure for all pixel kernels and two pixel signal pulses per pixel according to an embodiment. FIG. 11 shows a time diagram for signals applied to and generated in the analog compute module of FIG. 8 according to an operation method providing one exposure for all pixel kernels and a summarized kernel output according to an embodiment. FIG. 12 shows a time diagram for signals applied to and generated in the analog compute module of FIG. 8 according to an operation method providing one exposure for all pixel kernels and a summarized kernel output according to another embodiment. FIG. 13 is a simplified circuit diagram of a pixel circuit and an analog compute module assigned to the pixel circuit in accordance with an embodiment related to a pixel circuit using a global ramp voltage for pulse width modulating the pixel signal pulses.
FIG. 14 shows a time diagram for signals applied to and generated in the analog compute module of FIG. 13 in accordance with an embodiment of the present technology. FIG. 15 illustrates a simplified circuit diagram of a pixel circuit and an analog compute module with bypass option in accordance with an embodiment of the present technology. FIG. 16 is a diagram showing an example of a laminated structure of an image sensor assembly according to an embodiment of the present disclosure. FIG.17 is a block diagram depicting an example of a schematic configuration of a vehicle control system. FIG. 18 is a diagram of assistance in explaining an example of installation positions of an outside-vehicle information detecting section and an imaging section of the vehicle control system of FIG. 17. DETAILED DESCRIPTION Embodiments for implementing techniques of the present disclosure (also referred to as “embodiments” in the following) will be described below in detail using the drawings. The techniques of the present disclosure are not limited to the described embodiments, and various features in the embodiments are illustrative only. The same elements or elements with the same functions are denoted by the same reference signs. Duplicate descriptions are omitted. Connected electronic elements may be electrically connected through a direct, permanent low-resistive connection, e.g., through a conductive line. The terms “electrically connected” and “signal-connected” may also include a connection through other electronic elements provided and suitable for permanent and/or temporary signal transmission and/or transmission of energy. Electronic elements can be electrically connected or signal-connected via resistors, capacitors, electronic switches such as FETs (field effect transistors), or transistor circuits such as transmission gates. Directly electrically connected electronic elements are connected through a low-resistive wiring, an ohmic contact and/or a unipolar semiconductor junction. The load path of a transistor is the controlled current path through a transistor. A voltage applied to the gate of a field effect transistor (FET) controls the current flow through the load path (controlled path) between source and drain of the FET by field effect. A digital signal alternates between at least one active level and at least one inactive level. A digital signal having an active level is active. A digital signal having an inactive level is inactive. For each signal
5 separately, the active level can be a digital high level and the inactive level a digital low level, or the active level can be the digital low level and the inactive level the digital high level. Though in the following a technology for reducing power consumption in image sensor assemblies is described in the context of certain types of pixel circuits for intensity readout with one photoelectric conversion device and three or four pixel transistors, the technology may also be used for pixel circuits having more than one photoelectric conversion device and more than three or four pixel transistors. FIG. 1A illustrates a configuration example of an image sensor assembly 70 in accordance with embodiments of the present technology. The image sensor assembly 70 includes an image sensor array 10, a signal processing array 20, a pixel driver unit 30, a peripheral processing unit 40, and a sensor controller 50. The image sensor array 10 includes a plurality of identical pixel circuits 100 outputting pulse width modulated (PWM) pixel signal pulses. Each pixel circuit 100 includes at least one photoelectric conversion element and three or more pixel transistors. The photoelectric conversion element converts incident radiation into a pixel voltage signal (analog pixel signal) with a voltage level being a function of the incident radiation intensity. The pixel transistors may be field effect transistors (FETs) that control reset and exposure of the photoelectric conversion element and, if applicable, a pulse width modulation for converting the pixel voltage signal into a pixel signal pulse with a pulse width being a function of the voltage level of the pixel voltage signal. The pixel signal pulses are transmitted to a signal processing array 20. The pixel driver unit 30 generates pixel control signals for operating the pixel circuits 100 in response to timing control signals received from the sensor controller 50. The pixel control signals may control the pixel transistors to reset the pixel circuit 100 at the beginning or the end of an exposure period and/or to start a pulse width modulation of the pixel voltage signal. The pixel driver unit 30 controls all pixel circuits 100 of all pixel circuits 100 of the image sensor array 10 or of a selected group of pixel circuits 100 synchronously. The selected group of pixel circuits 100 may include one or more pixel blocks of laterally neighboring pixel circuits 100. The signal processing array 20 includes one analog computing module 200 for each pixel circuit 100 or group of pixel circuits 100. Each analog computing module 200 receives the pixel signal pulse from one of the pixel circuits 100 and processed signal pulses derived from the pixel signal pulses output from neighboring pixel circuits 100. The analog computing module 200 generates a computed pixel signal from the pixel signal pulse and the received processed signal pulses derived from the pixel signal pulses output
73342 6 from the neighboring pixel circuits 100. The analog computing module 200 further provides a processed signal pulse derived from the pixel signal pulse of the pixel circuit 100 to the neighboring pixel circuits 100. The computed pixel signals obtained from the same exposure define a pre-computed image data. The signal processing array 20 outputs the pre-computed image data through a data interface, for example, to the peripheral processing unit 40 for further high-level processing of the pre-computed image data. The sensor controller 50 generates the driver timing signals and outputs the driver timing signals to the pixel driver unit 30. The sensor controller 50 generates analog computing control signals for controlling the signal processing array 20 and may generate a readout control signal that controls the readout of the computed pixel values from the signal processing array 20 to the peripheral processing unit 40 and/or to a digital interface. According to an aspect of the present disclosure illustrated in FIG. 1B, the image sensor assembly 70 of FIG. 1A includes an image sensor array 10 that includes a pixel kernel 15. The pixel kernel 15 includes a plurality of pixel circuits 100. Each pixel circuit 100 is configured to generate a pixel signal pulse having a signal pulse width. The signal pulse width is a function of radiation intensity incident on the pixel circuit 100. The image sensor assembly 70 further includes a plurality of analog compute modules 200. Each analog compute module 200 is configured to receive the pixel signal pulse from a first one of the pixel circuits 100 of the pixel kernel 15, output weighted pixel signal pulses to each of the other pixel circuits 100 of the pixel kernel 15 with separate weighting coefficients, receive the weighted pixel signal pulses from the other analog compute modules 200 assigned to the pixel kernel 15, and generate a computed pulse having a computed pulse width. The computed pulse width is a function of the signal pulse widths of the received weighted pixel signal pulses and the weighting coefficients applied to the received weighted pixel signal pulses. The computed pulse width is also a function of the signal pulse width of the pixel signal pulse of the first one of the pixel circuits and a pixel-internal weighting coefficient applied to the pixel signal pulse of the first one of the pixel circuits. Each pixel circuit 100 may be part of more than one pixel kernel 15. For example, each pixel circuit 100 may be part of as many pixel kernels 15 as each pixel kernel 15 includes pixel circuits 100. The number N of pixel circuits 100 per pixel kernel 15 may be any number greater one, for example three, five or nine. The number of analog compute modules 200 associated with the same pixel kernel 15 is equal to the number N of pixel circuits 100 per pixel kernel 15.
The pixel circuit 100 includes a conversion portion 110 and a PWM portion 130. The conversion portion 110 converts the radiation received during an exposure period into a pixel voltage signal (analog pixel signal), wherein a voltage level of the pixel voltage signal is a function of the radiation intensity accumulated in the exposure period. The PWM portion 130 converts the pixel voltage signal into a pixel signal pulse, wherein a signal pulse width of the pixel signal pulse is a function of the voltage level of the pixel voltage signal. Each analog compute module 200 receives the pixel signal pulse output from a first one of the pixel circuits 100 of the pixel kernel 15 (main pixel). Each analog compute module 200 generates N-1 weighted pixel signal pulses from the pixel signal pulse received from the main pixel (main pulse signal) and outputs each single of the N-1 weighted pixel signal pulses to exactly one of the other analog compute modules 200 assigned to the same pixel kernel (satellite pixels). All transmitted weighted pixel signal pulses have the same signal pulse width, wherein the signal pulse width may be equal to the signal pulse width of the main pulse signal. The weighting affects a signal amplitude of the weighted pixel signal pulses. The signal amplitude may be a current amplitude or a voltage amplitude. The signal amplitudes of the weighted pixel signal pulses can be different for each of the N-1 pixel output pulses. At the same time, each analog compute module 200 receives one weighted pixel signal pulse from each of the satellite pixels (pixel input pulse). The received satellite signal pulses can differ from each other in terms of signal pulse width and/or signal amplitude, wherein the signal amplitude may be a current amplitude or a voltage amplitude. In the illustrated embodiment, the pixel kernel 15 includes nine pixel circuits 100-0, …, 100-8 arranged in a 3x3 matrix. Each pixel circuit 100-0, …, 100-8 transmits a pixel signal pulse (pixel output pulse) to a corresponding analog compute module 200-0, …, 200-3; 200-5, …,200-8. The main pixel includes the fourth pixel circuit 100-4 and the fourth analog compute module 200-4, wherein the fourth pixel circuit 100-4 outputs a pixel signal pulse with a signal pulse width t4 to the fourth analog compute module 200-4. The analog compute module 200-4 of the main pixel transmits eight pixel output pulses O0-O3, O5-O8 having the same pulse width t4 and different pulse amplitudes defined by the weights w0-w3, w5-w8 to the analog compute modules 200-0, …, 200-3; 200-5, …,200-8 of the satellite pixels. The analog compute module 200-4 of the main pixel receives eight pixel input pulses I0-I3, I5-I8 having different pulse widths t0-t3, t5-t8 and different pulse amplitudes defined by the weights w8-w5, w3-w0 from the analog compute modules 200-0, …, 200-3; 200-5, …,200-8 of the satellite pixels. The analog compute module 200-4 generates a sum signal over the eight pixel input pulses I0-I3, I5-I8 and the weighted pixel signal pulse of the main pixel, and pulse width modulates the sum signal to generate a computed pulse VD4 with a computed pulse width tp4. The computed pulse width tp4 is a function of the signal pulse width t4 of the
8 pixel signal pulse of the main pixel, the main pixel weight w4 for the pixel signal pulse of the main pixel 100-4, the signal pulse widths t0-t3, t5-t8 of the pixel input pulses I0-I3, I5-I8, and the amplitudes of the pixel input pulses I0-I3, I5-I8. Conventionally, when pixel circuits of image sensors convert radiation into analog voltage signals with the resulting voltage level being a function of the detected radiation intensity, all analog voltage signals obtained from one exposure are converted into a digital image, which is transferred to a processing system with memory and computing blocks. The computing blocks process the digital image in the digital sphere. Due to the large amount of image data, the computing blocks can consume a considerable amount of electrical energy. By shifting a part of the computational load into the analog sphere, power consumption can be significantly reduced. The weighting coefficients w0, …, w8 of each analog compute module 200 are configurable and reconfigurable. The weighting coefficients w0, …, w8 can be learned in a training phase. In the training phase known test images are fed into a model of the image sensor and the result output by the image sensor is classified. A loss function is determined based on what the image sensor outputs and what is the intended target. The loss function is used to determine the weighting coefficients such that the image sensor output approximates the intended target. For each weighting coefficients w0, …, w8 different values can be trained for different application tasks. The trained weighting coefficients for a specific image classification task are then programmed in the analog compute modules, e.g., in the form of charge in the gate dielectrics of charge trapping transistors, and the image sensor assembly operates with the programmed weighting coefficients. The weighting coefficients can be reconfigured during operation of the image sensor assembly 70, when the application task changes. The reconfigurable weights for the output signals of neighboring pixel circuits allow to combine and spatially filter the information obtained from all pixel circuits. The reconfigurable weights allow a training of the filter coefficients and an adaption of the filter coefficients in an application for improving further processing of the captured images, for example, for feature recognition and/or motion estimation. The pulse width modulation enables the weighting of the pixel signals by modifying the signal amplitude. In this way, a significant part of the image processing can be done with low power consumption within the analog part of the image sensor assembly. As shown on the left side in FIG. 2, the conversion portion 110 of the pixel circuits 100 of an image sensor array 10 are arranged to form a two-dimensional matrix with columns and rows, each pixel circuit 100 being uniquely identifiable by a combination of row address i and column address j. The conversion portion 110 of the illustrated pixel circuit 100 includes a photoelectric conversion element PD, a charge accumulation structure FD, a transfer gate 101, a reset transistor 107, and an amplifier transistor 108.
73342 9 The photoelectric conversion element PD photoelectrically converts incident electromagnetic radiation into electric charges. The amount of electric charge generated in the photoelectric conversion element PD is a function of the intensity of the incident electromagnetic radiation. The photoelectric conversion element PD may include or consist of a photodiode with the photodiode anode electrically connected to a reference potential, wherein the photoelectric conversion element converts electromagnetic radiation incident on a detection surface into a photocurrent by means of the photoelectric effect. The electromagnetic radiation may include visible light, infrared radiation and/or ultraviolet radiation. The amplitude of the photocurrent corresponds to the intensity of the incident electromagnetic radiation, wherein in the intensity range of interest the photocurrent increases at least approximately linearly with increasing intensity of the detected electromagnetic radiation. The photocurrent changes the voltage at the photodiode cathode. For a pinned photodiode the photodiode cathode can be depleted to a positive reset voltage before an exposure period and the photocurrent gradually decreases the voltage at the photodiode cathode during the exposure period. The charge accumulation structure FD is suitable for accumulating charge and may include a capacitor electrode, a floating diffusion region or a combination of both. The reset transistor 107 initializes a charge in the charge accumulation structure FD in response to an active reset signal rst applied to a gate of the reset transistor 107. The reset signal rst is supplied to the gate of the reset transistor 107 through a reset control line. The reset signal rst changes between an active signal level (“active reset signal”) and an inactive signal level (“inactive reset signal”). In the illustrated embodiment, the active signal level is the high level. In response to an active reset signal rst, the reset transistor 107 connects the charge accumulation structure FD to a positive reset potential. The positive reset potential may be an analog supply potential VDDH of the pixel circuit 100. A load path of the transfer transistor 102 is electrically connected between a cathode of the photoelectric conversion element PD and the charge accumulation structure FD. The transfer transistor 102 serves as transfer element for transferring charge from the photoelectric conversion element PD to the charge accumulation structure FD in an integration period (exposure period). The charge accumulation structure FD serves as temporary local charge storage. A transfer signal tg is supplied to the gate (transfer gate) of the transfer transistor 102 through a transfer control line. The transfer signal tg changes between an active signal level (“active transfer signal”) and an inactive signal level (“inactive transfer signal”). In response to an active transfer signal tg, the transfer transistor 102 transfers electrons photoelectrically converted by the photoelectric conversion element 101 to the charge accumulation structure FD. In the illustrated embodiment, the active signal level is the high level.
73342 10 A controlled load path of the amplifier transistor 108 is electrically connected between the analog supply potential VDDH and a data node 18. The data node 18 forms a signal interface between a conversion portion 110 and a PWM portion of the pixel circuit 100. The charge accumulation structure FD is connected to the gate of the amplifier transistor 108. A potential at the gate of the amplifier transistor 108 is equal to the potential of the charge accumulation structure FD. As shown on the right side of FIG. 2, the analog compute modules 200 of a signal processing array 20 may be arranged to form another two-dimensional array with columns and rows, each analog compute modules 200 being uniquely identifiable by a combination of row address i and column address j. In the illustrated example, the number of analog compute modules 200 and the number of pixel circuits 100 are equal. Some or all of the elements of the image sensor array 10 can be formed in a first semiconductor die (top die 910). The signal processing array 20 and some of the elements of the image sensor array 10 can be formed in a second semiconductor die (bottom die 920). The top die 910 can be stacked on top of the bottom die 920 using a hybrid bonding technique. The pixel circuit 100 with the row address i and the column address j may include a first main pad vij-1. The analog compute module 200 with the row address i and the column address j may include a second main pad vij-2. The first main pad vij-1 and the second main pad vij-2 may be directly bonded and electrically connected to each other and form the data node 18. The same holds for all other pixel circuits 100 and analog compute modules 200. In the illustrated example, the conversion portion 110 of the pixel circuit 100 outputs the pixel voltage signal via the first main pad vij1. The PWM portion 130 and the analog compute module 200 receive the pixel voltage signal via the second main pad vij2. In other examples, the pixel circuit 100 may output the pixel signal pulse via the first main pad vij1 and the analog compute module 200 receives the pixel signal pulse via the second main pad vij2. An analog compute module 200 computes a convolution of the neighborhood of the main pixel according to an analog convolution function f(x). The convolution involves the pixel signal pulses received via the second main pad vij2 or derived from pixel voltage signals received via the second main pad vij2, and signals derived from pixel signal pulses received via other main pads. Each analog compute module 200 outputs the computed pulse or a signal derived from the computed pulse to a peripheral processing unit 40. A sensor controller 50 provides timing signals for the control signals for the image sensor array 10 and the analog compute modules 200. A pixel driver unit 30 receives timings signals from the sensor controller 50 and outputs pixel control signals to control the pixel circuits 100.
73342 11 The peripheral processing unit 40, the pixel driver unit 30 and the sensor controller 50 can be placed at the periphery of the bottom die 920 or formed in a third die, wherein the third die can be connected to the bottom of the bottom die 920. According to an embodiment, the analog compute modules 200 may include programmable current sources 210 configured to apply the separate weighting coefficients to the pixel signal pulse to obtain the weighted pixel signal pulses for the other pixel circuits 100 of the pixel kernel 15. Each analog compute module 200 includes at least one programmable current source 210 for each pixel signal pulse output to any of the other pixel circuits 100. In other words, each analog compute module 200 can include one or more programmable current source 210 for each satellite pixel. Different weighting coefficients are realized by different currents through the programmable current sources for the same pixel signal pulse. According to an embodiment, each of the programmable current sources 210 may include a charge trapping transistor 211, different weighting coefficients correspond to different amounts of charge trapped in a gate dielectric of the charge trapping transistors 211, and saturation currents of the charge trapping transistors 211 are a function of the charge trapped in the gate dielectric. FIG. 3A shows an example for a programmable current source 210 based on a charge trapping transistor (CTT) 211. A gate of the CTT 211 receives a 3-level gate signal VG. The source of the CTT 211 is connected to a 2-state source signal VL. An output voltage VD depends on the CTT drain current ID and can be tapped from the drain of the CTT 211. The CTT 211 is in a read mode when a voltage difference VG - VL between the gate and the source is in a read voltage window between a negative lower program voltage and a positive upper program voltage. For a gate voltage VG within the read voltage window and exceeding the transistor threshold voltage Vth, the CTT 211 turns on. For a given gate voltage VG above the transistor threshold voltage Vth, the collector current ID depends on the charge trapped in the CTT gate dielectric. According to an embodiment, each of the programmable current sources 210 may include a control circuit 212 configured to program the charge trapped in the gate dielectric of the charge trapping transistor 211. The amount of trapped charge can be altered in a write mode and in an erase mode. The CTT 211 is in the write mode, when for the duration of a gate programming pulse a drain-to-source voltage VDS and a gate- to-source voltage VGS are sufficiently high to facilitate channel hot electron injection from the transistor channel into the gate dielectric. The write operation increases a negative charge or decreases a positive
73342 12 charge in the gate dielectric. The CTT 211 is in the erase mode, when for the duration of a gate programming pulse a gate-to-source voltage VGS is sufficiently negative and a drain-to-source voltage VDS is sufficiently low to facilitate injection of electrons from the gate dielectric into the transistor channel and/or injection of holes from the transistor channel into the gate dielectric. The erase operation increases a positive charge or decreases a negative charge in the gate dielectric. In both cases the final charge in the gate dielectric is a function of the respective voltage differences and the duration of the gate programming pulses. The control circuit 212 illustrated in FIG. 3A and FIG. 3B includes a CMOS inverter 212 with the load paths of a pFET and an nFET electrically connected in series between a switched voltage VAk and GND or a reference potential VSS in FIG. 3B. The output of the CMOS inverter 212 between load paths of the pFET and the nFET is electrically connected to the gate of the CTT 211. The switched voltage VAk may change between a write voltage higher than a maximum gate read voltage, a negative erase voltage lower than the reference potential VSS, and a positive gate read voltage. The write voltage may be greater than 1.5V. The erase voltage may be lower than -0.4V. The 2-state source signal VL may be floating for the gate programming pulse for erase and may have the reference potential VSS in the rest, in particular, for the read pulse and the gate programming pulse for write. According to an embodiment, the programmable current source 210 may include at least two charge trapping transistors 211-1, 211-2 in separate, parallel current paths, and further includes a selection circuit 218-1, 218-2 configured to selectively enable the parallel current paths. In FIG. 3B the selection circuit includes select transistors 218-1, 218-2, … in the parallel current paths. A first select transistor 218-1 is electrically connected in series with a first charge trapping transistor 211-1. An active first select signal sa turns on the first select transistor 218-1. A second select transistor 218-2 is electrically connected in series with a second charge trapping transistor 211-2. An active second select signal sb turns on the second select transistor 218-2. Further select transistors 218-x can be electrically connected in series with further charge trapping transistor 211-x. Active further select signals sx turn on the further select transistors 218-x, respectively. A common control circuit 212 can be controlled to change the charge trapped in the gate dielectric of the charge trapping transistors 211-1, 211-2 selected for writing or erasing. According to some examples, only one of the charge trapping transistors 211-1, 211-2 is selected at a time. According to other examples, more than one or all of the charge trapping transistors 211-1, 211- 2, … can be selected simultaneously. The following embodiments are described with reference to a 3x3 pixel kernel with a total of nine neighboring pixel circuits 100 arranged in three neighboring columns and three neighboring rows.
73342 13 FIG. 4 shows the numbering used to identify the individual pixel circuits 100-0 to 100-8 of the 3x3 pixel kernel 15 of FIG. 1B. Where appropriate, the combination of pixel circuit 100-4 and the corresponding analog compute module is referred to as “main pixel” and the other combinations of pixel circuit and corresponding analog compute module are referred to as “satellite pixels” of the pixel kernel 15 for better readability. FIG. 5 shows the connections between the main pixel (pixel 4) and each of the satellite pixels of the same pixel kernel 15. The main pixel outputs eight weighted pixel output pulses O0 to O3 and O5 to O8 derived from the radiation incident on the main pixel, one weighted pixel output pulse to each of the eight satellite pixels. The main pixel receives eight weighted pixel input pulses I0 to I3 and I5 to I8 derived from the radiation incident on the satellite pixels, one weighted pixel input pulse from each of the eight satellite pixels. The main pixel computes a computed pulse having a computed pulse width from the eight weighted input pixel signal pulses I0 to I3 and I5 to I8 received from the satellite pixels and the pixel signal pulse generated by the pixel circuit 100 of the main pixel. FIG. 6 schematically shows the signal flow from radiation detected by the photoelectric conversion device PD of a pixel circuit 100 through an analog computing module 200 to a peripheral processing unit 40 for high-level signal processing. The pixel circuit 100 includes a conversion portion 110 and a PWM portion 130. The conversion portion 110 includes the photoelectric conversion device PD and the pixel transistors to convert the radiation received during an exposure period into an pixel voltage signal, wherein a voltage level of the pixel voltage signal is a function of the radiation intensity accumulated in the exposure period. The pixel circuit 100 is formed in a top die 910. The PWM portion 130 converts the pixel voltage signal into a pixel signal pulse Dp. The pulse width tk of the pixel signal pulse Dp is a function of the voltage level of the pixel voltage signal, wherein the higher the received radiation intensity, the longer the pulse width is. In the illustrated example, the PWM portion 130 includes a comparator that receives the pixel voltage signal at the non-inverting input and a ramp signal VRAMP at the inverting input. At the end of an exposure, the ramp signal VRAMP starts at a high voltage level and linearly decreases with time. When the ramp signal VRAMP falls below the pixel voltage signal, the comparator output signal changes from a high level to a low level. The change to the low level terminates the pixel signal pulse Dp. In the illustrated example, the PWM portion 130 is formed on the bottom die 920. In other examples, the PWM portion 130 or parts of the PWM portion 130 may be formed on the top die 910.
73342 14 The analog compute module 200 receives the pixel signal pulse Dp from the pixel circuit 100. Eight programmable current sources 210 with eight different weighting coefficients output eight differently weighted pixel output pulses O0 to O3 and O5 to O8. The weighting coefficients are programmable gains of a unit current I0 supplied through the programmable current sources 210. According to an embodiment, the analog compute modules 200 of the image sensor assembly may include collecting circuits 250. Each collecting circuit 250 is configured to generate a sum signal, wherein the sum signal depends on the weights of the weighted pixel signal pulses Dp received from the other analog compute modules 200 and the weighted pixel signal pulse Dp from the first one of the pixel circuits 100 of the pixel kernel 15. In addition, the shape of the sum signal depends on the signal pulse widths of the weighted pixel signal pulses Dp received from the other analog compute modules 200 and on the signal pulse width of the weighted pixel signal pulse Dp from the first one of the pixel circuits 100 of the pixel kernel 15. A current supplied from the collecting circuit 250 to a signal converter 260 is a function of time of the pulse widths (light level in each pixel) and the individual current levels (weights for each pixel). The total charge is the result of a mixed multiply/accumulate operation. According to an embodiment, each collecting circuit 250 may include a programmable main charge trapping transistor 255 configured to change an amplitude of the pixel signal pulse Dp from the first one of the pixel circuits 100. In the illustrated example, the collecting circuit 250 includes the programmable main charge trapping transistor 255 and a switch 252 for enabling a current flow through the programmable main charge trapping transistor 255 for the signal pulse width tk. The switch 252 is electrically connected between a summation node for the pixel input pulses and the programmable main charge trapping transistor 255. In the rest, the switch 252 and the programmable main charge trapping transistor 255 can have the same configuration as a programmable current source 210 described above. The programmable auxiliary current source 255 supplies a current w4 * I0. The switch 252 is controlled by the pixel signal pulse Dp and is on for the signal pulse width tk. When the switch 252 is on, each of the pixel input pulses I0 to I3 and I5 to I8 and the pixel signal pulse Dp of the main pixel contributes to a potential of the summation node, wherein the pixel signal pulse Dp of the main pixel is weighted with the weighting coefficient w4. The collecting circuit 250 samples a total current over time that includes one portion I0*w4*tk4 from the main pixel and eight portions I0*wx*tkx from each of the eight satellite pixels x, with x running from 0 to 3 and from 5 to 8. In the convolution, I0*w4*tk4 is part of
73342 15 the sum signal. The switch 252 and corresponding switches in the other pixels of the pixel kernel 15 are used to make the total amount of charge transferred to the ramp capacitor 263 proportional to the light intensity. Each pixel computes a convolution of its neighborhood as indicated in FIG. 4 and FIG. 5. Considering only pixel 4 and pixel 1 for illustrative purpose, pixel 1 receives the weighted output signal pulse O1 of pixel 1, which is a weighted current flowing to ground and whose pulse length is proportional to the pixel 4 light intensity. Pixel 4 receives the weighted input signal pulse I1 which corresponds to the weighted output signal pulse O7 of pixel 1. The weighted output signal pulse O7 of pixel 1 is a weighted current flowing to ground and proportional to the pixel 1 light intensity. The analog computing module 200 further includes a signal converter 260 converting the sum signal to the computed pulse. The signal converter 260 may include a dual-slope converter. The signal converter 260 may include a multiplexer 261, a ramp capacitor 263, and a comparator circuit 268. According to an embodiment, a ramp capacitor 263 may be configured to be discharged with a ramp current IRMP, the ramp current IRMP being a function of the weights of the weighted pixel signal pulses Dp received from the other analog compute modules 200 and the weighted pixel signal pulses Dp from the first one of the pixel circuits 100 of the pixel kernel 15. The ramp capacitor 263 can be alternatingly charged through a temporary connection to positive potential and discharged by the ramp current IRMP, wherein the ramp current IRMP is derived from the sum signal obtained from the collecting circuit 250. The multiplexer 261 may be controlled to connect a charging electrode of the ramp capacitor 263 with an auxiliary supply potential VDDM in precharge periods and with the summation node of the collecting circuit 250 in conversion periods. According to an embodiment, each analog computing module 200 may further include a comparator circuit 268 configured to output the computed pulse, wherein the computed pulse width of the computed pulse is a function of a slope of the ramp current IRMP. The comparator circuit 268 receives the voltage across the ramp capacitor 268 at a first input and a reference voltage V0 at a second input. The first input may be the non-inverting input and the second input may be the inverting input. When the voltage across the ramp capacitor 263 exceeds or falls below the reference voltage V0, the comparator output signal changes from an active voltage level to an inactive voltage level, for example, from the logic high level to the logic low level, and the computed pulse is terminated. The voltage change at the ramp capacitor 263 is proportional to the convolution output, which is the sum of
73342 16 weighted pixel values. The reference current IREF determines the pulse length tp4 this voltage change corresponds to. According to an embodiment, a spatial filter circuit 290 may be configured to output a filtered pulse with a filtered pulse width, the filtered pulse width being a function of the computed pulse widths output from at least two pixel circuits 100 of the pixel kernel 15. In the illustrated example, the spatial filter circuit 290 is a NOR gate with four inputs D1, D2, D4, D5 from pixels 1, 2, 4, and 5 of FIG. 4. The inputs of the NOR gate receive the computed pulses of four analog compute modules 200 assigned to a 2x2 pixel field within the 3x3 pixel kernel 15. According to an embodiment, the image sensor assembly may further include a gate circuit 410 configured to gate the computed pulse or the filtered pulse with a global gating pulse VGG indicating a saturating exposure. The gate circuit 410 may be used for exposure control. For example, if the output of the gate circuit 410 is still high when the global gating pulse VGG is applied it means that the signal output from the analog computing module 200 is saturated, since the signal is proportional to the pulse length. Looking at all the outputs of all gate circuits 410, this information can be used to adjust the exposure time to make the signals large but not saturated. The gate circuit 410 may output a gated pulse having a rising edge delayed with respect to a rising edge of the computed pulse width. The gate circuit 410 may include an AND gate. A first input of the AND gate receives the computed pulse output from the comparator circuit 268 or, if applicable, the filtered pulse output from the spatial filter circuit 290. A second input of the AND gate receives the global gating pulse VGG. An endpoint of the global gating pulse VGG indicates a boundary pulse width indicating a transition from a non-saturated operation to a saturated operation of the pixel circuit. The gate circuit 410 may be integral part of a rectified linear unit (ReLU) 41. According to an embodiment, the image sensor assembly 70 may further include an artificial neural network 49 configured to receive and process signals derived from the computed pulses. The artificial neural network 49 applies additional filtering to get high-level details, and then additional fully connected layers for the classification. In FIG. 7A, the amplifier transistor 108 of the conversion portion 110 of the pixel circuit 100 is used to feed in the ramp signal VRAMP for the pulse width modulation. The amplifier transistor 108 receives the ramp signal VRAMP at the source. A load path of a first pFET p1 controlled by a second reset signal rst2
73342 17 is connected between the logic supply potential VDDL and the drain of the amplifier transistor 108. The PWM portion 130 of the pixel circuit 100 further includes a second pFET p2 and a second nFET n2 electrically connected between the logic supply potential VDDL and a negative potential -VP. The gate of the second pFET p2 receives the signal tapped from the drain of the amplifier transistor 108. The gate of the second nFET n2 receives a first reset signal rst1. The second pFET p2 forms a dynamic comparator generating a pixel signal pulse Dp with a signal pulse width t4 being a function of the potential of the charge accumulation structure. The second pFET p2 is operated as comparator amplifier. The first and second reset signals rst1 and rst2 are used to reset the dynamic comparator. The voltage VSR at the gate of the second pFET p2 starts at the positive supply potential resulting in an extremely small drain current for the second pFET p2 in the order of fA to pA. As VSR is discharged, the drain current for the second pFET p2 increases exponentially, and at some point in time it becomes large enough to charge the pixel signal pulse Dp to a high voltage, and the comparator flips. Each programmable current sources 210, 255 of the analog compute module 200 includes one convolution kernel. The control circuit 212, 252 of each programmable current source 210, 255 includes a CMOS inverter with a third pFET p3 and a third nFET n3 electrically connected in series between the logic supply potential VDDL and the reference potential VSS. The CMOS inverter receives the pixel signal pulse Dp at the gate of the third nFET n3 and the inverted pixel signal pulse at the gate of the third pFET p3. The signal converter 260 includes a fourth nFET n4 in a common gate amplifier configuration. The summation node of the collecting circuit 250 is electrically connected to the source of the fourth nFET n4. The drain of the fourth nFET n4 is electrically connected to the charging of the ramp capacitor 263. A gate of the fourth nFET n4 receives a constant gate bias voltage VC. The comparator circuit 268 includes the ramp capacitor 263 (C1), a storage capacitor 264 (C2), a fifth nFET n5, a first constant current source supplying the reference current IREF, a second constant current source supplying an auxiliary current IB and four comparator switches. A charging electrode of the storage capacitor C2 is connected to the drain of the fourth nFET n4 and the charging electrode of the ramp capacitor C1. The first current source supplying the reference current IREF and the first comparator switch are electrically connected in series between an auxiliary supply potential VDDM and the drain of the fourth nFET n4. An output enable signal oe controls the first comparator switch. The second current source supplying the auxiliary current IB and the second comparator switch are electrically connected between the auxiliary supply potential VDDM and an output node of the compactor circuit 268. A signal pon controls the second comparator switch. The load path of the fifth nFET n5 is electrically connected between the output node and the reference potential VSS. The gate of the fifth nFET n5 receives the potential of the counter electrode of the storage capacitor C2. A third comparator switch controlled by an autozero signal az is electrically connected between the drain and the gate of the fifth nFET n5. A fourth comparator switch
73342 18 controlled by a third reset signal rst3 is electrically connected between the auxiliary supply potential VDDM and the charging electrode of the ramp capacitor C1 and the storage capacitor C2. The fifth comparator switch can be used to clamp the node VQ to the auxiliary supply potential VDDM to quickly get to a known and well-defined starting point. The spatial filter circuit 290 includes a pixel portion of a NOR gate distributed across a number of analog compute modules equal to the number of input signals of the NOR gate. FIG. 7B shows a part of the NOR gate for four analog compute modules. The drains of a sixth nFET of all four analog compute modules are connected to DI. Only one of the four analog compute modules has the node sb connected to DI. The signal converter 260 can be controlled for computing a noise-compensated computed pulse on the basis of a pixel signal pulse of the illuminated pixel and a pixel reset pulse obtained from the non-illuminated pixel in close temporal proximity with the pixel signal pulse. According to an embodiment, each pixel circuit 100 may be controllable to successively output a pixel reset pulse and the pixel signal pulse Dp, wherein a reset pulse width of the pixel reset pulse is a function of a reset potential of a floating diffusion region of the pixel circuit 100; and wherein the analog compute module 200 includes a signal converter 260 configured to obtain the computed pulse with a computed pulse width being a function of a difference between the signal pulse width and the reset pulse width. The reset potential of the floating diffusion is the potential of the charge accumulation structure FD, which is referred to as the floating diffusion in the following, after reset and before receiving charges generated by incident radiation. After exposure, the potential of the floating diffusion is a function of the received radiation intensity. According to an example, the ramp capacitor C1 and the storage capacitor C2 of the signal converter 260 of FIG. 7A are successively charged to store charges proportional to the reset pulse width tR and the signal pulse width tS, and simultaneously discharged in a way that a pulse width of the output pulse of the signal converter is a function of the difference between the signal pulse width tS and the reset pulse width tR. The voltage VQ at the charging electrodes of the ramp capacitor C1 and the storage capacitor C2 after the first rising edge of DP is set as the zero voltage for the comparator circuit 268 including the fifth nFET n5 and the second constant current source supplying the auxiliary current IB. The flipping point of the comparator output signal produces the falling edge of the computed pulse after tp. Thus, the computed pulse width tP4 is proportional to tS – tR.
73342 19 FIG. 7C shows a corresponding time chart of the control signals. Equation #1 indicates the dependence of the computed pulse width tP4 from the weighting coefficients and the signal pulse widths detected in the pixel kernel.
Equation #2 indicates the effect of the spatial filtering. The index n counts from 1 to the number of pixels of the pixel field used for spatial filtering. #2 ^^ = ^^^ (^^^), FIG. 8A integrates the multiple convolution kernel of FIG. 3B in the analog compute module 200 of FIG. 7A. According to an embodiment, each of the analog compute modules 200 may include programmable current sources 210 configured to output the weighted pixel signal pulse according to the separate weighting coefficients, wherein each programmable current source 210 includes at least two charge trapping transistors 211-1, 211-2 in parallel current paths and a selection circuit 218-1, 218-2 configured to turn on and off the parallel current paths, and wherein different weighting coefficients correspond to different amounts of charge trapped in a gate dielectric of the charge trapping transistor 211-1, 211-2 in the selected current path. Each current path with a charge trapping transistor 211, 211-2 represents one convolution kernel. The pixel readout can use different convolution kernels allowing a fine adaption of the image sensor assembly to a given application. A selection circuit can include an nFET between the CTT and the reference potential. FIG. 8B shows a signal controller 50 controlling a pixel driver circuit 30 to output the control signals for the pixel circuit 100 and the analog compute module 200 of FIG. 8A to operate as described with reference to the following figures. According to an embodiment, a sensor controller 50 may be configured to control each of the pixel circuits 100 to successively output a pixel reset pulse and the pixel signal pulse, wherein a reset pulse width of the pixel reset pulse is a function of a reset potential of a floating diffusion region of the pixel circuit 100, and wherein the sensor controller 50 is further configured to control the analog compute modules 200 to use the same charge trapping transistor 211-1, 211-2 successively for the pixel reset pulse and the pixel signal pulse for each exposure.
73342 20 The readout for the signal pulse directly follows the readout of the reset pulse of the same pixel circuit, wherein the reset pulse and the directly following signal pulse from the same pixel circuit use the same weight. This allows analog CDS within the analog compute modules for each single exposure. The sensor controller 50 can change the weights of the convolution kernels after each exposure. In analog compute modules 200 with two convolution kernels, the sensor controller 50 can strictly alternate between the two convolution kernels with each exposure. FIG. 9 shows a corresponding time chart of the control signals and some internal signals. Equation #3 indicates the dependence of the computed pulse width tP4 from the weighting coefficients of and the signal pulse widths detected in the pixel kernel.
Equation #4 indicates the effect of the spatial filtering. The index n counts from 1 to the number of pixels of the pixel field used for spatial filtering. #4 ^^ = ^^^ (^^^), According to an embodiment, a sensor controller 50 may be configured to control each of the pixel circuits 100 to successively output pixel reset pulses using different charge trapping transistors 211-1, 211-2, wherein a reset pulse width of the pixel reset pulse is a function of a reset potential of a floating diffusion region of the pixel circuit 100, and successively output, after one exposure, a same number of pixel signal pulses using the same charge trapping transistors 211-1, 211-2, and wherein the sensor controlerl 50 is configured to control the analog compute module 200 to obtain a computed reset pulse from each pixel reset pulse and a computed signal pulse from each pixel signal pulse. The readout for the reset pulse is repeated for each convolution kernel. The peripheral processing unit 40 may perform a sort of digital CDS by subtracting, for each convolution kernel separately, the reset pulse width from the signal pulse width obtained using the same convolution kernel. FIG. 10 shows a corresponding time chart of the control signals and some internal signals. Equations #5 and #6 indicate the dependences of the computed pulse width tPRx4 for the reset pulses and the computed pulse width tPSx4 for the signal pulses the from the weighting coefficients of and the signal pulse widths detected in the pixel kernel.
73342 21
Equations #7 and #8 indicate the effect of the spatial filtering: #7 ^^^^ = ^^^ (^^^^^ ) #8 ^^^^ = ^^^ (^^^^^ ) A noise-compensated computed pulse width tPA can be obtained from equation #9: #9 ^^^ = ^^^^ − ^^^^ According to an embodiment, a sensor controller 50 may be configured to control each of the pixel circuits 100 to successively output a pixel reset pulse with all charge trapping transistors 211-1, 211-2 being simultaneously selected, and the pixel signal pulse with all charge trapping transistors 211-1, 211-2 being simultaneously selected, wherein a reset pulse width of the pixel reset pulse is a function of a reset potential of a floating diffusion region of the pixel circuit 100. All convolution kernels are simultaneously used for the same exposure. The total weight of the reset pulse results from all weights of the charge trapping transistors 211, 211-2 assigned to the same pixel output. The total weight of the signal pulse results from the same weights used for the reset pulses. The peripheral processing unit 40 may subtract a digital value obtained from the sum reset pulse width by AD conversion from a digital value obtained from the sum signal by AD conversion. FIG. 11 shows a corresponding time chart of the control signals and some internal signals. Equations #10 and #11 indicate the dependences of the computed pulse width tPR4 for the reset pulses and the computed pulse width tPS4 for the signal pulses from the weighting coefficients and the signal pulse widths detected in the pixel kernel.
Equations #12 and #13 indicate the effect of the spatial filtering: #12 ^^^ = ^^^ (^^^^) #13 ^^^ = ^^^ (^^^^)
73342 22 A noise-compensated computed pulse width tPB can be obtained from equation #14: #14 ^^^ = ^^^ − ^^^ According to an embodiment, the image sensor assembly may further include a sensor controller 50 configured to control each of the pixel circuits 100 to successively output pixel reset pulses using different charge trapping transistors 211-1, 211-2, wherein a reset pulse width of the pixel reset pulse is a function of a reset potential of a floating diffusion region of the pixel circuit 100, and output, after one exposure, a same number of pixel signal pulses using the same charge trapping transistors 211-1, 211-2, and wherein the sensor controller 50 is configured to control the analog compute module 200 to obtain a sum reset pulse from the pixel reset pulses and a sum signal pulse from the pixel signal pulses. A sum reset pulse width of the sum reset pulse is a function, e.g., the sum of the reset pulse widths of the pixel reset pulses. A sum signal pulse width of the sum signal pulse is a function, e.g., the sum of the signal pulse widths of the pixel signal pulses. A signal converter 260 as described above can be controlled to obtain the sum reset pulse and the sum signal pulse. The peripheral processing unit 40 may subtract a digital value obtained from the sum reset pulse width by AD conversion from a digital value obtained from the sum signal pulse width by AD conversion. FIG. 12 shows a corresponding time chart of the control signals and some internal signals. The dependences of the computed pulse width tPR4 for the reset pulses and the computed pulse width tPS4 for the signal pulses from the weighting coefficients and the signal pulse widths detected in the pixel kernel are given in equations #10 and #11. The effect of filtering is given by equations #12 and #13. The noise-compensated computed pulse width tPB can be obtained from equation #14. According to an embodiment, each pixel circuit 100 may be configured to generate a pixel signal pulse having a signal pulse width being a function of both a radiation intensity incident on the pixel circuit 100 and a reset potential of a floating diffusion region of the pixel circuit 100, and the image sensor assembly may further include a sensor controller 50 configured to control each of the pixel circuits 100 to successively output pixel signal pulses using different charge trapping transistors 211-1, 211-2 after one exposure. The signal pulse width may be a function of a difference between potentials on the floating diffusion after and before exposure. In FIG. 13, the pixel circuit 100 receives the ramp signal VRAMP at the counter electrode of the charge accumulation structure FD. The charge accumulation structure FD is connected to a first electrode of a comparator capacitor C7. A second electrode of the comparator capacitor C7 I electrically connected to the
73342 23 gate of the amplifier transistor 108. A load path of a feedback transistor 106 is connected between the drain and gate of the amplifier transistor 108. A feedback signal fb is applied to the gate of the feedback transistor 106. FIG. 14 shows a corresponding time chart of the control signals and some internal signals. Equation #15 indicates the dependence of the computed pulse width tPRx4 from the weighting coefficients of and the signal pulse widths detected in the pixel kernel.
Equation #16 indicates the effect of the spatial filtering. The index n counts from 1 to the number of pixels of the pixel field used for spatial filtering. #16 ^^^^ = ^^^ (^^^^^) According to an embodiment illustrated in FIG. 15, the image sensor assembly may further include a bypass multiplexer 295 configured to output the pixel signal pulse in a first operation mode and the computed pulse in a second operation mode. FIG. 16 is a diagram illustrating an example in which the image sensor assembly 70 of FIG. 1A is formed by a stacked CMOS image sensor (CIS) having a two-layer structure with a top die 910 (radiation receiving chip) and a bottom die 920 (processing chip). The radiation receiving chip includes at least the photoelectric conversion elements. For example, the radiation receiving chip may include only the photoelectric conversion elements, only the conversion portions of the pixel circuits, or the conversion portions and at least some elements of the PWM portions of the pixel circuits. The image sensor assembly is formed as one sensor by bonding the radiation receiving chip and the processing chip while electrically bringing contact pads on the radiation receiving chip in contact with corresponding contact pads on the processing chip. FIG.17 is a block diagram depicting an example of schematic configuration of a vehicle control system as an example of a system to which the technology according to an embodiment of the present disclosure can be applied. The vehicle control system 12000 includes a plurality of electronic control units connected to each other via a communication network 12001. In the example depicted in FIG.17, the vehicle control system 12000 includes a driving system control unit 12010, a body system control unit 12020, an outside-vehicle information detecting unit 12030, an in-vehicle information detecting unit 12040, and an integrated control unit 12050. In addition, a microcomputer 12051, a sound/image output section 12052, and a vehicle-
73342 24 mounted network interface 12053 are illustrated as a functional configuration of the integrated control unit 12050. The driving system control unit 12010 controls the operation of devices related to the driving system of the vehicle in accordance with various kinds of programs. For example, the driving system control unit 12010 functions as a control device for a driving force generating device for generating the driving force of the vehicle, such as an internal combustion engine, a driving motor, or the like, a driving force transmitting mechanism for transmitting the driving force to wheels, a steering mechanism for adjusting the steering angle of the vehicle, a braking device for generating the braking force of the vehicle, and the like. The body system control unit 12020 controls the operation of various kinds of devices provided to a vehicle body in accordance with various kinds of programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various kinds of lamps such as a headlamp, a backup lamp, a brake lamp, a turn signal, a fog lamp, or the like. In this case, radio waves transmitted from a mobile device as an alternative to a key or signals of various kinds of switches can be input to the body system control unit 12020. The body system control unit 12020 receives these input radio waves or signals, and controls a door lock device, the power window device, the lamps, or the like of the vehicle. The outside-vehicle information detecting unit 12030 detects information about the outside of the vehicle including the vehicle control system 12000. The outside-vehicle information detecting unit 12030 can be connected with an imaging section 12031. The outside-vehicle information detecting unit 12030 makes the imaging section 12031 imaging an image of the outside of the vehicle and receives the imaged image. Based on the received image, the outside-vehicle information detecting unit 12030 may perform processing of detecting an object such as a human, a vehicle, an obstacle, a sign, a character on a road surface, or the like, or processing of detecting a distance thereto. The imaging section 12031 may be or may include an image sensor assembly according to the embodiments of the present disclosure. The light received by the imaging section 12031 may contain visible light and/or invisible light such as infrared rays or the like. The in-vehicle information detecting unit 12040 detects information about the inside of the vehicle and may be or may include an image sensor assembly according to the embodiments of the present disclosure. The in-vehicle information detecting unit 12040 is, for example, connected with a driver state detecting section 12041 that detects the state of a driver. The driver state detecting section 12041, for example, includes a camera that includes the solid-stage imaging device and that is focused on the driver. Based on detection information input from the driver state detecting section 12041, the in-vehicle information detecting unit
73342 25 12040 may calculate a degree of fatigue of the driver or a degree of concentration of the driver or may determine whether the driver is dozing. The microcomputer 12051 can calculate a control target value for the driving force generating device, the steering mechanism, or the braking device based on the information about the inside or outside of the vehicle which information is obtained by the outside-vehicle information detecting unit 12030 or the in- vehicle information detecting unit 12040 and output a control command to the driving system control unit 12010. For example, the microcomputer 12051 can perform cooperative control intended to implement functions of an advanced driver assistance system (ADAS) which functions include collision avoidance or shock mitigation for the vehicle, following driving based on a following distance, vehicle speed maintaining driving, a warning of collision of the vehicle, a warning of deviation of the vehicle from a lane, or the like. In addition, the microcomputer 12051 can perform cooperative control intended for automatic driving, which makes the vehicle to travel autonomously without depending on the operation of the driver, or the like, by controlling the driving force generating device, the steering mechanism, the braking device, or the like on the basis of the information about the outside or inside of the vehicle which information is obtained by the outside-vehicle information detecting unit 12030 or the in-vehicle information detecting unit 12040. In addition, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information about the outside of the vehicle which information is obtained by the outside- vehicle information detecting unit 12030. For example, the microcomputer 12051 can perform cooperative control intended to prevent a glare by controlling the headlamp so as to change from a high beam to a low beam, for example, in accordance with the position of a preceding vehicle or an oncoming vehicle detected by the outside-vehicle information detecting unit 12030. The sound/image output section 12052 transmits an output signal of at least one of a sound or an image to an output device capable of visually or audible notifying information to an occupant of the vehicle or the outside of the vehicle. In the example of FIG. 17, an audio speaker 12061, a display section 12062, and an instrument panel 12063 are illustrated as the output device. The display section 12062 may, for example, include at least one of an on-board display or a head-up display. FIG. 18 is a diagram depicting an example of the installation position of the imaging section 12031, wherein the imaging section 12031 may include imaging sections 12101, 12102, 12103, 12104, and 12105. The imaging sections 12101, 12102, 12103, 12104, and 12105 are, for example, disposed at positions on a front nose, side-view mirrors, a rear bumper, and a back door of the vehicle 12100 as well as a position on an upper portion of a windshield within the interior of the vehicle. The imaging section 12101 provided to
73342 26 the front nose and the imaging section 12105 provided to the upper portion of the windshield within the interior of the vehicle obtain mainly an image of the front of the vehicle 12100. The imaging sections 12102 and 12103 provided to the side view mirrors obtain mainly an image of the sides of the vehicle 12100. The imaging section 12104 provided to the rear bumper or the back door obtains mainly an image of the rear of the vehicle 12100. The imaging section 12105 provided to the upper portion of the windshield within the interior of the vehicle is used mainly to detect a preceding vehicle, a pedestrian, an obstacle, a signal, a traffic sign, a lane, or the like. Incidentally, FIG. 18 depicts an example of photographing ranges of the imaging sections 12101 to 12104. An imaging range 12111 represents the imaging range of the imaging section 12101 provided to the front nose. Imaging ranges 12112 and 12113 respectively represent the imaging ranges of the imaging sections 12102 and 12103 provided to the side view mirrors. An imaging range 12114 represents the imaging range of the imaging section 12104 provided to the rear bumper or the back door. A bird's-eye image of the vehicle 12100 as viewed from above is obtained by superimposing image data imaged by the imaging sections 12101 to 12104, for example. At least one of the imaging sections 12101 to 12104 may have a function of obtaining distance information. For example, at least one of the imaging sections 12101 to 12104 may be a stereo camera constituted of a plurality of imaging elements, imaging element having pixels for phase difference detection or may include a ToF module including an image sensor assembly according to the embodiments of the present disclosure. For example, the microcomputer 12051 can determine a distance to each three-dimensional object within the imaging ranges 12111 to 12114 and a temporal change in the distance (relative speed with respect to the vehicle 12100 on the basis of the distance information obtained from the imaging sections 12101 to 12104, and thereby extract, as a preceding vehicle, a nearest three-dimensional object in particular that is present on a traveling path of the vehicle 12100 and which travels in substantially the same direction as the vehicle 12100 at a predetermined speed (for example, equal to or more than 0 km/hour). Further, the microcomputer 12051 can set a following distance to be maintained in front of a preceding vehicle in advance and perform automatic brake control (including following stop control), automatic acceleration control (including following start control), or the like. It is thus possible to perform cooperative control intended for automatic driving that makes the vehicle travel autonomously without depending on the operation of the driver or the like. For example, the microcomputer 12051 can classify three-dimensional object data on three-dimensional objects into three-dimensional object data of a two-wheeled vehicle, a standard-sized vehicle, a large-sized vehicle, a pedestrian, a utility pole, and other three-dimensional objects on the basis of the distance information obtained from the imaging sections 12101 to 12104, extract the classified three-dimensional
73342 27 object data, and use the extracted three-dimensional object data for automatic avoidance of an obstacle. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 as obstacles that the driver of the vehicle 12100 can recognize visually and obstacles that are difficult for the driver of the vehicle 12100 to recognize visually. Then, the microcomputer 12051 determines a collision risk indicating a risk of collision with each obstacle. In a situation in which the collision risk is equal to or higher than a set value and there is thus a possibility of collision, the microcomputer 12051 outputs a warning to the driver via the audio speaker 12061 or the display section 12062 and performs forced deceleration or avoidance steering via the driving system control unit 12010. The microcomputer 12051 can thereby assist in driving to avoid collision. At least one of the imaging sections 12101 to 12104 may be an infrared camera that detects infrared rays. The microcomputer 12051 can, for example, recognize a pedestrian by determining whether there is a pedestrian in imaged images of the imaging sections 12101 to 12104. Such recognition of a pedestrian is, for example, performed by a procedure of extracting characteristic points in the imaged images of the imaging sections 12101 to 12104 as infrared cameras and a procedure of determining whether it is the pedestrian by performing pattern matching processing on a series of characteristic points representing the contour of the object. When the microcomputer 12051 determines that there is a pedestrian in the imaged images of the imaging sections 12101 to 12104, and thus recognizes the pedestrian, the sound/image output section 12052 controls the display section 12062 so that a square contour line for emphasis is displayed so as to be superimposed on the recognized pedestrian. The sound/image output section 12052 may also control the display section 12062 so that an icon or the like representing the pedestrian is displayed at a desired position. The example of the vehicle control system to which the technology according to an embodiment of the present disclosure is applicable has been described above. By applying an image sensor assembly according to the embodiments of the present disclosure, better sensitivity can be achieved for different pixels. Flickering traffic lights can be safely detected. Additionally, embodiments of the present technology are not limited to the above-described embodiments, but various changes can be made within the scope of the present technology without departing from the gist of the present technology. The image sensor assembly according to the present disclosure may be any device used for analyzing and/or processing radiation such as visible light, infrared light, ultraviolet light, and X-rays. For example, an image sensor assembly according to the embodiments may be any electronic device in the field of traffic, the field of home appliances, the field of medical and healthcare, the field of security, the field of beauty, the field of sports, the field of agriculture, the field of image reproduction or the like.
73342 28 Specifically, in the field of image reproduction, the image sensor assembly according to the embodiments may be a device for capturing an image to be provided for appreciation, such as a digital camera, a smart phone, or a mobile phone device having a camera function. In the field of traffic, for example, a solid-state imaging device including an image sensor assembly according to the embodiments may be integrated in an in-vehicle sensor that captures the front, rear, peripheries, an interior of the vehicle, etc. for safe driving such as automatic stop, recognition of a state of a driver, or the like, in a monitoring camera that monitors traveling vehicles and roads, or in a distance measuring sensor that measures a distance between vehicles or the like. In the field of home appliances, the image sensor assembly according to the embodiments may be integrated in any type of sensor that can be used in devices provided for home appliances such as TV receivers, refrigerators, and air conditioners to capture gestures of users and perform device operations according to the gestures. Accordingly, the image sensor assembly according to the embodiments may be integrated in home appliances such as TV receivers, refrigerators, and air conditioners and/or in devices controlling the home appliances. Furthermore, in the field of medical and healthcare, the image sensor assembly according to the embodiments may be integrated in any type of sensor, e.g., a solid-state image device, provided for use in medical and healthcare, such as an endoscope or a device that performs angiography by receiving infrared light. In the field of security, the image sensor assembly according to the embodiments can be integrated in a device provided for use in security, such as a monitoring camera for crime prevention or a camera for person authentication use. Furthermore, in the field of beauty, an image sensor assembly according to the embodiments can be used in a device provided for use in beauty, such as a skin measuring instrument that captures skin or a microscope that captures a probe. In the field of sports, an image sensor assembly according to the embodiments can be integrated in a device provided for use in sports, such as an action camera or a wearable camera for sport use or the like. Furthermore, in the field of agriculture, the image sensor assembly can be used in a device provided for use in agriculture, such as a camera for monitoring the condition of fields and crops. The present technology can also be configured as described below: [1] An image sensor assembly (70), including: an image sensor array (10) including a pixel kernel (15), the pixel kernel (15) including pixel circuits (100), each pixel circuit (100) being configured to generate a pixel signal pulse having a signal pulse width, the signal pulse width being a function of radiation intensity incident on the pixel circuit (100); and a plurality of analog compute modules (200), each analog compute module (200) configured to receive the pixel signal pulse from a first one of the pixel circuits (100) of the
73342 29 pixel kernel (15), output weighted pixel signal pulses to each of the other pixel circuits (100) of the pixel kernel (15) with separate weighting coefficients, receive the weighted pixel signal pulses from the other analog compute modules (200) assigned to the pixel kernel (15), and generate a computed pulse having a computed pulse width, the computed pulse width being a function of the signal pulse widths of the received weighted pixel signal pulses and the weighting coefficients applied to the received weighted pixel signal pulses. [2] The image sensor assembly according to [1], wherein the analog compute modules (200) include programmable current sources (210) configured to apply the separate weighting coefficients to the pixel signal pulse to obtain the weighted pixel signal pulses for the other pixel circuits (100) of the pixel kernel (15). [3] The image sensor assembly according to [2], wherein each of the programmable current sources (210) includes a charge trapping transistor (211), different weighting coefficients correspond to different amounts of charge trapped in a gate dielectric of the charge trapping transistors (211), and saturation currents of the charge trapping transistors (211) are a function of the charge trapped in the gate dielectric. [4] The image sensor assembly according to [3], wherein each of the programmable current sources (210) includes a control circuit (212) configured to program the charge trapped in the gate dielectric of the charge trapping transistor (211). [5] The image sensor assembly according to any of [2] to [4], wherein the programmable current source (210) includes at least two charge trapping transistors (211-1, 211-2) in separate, parallel current paths, and further includes a selection circuit (218-1, 218-2) configured to selectively enable the parallel current paths. [6] The image sensor assembly according to any of [1] to [5], wherein the analog compute modules (200) include collecting circuits (250), each collecting circuit (250) configured to generate a sum signal, wherein the sum signal depends on the weights of the weighted pixel signal pulses received from the other analog compute modules (200) and the weighted pixel signal pulse from the first one of the pixel circuits (100) of the pixel kernel (15). [7] The image sensor assembly according to [6], wherein each collecting circuit (250) includes a programmable main charge trapping transistor (255) configured to change an amplitude of the pixel signal pulse from the first one of the pixel circuits (100). [8] The image sensor assembly according to any of [6] to [7], further including: a ramp capacitor (263) configured to be discharged with a ramp current IRMP, the ramp current IRMP being a function of the
73342 30 weights of the weighted pixel signal pulses received from the other analog compute modules (200) and the weighted pixel signal pulse from the first one of the pixel circuits (100) of the pixel kernel (15). [9] The image sensor assembly according to [8], wherein each analog computing module (200) further includes a comparator circuit (268) configured to output the computed pulse, wherein the computed pulse width is a function of a slope of the ramp current IRMP. [10] The image sensor assembly according to any of [1] to [9], further including: a spatial filter circuit (290) configured to output a filtered pulse with a filtered pulse width, the filtered pulse width being a function of the computed pulse widths output from at least two pixel circuits (100) of the pixel kernel (15). [11] The image sensor assembly according to any of [1] to [10], further including: a gate circuit (410) configured to gate the computed pulse or the filtered pulse with a global gating pulse indicating a saturating exposure. [12] The image sensor assembly according to any of [1] to [11], further including: an artificial neural network (49) configured to receive and process signals derived from the computed pulses. [13] The image sensor assembly according to any of [1] to [12], wherein each pixel circuit (100) is controllable to successively output a pixel reset pulse and the pixel signal pulse, wherein a reset pulse width of the pixel reset pulse is a function of a reset potential of a floating diffusion region of the pixel circuit (100); and wherein the analog compute module (200) includes a signal converter (260) configured to obtain the computed pulse with a computed pulse width being a function of a difference between the signal pulse width and the reset pulse width. [14] The image sensor assembly according to any of [1] to [13], wherein each of the analog compute modules (200) includes programmable current sources (210) configured to output the weighted pixel signal pulses according to the separate weighting coefficients, wherein each programmable current source (210) includes at least two charge trapping transistors (211-1, 211-2) in parallel current paths and a selection circuit (218-1, 218-2) configured to turn on and off the parallel current paths, and wherein different weighting coefficients correspond to different amounts of charge trapped in a gate dielectric of the charge trapping transistor (211-1, 211-2) in the selected current path. [15] The image sensor assembly according to [14], further including: a sensor controller (50) configured to control each of the pixel circuits (100) to successively output a pixel reset pulse and the pixel signal pulse, wherein a reset pulse width of the pixel reset pulse is a function of a reset potential of a floating diffusion region of the pixel circuit (100), and wherein the sensor controller (50) is further configured to control the
73342 31 analog compute modules (200) to use the same charge trapping transistor (211-1, 211-2) successively for the pixel reset pulse and the pixel signal pulse for each exposure. [16] The image sensor assembly according to [14], further including: a sensor controller (50) configured to control each of the pixel circuits (100) to successively output pixel reset pulses using different charge trapping transistors (211-1, 211-2), wherein a reset pulse width of the pixel reset pulse is a function of a reset potential of a floating diffusion region of the pixel circuit (100), and successively output, after one exposure, a same number of pixel signal pulses using the same charge trapping transistors (211-1, 211-2), and wherein the sensor controller (50) is configured to control the analog compute module (200) to obtain a computed reset pulse from each pixel reset pulse and a computed signal pulse from each pixel signal pulse. [17] The image sensor assembly according to [14], further including: a sensor controller (50) configured to control each of the pixel circuits (100) to successively output a pixel reset pulse with all charge trapping transistors (211-1, 211-2) being simultaneously selected, and the pixel signal pulse with all charge trapping transistors (211-1, 211-2) being simultaneously selected, wherein a reset pulse width of the pixel reset pulse is a function of a reset potential of a floating diffusion region of the pixel circuit (100). [18] The image sensor assembly according to [14], further including: a sensor controller (50) configured to control each of the pixel circuits (100) to successively output pixel reset pulses using different charge trapping transistors (211-1, 211-2), wherein a reset pulse width of the pixel reset pulse is a function of a reset potential of a floating diffusion region of the pixel circuit (100), and output, after one exposure, a same number of pixel signal pulses using the same charge trapping transistors (211-1, 211-2), and wherein the sensor controller (50) is configured to control the analog compute module (200) to obtain a sum reset pulse from the pixel reset pulses and a sum signal pulse from the pixel signal pulses. [19] The image sensor assembly according to any of [14] to [18], wherein each pixel circuit (100) is configured to generate a pixel signal pulse having a signal pulse width being a function of both a radiation intensity incident on the pixel circuit (100) and a reset potential of a floating diffusion region of the pixel circuit (100), and further including a sensor controller (50) configured to control each of the pixel circuits (100) to successively output pixel signal pulses using different charge trapping transistors (211-1, 211-2) after one exposure. [20] The image sensor assembly according to [1] to [19], further including: a bypass multiplexer (295) configured to output the pixel signal pulse in a first operation mode and the computed pulse in a second operation mode.