WO2020049460A1

WO2020049460A1 - Imaging device with a high acquisition speed and an extended dynamic range

Info

Publication number: WO2020049460A1
Application number: PCT/IB2019/057417
Authority: WO
Inventors: Fabien CLOP; Stéphane LEMARCHAND; Thomas CARMIGNANI
Original assignee: First Light Imaging
Priority date: 2018-09-03
Filing date: 2019-09-03
Publication date: 2020-03-12
Also published as: FR3085571B1; FR3085571A1

Abstract

The invention relates to a method for controlling an imaging device, comprising steps consisting in: generating photocurrents using photosensitive elements of pixel circuits in an imaging device; in each pixel circuit, selecting a first capacitor during a first integration phase (EXH); integrating an electrical charge resulting from the photocurrent into the first capacitor; measuring a first pixel signal (PH) at the end of the first integration phase; forming a first image from the first measured pixel signals; in each pixel circuit, selecting a second capacitor during a second integration phase (EXL); integrating the electrical charge resulting from the photocurrent into the second capacitor; measuring a second pixel signal (PL) at the end of the second integration phase; forming a second image from the second measured pixel signals; and forming a composite image by combining the first image with the second image.

Description

HIGH ACQUISITION AND RANGE IMAGING DEVICE

EXTENDED DYNAMICS

The present invention relates to matrix image sensors in CMOS technology. The invention can be applied to the visible range, as well as to the infrared range SWIR, MWIR, LWIR (Short-, Mid-, Long-Wave Infra Red). It can be used in particular for video surveillance or in the biological field, for example for observing biological markers in real time, and more generally in all fields where a large dynamic range is desirable.

Generally, a CMOS type image sensor comprises pixels or photosites arranged in a matrix configuration. Each pixel includes a photosensitive area, typically a photodiode, configured to accumulate electrical charges based on the light it receives, and a readout circuit to measure the amount of charges accumulated by the photodiode. The reading circuit includes a transfer transistor for controlling the transfer of the electrical charges accumulated in the photodiode to a reading node. The pixel is thus controlled according to a cycle comprising an initialization phase, an integration phase, and a read phase. During the integration phase, the photodiode accumulates electrical charges depending on the light it receives. The reading phase consists in generating a signal corresponding to the amount of electrical charges accumulated by the photodiode during the integration phase. The initialization phase consists in eliminating the electrical charges accumulated by the photodiode during the integration phase.

Generally, imaging devices (cameras, cameras) have a dynamic range ("dynamic range") much lower than that generally encountered in the real world, as it can be perceived by the human eye. The dynamic range can be defined by the ratio between the maximum light intensity of an image (in the brightest or brightest area of the image), and the minimum intensity of the image (in the area the darkest or the least lit of the image). The dynamic range of an image is generally limited by the circuit for reading the photosensitive area.

To extend the dynamic range of the images obtained, a high dynamic range HDR ("High Dynamic Range") mode has been proposed. to take several successive images, with different exposure or integration times. A first image obtained from a long integration time makes it possible to acquire details relating to dark areas of a scene, and a second image obtained from a shorter integration time makes it possible to acquire details of bright areas of the scene. These two images are then combined into a composite image containing details of both light and dark areas.

The acquisition time of two images with different exposure times of moving scenes can generate a ghost effect according to which a moving object can appear several times in the composite image, as well as a high latency. The reconstruction of HDR images therefore does not lend itself well to video sequences, and in particular to video sequences having a high frame rate. This technique is therefore mainly used for taking separate images (in cameras).

Other methods of extending the dynamic range of images use several analog channels with different amplification gains. However, this amplification is carried out after a respective first acquisition stage of each pixel. As a result, the noise generated by the first stage is also amplified. The images obtained using the highest gain of amplification can therefore present a higher noise.

It is therefore desirable to be able to generate images at high acquisition speed, with wide dynamic range, without amplification of the noise, and without additional blurring or phantom effect resulting from the extension of the dynamic range.

Embodiments relate to a method for controlling an imaging device, comprising the steps of: generating current photos by photosensitive elements of pixel circuits in an imaging device comprising a plurality of pixel circuits, selecting from each pixel a first capacity, during a first integration phase, during the first integration phase, integrate, in each pixel circuit, an electric charge resulting from the photo-current in the first capacity of the pixel circuit, activate a command signal for measuring in each pixel circuit, a first pixel signal, as a function of the electric charge integrated in the first capacitor, during or at the end of the first integration phase, during the first integration phase or a second phase integration, forming a first image from the first measured pixel signals, selecting a second capacitance in each pixel circuit, during the second integration phase, the first and second capacitors corresponding to respective respective gain values, during the second phase d integration, integrate, in each pixel circuit, the electric charge resulting from the photo-current in the second capacity of the pixel circuit, activate the control signal to measure in each pixel circuit a second pixel signal as a function of the charge electric integrated in the second capacity, during or at the end of the second integration phase, during the second integration phase or a following integration phase, forming a second image from the second measured pixel signals, and forming a composite image by combining the first image with the second image.

According to one embodiment, the first and second integration phases have identical durations.

According to one embodiment, the method comprises steps consisting in: successively carrying out the first integration phase by selecting the first capacity, alternating with the second integration phase by selecting the second capacity, to obtain a first flow of firsts images alternately with second images, and form a second stream of composite images by combining each second image of the first stream with a first adjacent previous image in the first stream, to generate a first composite image, and by combining each second image of the first stream with a next adjacent first image in the first stream, to generate a second composite image.

According to one embodiment, the method comprises steps of dividing the second stream into groups of several successive composite images, and of merging the composite images of each group into a single respective image which is supplied at the output of the imaging device.

According to one embodiment, the method comprises, for each pixel circuit, steps consisting in: carrying out several measurements of the pixel signal, during each of the first and second integration phases, and determining the first and second pixel values to from signal slopes defined by the pixel signal measurements during the first and second integration phases, respectively. According to one embodiment, the method comprises, for each pixel circuit, steps consisting in: performing during each of the first and second integration phases, several pixel signal measurements, by alternately selecting the first capacitance and the second capacitance , determining a first pixel value from a signal slope defined by the pixel signal measurements performed with the first capacitance, the first pixel value being used to form the first image, and determining a second pixel value to from a signal slope defined by the pixel signal measurements made with the second capacitance, the second pixel value being used to form the second image.

According to one embodiment, the signal slope is calculated by eliminating saturated pixel measurements, the slope being used to determine each of the first and second pixel values at the end of the first and second integration phases, respectively.

According to one embodiment, the first gain value is greater than the second gain value, the formation of the composite image comprising, for each pixel circuit, steps consisting in: determining whether the first pixel signal corresponds to a saturated value, if the first pixel signal does not correspond to a saturated value, use the first pixel signal, to form a corresponding pixel of the composite image, and if the first pixel signal corresponds to a saturated value, form the corresponding pixel of the composite image using the second pixel signal, or alternatively steps of: comparing the first pixel signal with a first saturated value and with a second saturated value, the second saturated value corresponding to greater saturation than the first saturated value, if the first pixel signal is less saturated than the first saturated value, use the first pixel signal, to form a correct pixel spanning the composite image, if the first pixel signal is more saturated than the second saturated value, use the second pixel signal to form the corresponding pixel of the composite image, if the first pixel signal is more saturated than the first saturated value and less saturated than the second saturated value, forming the corresponding pixel of the composite image by one of the following methods: using the first pixel signal if for a previous image the first pixel signal has been used, and the second pixel signal if for the previous image the second pixel signal was used, using an average of the first pixel signal and the second pixel signal. According to one embodiment, the first gain value is greater than the second gain value, the formation of the composite image comprising, for each pixel circuit, steps consisting in: determining a number of saturated pixel measurements obtained for determining the first pixel signal, if the number of saturated pixel measurements is less than a first threshold value, using the first pixel signal, to form a corresponding pixel of the composite image, and if the number of pixel measurements saturated is greater than the first threshold value, forming the corresponding pixel of the composite image using the second pixel signal, or alternatively steps consisting in: comparing the number of saturated pixel measurements with the first threshold value, and a second threshold value, if the number of saturated pixel measurements is less than the first threshold value, use the first pixel signal, to form a corresponding pixel d e the composite image, if the number of saturated pixel measurements is greater than the second threshold value, use the second pixel signal to form the corresponding pixel of the composite image, if the number of saturated pixel measurements is included between the first threshold value and the second threshold value, form the corresponding pixel of the composite image by one of the following methods: using the first pixel signal if for a previous image the first pixel signal has been used , and the second pixel signal if for the previous image the second pixel signal was used, using an average of the first pixel signal and the second pixel signal.

According to one embodiment, the method comprises steps of initialization of each pixel circuit, to remove electrical charges accumulated in the selected capacity, before each of the first and second integration phases, or else only before each first phase d 'integration.

Embodiments may also relate to an imaging device comprising: a matrix of pixel circuits arranged in a plurality of lines and in a plurality of columns, each pixel circuit comprising a read circuit controlled by a single signal to trigger the measurement of a pixel signal, as a function of an electric charge integrated into a capacitance of the pixel circuit, and of the control circuits configured to implement the method defined above.

According to one embodiment, each pixel circuit has an architecture of the CTIA or SF type. According to one embodiment, the second capacitance of each pixel circuit is formed by the combination of two capacitors, or is formed by a capacitor distinct from a capacitor forming the first capacitor.

Examples of embodiment of the invention will be described in the following, without implied limitation in relation to the attached figures, among which:

FIG. 1 schematically represents a conventional imaging device,

FIG. 2 schematically represents a conventional pixel circuit,

FIG. 3 schematically represents a timing diagram of different signals which may appear in the pixel circuit of FIG. 1,

FIG. 4 schematically represents a timing diagram of different signals which may appear in the pixel circuit of FIG. 1, according to a first embodiment,

FIG. 5 schematically represents a timing diagram of different signals which may appear in the pixel circuit of FIG. 1, according to a second embodiment,

FIG. 6 schematically represents a pixel circuit, according to one embodiment,

FIG. 7 schematically represents a timing diagram of different signals which may appear in the pixel circuit of FIG. 6, according to another embodiment,

FIG. 8 schematically represents another conventional pixel circuit,

FIG. 9 schematically represents a pixel circuit, according to another embodiment,

FIG. 10 schematically represents a circuit for combining pixels, according to one embodiment,

FIG. 11 schematically represents a circuit for combining pixels, according to another embodiment,

FIG. 12 illustrates a mode of combining images, according to one embodiment.

FIG. 1 represents an imaging device IS incorporating an image sensor. The IS device can be an image sensor, a portable device such as a camera, a camera, a mobile phone, or any other device having an image capture function. The IS device includes conventionally a PXA matrix of PX pixel circuits. The PXA matrix includes PX pixel circuits arranged in a plurality of rows and a plurality of columns. The IS device also includes RDRV, RDEC, CDRV, CDEC, TAC control circuits, configured to provide different control signals to the PX pixel circuits according to different phases to be chained to capture images. The PXA matrix provides pixel signals to an AMP, ADC, IPRC processing circuit configured to provide IMG images from the pixel signals.

The pixel lines are selectively activated by an RDRV line control circuit in response to an RDEC line address decoder. The pixel columns are also activated by a column control circuit controlled by a CDEC column decoder. The RDRV and CDRV circuits provide the appropriate voltages to drive the PX pixel circuits. The IS device also includes a TAC control circuit which controls the RDEC and CDEC address decoders and the RDRV, CDRV control circuits to select the appropriate line and column of pixels at all times for pixel reading. The pixel signals read are amplified by the AMP amplifier, and digitized by the ADC analog / digital converter. The digitized pixel signals are processed by an IPRC image processor which provides an IMG image from the digitized pixel signals. The IPRC image processor may include memories for processing and storing the received image signals.

FIG. 2 represents a conventional PX pixel circuit with architecture of the CTIA (Charge Trans Impedance Amplifier) type, for example described in the patent application US 2016/0014366. The pixel circuit PX comprises a photosensitive element EP such as a photodiode, an amplifier A1 whose gain depends on the value of a capacitor connected in parallel between an input and the output of the amplifier A1. In the example of FIG. 2, a direct input of the amplifier A1 is connected to ground, and an inverting input of the amplifier is connected to a direct terminal of the photodiode EP and to a respective terminal of integration capacitors C1, C2. The output of the amplifier A1 is connected to the other terminal of the capacitor C1, and connected to the other terminal of the capacitor C2 by means of conduction terminals of a transistor TL mounted as a switch and controlled by a signal TL activation of a low gain mode. The inverting input and the output of amplifier A1 are connected at the conduction terminals of an initialization transistor TR mounted as a switch and controlled by an initialization signal RST. The output of amplifier A1 supplies a pixel measurement signal VX to a sample-and-hold circuit SHC controlled by a control signal SH. The measurement signal is stored in a buffer memory BF awaiting a reading of the pixel controlled by a read signal RD.

Thus, when the transistor TL is open, only the capacitor C1 is connected in parallel with the amplifier A1. The PX pixel circuit is therefore in high gain mode (low capacity). When the transistor TL is closed, the capacitors C1, C2 are connected in parallel with the amplifier A 1. The pixel circuit PX is then in low gain mode (higher capacity).

FIG. 3 represents timing diagrams of the signals RST, LG, VX and SH, illustrating a conventional mode of operation of the pixel circuit PX (described in patent application US 2016/0014366). Between instants t0 and t1, the pixel circuit PX is initialized by a pulse of the signal RST and a pulse of the signal LG which make the transistors TR and TL conducting. The electric charges accumulated by the transistors C1, C2 are therefore removed. The output signal VX of amplifier A1 is at a maximum voltage. At time t1, when the initialization signal RST drops to 0, the transistor TL is blocked (signal LG at 0), the signal VX begins to decrease as the photodiode EP accumulates electrical charges. The decrease slope of the signal VX depends on the illumination of the photodiode EP and on the gain of the amplifier A1, which depends on the capacitance C1 (between times t1 and t2) placed in parallel with the amplifier A1. Just before the end of an integration phase at time t2, a read pulse is sent to the read transistor TX to read the signal strength VX.

Between instants t2 and t3, the pixel circuit PX is again initialized by a pulse of the signal RST and a pulse of the signal LG, to start a new phase of integration at the instant t3. During this integration phase, the illumination of the photodiode EP is more intense, and the signal VX reaches a saturation threshold value VT. A circuit (not shown) detects the saturation ZS of the circuit, and triggers a reading of the signal VX using a pulse of the signal SH, then triggers the low gain mode at time t6 by turning the transistor TL on (LG signal at 1). The integration phase therefore continues for the VX signal by a rising edge, then a decrease with a lower slope. Just before the end of the integration phase at time t7, a new pulse of the measurement control signal SH is sent to the read transistor TX to read the intensity of the signal VX. Between instants t7 and t8, the signal the circuit is again initialized using a pulse of the signal RST, the signal LG remaining at 1.

As a result, the pixel circuit PX is controlled according to FIG. 3, so as to carry out one or two readings at each integration phase, depending on whether saturation (two readings) is detected or not (only one reading). To exploit the switch from high gain mode to low gain mode, and determine the light intensity received by the photodiode EP during the integration phase, it is necessary to memorize all the measurements made during the integration phase, as well as the fact that the pixel was obtained in low gain mode. In addition, determining the light intensity received by the EP photodiode during the integration phase is complex. In addition, the saturation and gain threshold values related to the values of the capacitances C1 and C2 are analog values which necessarily vary from one pixel to another because they depend on the physical characteristics of the pixel circuits. This results in an imprecision in the value of each pixel finally determined from the measurements.

FIG. 4 represents timing diagrams of the signals RST, LG, SH, RD and VX, illustrating a control mode of the pixel circuit PX, according to an embodiment. Between instants t10 and t11 (initialization phase RS), the pixel circuit PX is initialized using the signal RST and the signal LG making the transistors TR and TL conducting. Moment t11 marks the start of an EXH integration phase. The EXH integration phase which ends at an instant t13 makes it possible to acquire the pixel signals of a high gain image (i). Between times t1 1 and t13, the signal LG is then at 0 to select the high gain mode.

During the EXH integration phase, an RDL reading phase of the imaging device IS takes place to constitute a previous image (i - 1) with low gain. During the RDL read phase, a pulse of the RD signal makes it possible to obtain a measurement of the pixel signal collected during the previous integration phase. During a measurement phase SB of the pixel circuit PX, between the instants t12 and t13, a pulse SH for controlling the sampler circuit SHC blocker collects a PH measurement of the VX signal. During the measurement phase SB, the measurement PH of the charges accumulated in the capacitor C1 is transferred to the buffer memory BF at the output of the sample-and-hold circuit SHC.

Between the instants t13 and t14, the circuit PX is again initialized by a pulse of the signal RST applied to the transistor TR and a pulse of the signal LG applied to the transistor TL. At time t14, a new EXL integration phase starts to acquire a next image (i + 1) with low gain. Thus, during the EXL integration phase, between times t14 and t15, the signal LG remains active to select the low gain mode. During the EXL integration phase, an RDH reading phase takes place to constitute the image (i) resulting from the EXH integration phase. During the RDH reading phase, a pulse of the RD signal commands the reading of the buffer memory BF to collect the measurement PH of the pixel signal obtained during the integration phase EXH. Just before the end of the integration phase EXL, a measurement phase SB of the pixel circuit PX takes place, between the instants T15 and T16, during which a pulse of the signal SH commands the circuit SHC to collect a measurement PL of the signal VX. During the measurement phase SB, the measurement PL of the charges accumulated in the capacitors C1, C2 is transferred to the buffer memory BF.

Between instants T16 and T17, a new RS initialization phase takes place, during which the signal RST is at 1 and the signal LG remains at 1. The instant T17 marks the start of a new integration phase EXH for acquire a next image (i + 2) with high gain. During this EXH integration phase which ends at time t19, an RDL reading phase of the pixels of the low gain image (i + 1) takes place, generated during the EXL integration phase. The measurement phase SB of the pixel circuit PX with high gain takes place between the instants t18 and t19, to collect a new measurement PH of the signal VX.

Thus, the control mode illustrated in FIG. 4 makes it possible to constitute alternately, a high gain image and a low gain image. It should be noted that these two types of image can be obtained without varying the duration of the EXH-EXL integration phases. Two successive images, respectively at high and at low gain can then be merged to generate a composite image with high dynamic range, for example by replacing the saturated pixels of the image with high gain by the corresponding pixels of the image with low gain . High and low gain images can thus be provided alternately in a first image stream. After the merge operation, composite images are provided in a second image stream. The bit rate of the second image stream can be equal to half the bit rate of the first image stream, or can be equal to this one, if each high or low gain image is used to generate two composite images, merging it a first time with the previous image, and a second time with the next image in the first stream. The previously described operating mode is therefore particularly favorable for the production of high bit rate video images, for example 600 frames per second, or at a higher bit rate.

FIG. 5 represents timing diagrams of the signals RST, LG, SH, RD and VX, illustrating a control mode of the pixel circuit PX, according to another embodiment. Between instants t20 and t21, an initialization phase RS initializes the pixel circuit PX by applying a pulse of the signal RST and a pulse of the signal LG to make the transistors TR and TL pass. The instant T21 marks the start of an EXH1 integration phase to acquire the pixels of a high gain image (i). During the integration phase EXH1 which ends at an instant t25, the signal LG is at 0 to select the high gain mode. During the EXH1 integration phase, an RDH1 reading phase takes place to constitute the image (i). A pulse of the signal SH controlling the sample-and-hold circuit SHC makes it possible to carry out a first measurement of the signal VX at the output of the pixel circuit PX, between times t22 and t23. This pulse is followed by a pulse of the signal RD to collect the measurement of the pixel signal which has just been stored in the buffer memory BF.

These read operations are repeated during the integration phase EXH1, to read the amplitude of several measurement points PH1, PH2, PHn-1, PHn of the signal VX, between times t21 and t25, the last measurement point PHn being measured at time t24, just before the end of the integration phase EXH1, the measurement of the point PHn can be read (RD) during the initialization phase RS or of the following integration EXL1. These read operations of several measurement points of the signal VX are performed for all the pixels PX of the imaging device IS. Thus, the integration phases EXH1 and reading RDH1 to constitute the same image (i) are substantially simultaneous.

Between instants t25 and t26, a pulse of the signal RST and a pulse of the signal LG are applied respectively to the transistors TR and TL of the pixel circuit PX to initialize the pixel circuit PX, and to start a new phase of integration EXL1. The integration phase EXL1 takes place between times t26 and t27 to generate a next image (i + 1) with low gain. Thus, during the integration phase EXL1, the signal LG is active to select the low gain mode. In parallel, a new RDL1 reading phase takes place for the same image (i + 1). Several pulses of the measurement signal SH for controlling the sample-and-hold circuit SHC, each followed by pulses of the signal RD, are emitted, to collect several measurement points PL1, PL2, PLn-1, PLn of the signal VX.

The number n of points measured during each integration phase EXH1, EXL1 can vary from 2 to, for example 60 in the case where the imaging device can provide 600 images per second. If you want to produce images at a rate of 24 images per second, n can be set to 25.

According to one embodiment, the measurement points PH1 -PHn, PL1 -PLn can be used to constitute n images for each integration phase EXH1, EXL1, each image i being formed of all the pixels PHi or PU read for all the pixel PX circuits. Thus the n images i obtained are more and more clear since they are obtained for increasingly long integration times. The n images i can then be merged to constitute a high gain image.

According to one embodiment, the measurement points PH1 -PHn, PL1 -PLn are used to determine the slope of the signal VX during each integration phase EXH1, EXL1. A slope can be determined from several points by applying for example the method of least squares or the method of Fowler. This method is detailed in the document SPIE vol. 1541, Infrared Sensors, Detectors, Electronics and Signal Processing (1991), p. 127-133. Each of the slopes thus determined makes it possible to deduce a corresponding pixel value from the image to be generated, at the end of the integration phase EXH1, EXL1.

Thus, the control mode illustrated in FIG. 5 makes it possible to constitute a high gain image (during the EXH1 / RDH1 phases) and a low gain image (during the EXL1 / RDL1 phases). These two images can then be merged to obtain a wide dynamic range image. As before, the frame rate of the high and low gain image stream and of the composite image stream may remain unchanged after such a merger, in particular merging each image in the high and low gain image stream, with the previous image, to generate a first composite image, and with the following image to generate a second composite image.

In the event of saturation of the pixel signal PX, the slope of the points PH1 -PHn and / or of the points PL1 -PLn can be calculated only from the unsaturated points, by eliminating the saturated points.

FIG. 6 represents a pixel circuit PX1, according to another embodiment. The pixel circuit PX1 differs from the circuit PX in that it comprises an additional transistor TH whose conduction terminals are connected in series with the capacitor C1, the transistor TH being controlled by a signal HG. In the high gain mode, the signal HG is for example at 1 so that the transistor TH is conducting, while the transistor TL is kept non-passing (signal LG at 0). In the low gain mode, the signals LG and HG are respectively at 1 and 0. Thus, it is possible to switch between the high and low gain modes, while retaining the charge accumulated in the capacitors C1, C2.

The pixel circuit PX1 can be used with the control mode of FIG. 5. In this case, only one initialization phase RS can be carried out, for two consecutive integration phases EXH1, EXL1. Thus, in the example of FIG. 5, the initialization of the pixel circuit PX1 between the instants t25 and t26 can be deleted (or else between the instants t20 and t21 and t27 and t28). This arrangement makes it possible to reduce the reading noise.

Of course, a lower gain mode can be selected by simultaneously activating the TL and TH transistors. It is then necessary to initialize the pixel circuit PX1 between each integration phase.

FIG. 7 represents timing diagrams of the signals RST, LG, HG and VX, illustrating a control mode of the pixel circuit PX1, according to another embodiment. This control mode differs from that of FIG. 5 in that during each integration phase, the high and low gain modes are selected alternately, several times, to acquire one or more measurement points of the VX signal following each mode selection.

Between instants t30 and t31, an initialization phase RS initializes the pixel circuit PX1 by applying a pulse of the signal RST to make the transistor TR on, and pulses of the signals LG and HG to make the transistors TL and TH on. Moment T31 marks the start of a phase EXP integration to acquire the pixels of a high gain image (i) and the pixels of a low gain image (i + 1). During the integration phase EXP which ends at an instant t39, the signals LG, HG are alternately at 0 and at 1, to alternately select the low and high gain modes. When the low gain mode is selected, a reading phase RL takes place to acquire one or more measurement points of the signal VX, corresponding to the voltage across the capacitor C2. Similarly, when the high gain mode is selected, a RH reading phase takes place to acquire one or more measurement points of the signal VX, corresponding to the voltage across the capacitor C1. At each measurement point, the signals SH and RD are activated successively as shown in FIG. 5.

Thus, in the example of FIG. 7, between times t31 and t32, the low gain mode is selected, and the measurement points PL1, PL2 are read. Between times t32 and t33, the high gain mode is selected, and the measurement points PH1, PH2 are read. Between instants t33 and t34, the low gain mode is selected, and the measurement points PL3, PL4 are read. Between instants t34 and t35, the high gain mode is selected, and the measurement points PH3, PH4 are read. Between instants t35 and t36, the low gain mode is selected, and the measurement points PL5, PL6 are read. Between instants t37 and t38, the low gain mode is selected, and the measurement points PLn-1, PLn are read. Between instants t38 and t39, the high gain mode is selected, and the measurement points PHn-1, PHn are read.

At the end of the integration phase EXP, the pixel circuit PX1 is initialized between times t39 and t40, by pulses of the signals RST, LG and HG, applied respectively to the transistors TR and TL and TH of the pixel circuit PX1, to start a new integration phase and generate the following two images. As previously described with reference to FIG. 5, the measurement points PL1 -PLn are used either to generate n images, or to determine the slope of the signal VX, during the integration phase EXP in the low gain mode. Similarly, the measurement points PH1 -PHn are used either to generate n images, or to determine the slope of the signal VX during the integration phase EXP in the high gain mode. The slopes thus determined of the signal VX are then used to determine the value of a pixel of the image (i) with low gain, and the value of this pixel in the image (i + 1) with high gain. Thus, the control mode illustrated in FIG. 7 makes it possible to constitute a high gain image and a low gain image (during the same integration phase EXP which can extend over a double duration of the integration phases EXL1, EXH1 in Figure 5). The two images (i) and (i + 1) can then be merged to obtain a composite image with a wide dynamic range. The command mode of FIG. 7 makes it possible to reduce or eliminate the ghost effect which can appear when the observed scene includes a moving object, which appears in the form of a single blurred object in the images obtained.

As before, the bit rate of the images can remain unchanged after such a fusion, by merging each image, a first time with the previous image, and a second time with the following image, to form two composite images.

FIG. 8 represents a pixel circuit PX2 with architecture of type S F (Source Follower). The pixel circuit PX2 comprises a photosensitive element EP1 such as a photodiode, and transistors TG, T1, TL1, TR1 and TS, for example of the N-channel MOS type. The photosensitive element EP1 comprises a first terminal connected to the ground and a second terminal connected to an electrical charge storage area RN via the transfer transistor TG. The storage area RN comprises a capacitor C3 connected between a conduction terminal of the transistor TG and the ground, and a capacitor C4 connected on one side to the ground and on the other to a conduction terminal of the transistor TL1 for selecting mode, the other conduction terminal of transistor TL1 being connected to the storage area RN. The transistors TG and TL1 are controlled respectively by signals VG and LG. The storage area RN is connected to the gate of the transistor T1 and is connected to a supply voltage source VS via the conduction terminals of the transistor TR1 controlled by the initialization signal RST.

The transistor T1 is connected between the supply voltage source VS and the read transistor TS. The transistor TS is controlled by a signal SH to supply a measurement signal VX representative of the quantity of electrical charges accumulated by the photosensitive element EP1 and transferred to the storage area RN. Thus, the transistor T1 supplies the signal VX, and the transistor TS supplies the voltage VX at the output of the pixel circuit PX2 when the latter is selected for reading by the signal SH. The transistor TR1 is controlled by the RST signal to initialize the RN storage area. When the signal LG puts the transistor TL1 in a non-conducting state, a high gain mode is selected, only the capacitor C3 being able to accumulate electrical charges. Conversely, when the signal LG puts the transistor TL1 in a conducting state, a low gain mode is selected, the capacitors C3 and C4 being able to accumulate electrical charges.

The PX2 pixel circuit can be used in the same way as the PX circuit in the control modes of Figures 4 and 5.

FIG. 9 represents a pixel circuit PX3, according to another embodiment. The pixel circuit PX3 differs from the pixel circuit PX2 in that it includes an additional transistor TH1 controlled by the signal HG, and interposed between the storage area RN and the capacitor C3. The pixel circuit PX3 can be used in the same way as the circuit PX1 in the control modes of FIGS. 4, 5 and 7. In the control mode of FIG. 5, the pixel circuit PX3 can be initialized only every two integration phases.

FIG. 10 shows a digital pixel processing circuit PCC for generating a PXC pixel of a composite image formed by the fusion of a high gain image and a low gain image. The PCC circuit receives the value of a PXH pixel from the high gain image and the value of a corresponding PXL pixel from the low gain image, the PXH and PXL pixel values having been previously scanned and corrected. The corrections applied to the PXL and PXH pixel values are intended to remove an initial offset and to standardize the gains of the pixel circuits of the image sensor in the high and low gain modes. The initial offsets in the high and low gain modes can be determined by averaging while the image sensor is placed in the dark (in the absence of illumination). The corrections to be applied individually to each pixel circuit can be determined from pixel measurements obtained by placing the image sensor in the presence of an image of uniform color.

The circuit PCC comprises a comparator CP for comparing the value of the pixel PXH with a threshold value TD. The comparator CP controls a switch S1 comprising an input receiving the value of the pixel PXH. The PCC circuit also includes an amplifier G of gain G receiving the value of the pixel PXL. The output of the AM amplifier is connected to another input of switch S1. Thus, the switch S1 supplies either the value of the pixel PXH, or a corrected value of the pixel PXL at the output of the amplifier AM.

The threshold value TD is fixed so as to correspond to a saturation value of the pixel PXH. If the value of the pixel PXH obtained with a high gain is greater than the threshold value TD, and therefore considered to be unsaturated (from the curve of the signal VX in the examples of FIGS. 4, 5, 7), the pixel PXH is transmitted at the output of the PCC circuit. Otherwise, the value of the pixel PXL obtained with a low gain is multiplied by the gain value G, to obtain a corrected value PXG, this value being transmitted at the output of the PCC circuit. Thus, the PXC output value of the PCC circuit is obtained as follows:

If PXH> TD, PXC = PXH (1)

If PXH <TD, PXC = PXG = G x PXL (2)

It should be noted that with the pixel circuits described above, the slope of the signal VX is negative between the start and the end of each of the integration phases EXH, EXL, EXH1, EXL1, EXP. On the other hand, with other pixel circuits, this slope of the VX signal can be positive. In the latter case, it goes without saying that the comparisons of the PXH signal with the threshold are reversed in formulas (1) and (2).

The value of the gain G is chosen so as to obtain a substantially linear response curve of the pixel circuit as a function of the light intensity applied or collected by the photosensitive element EP, EP1 of the pixel circuit.

According to an embodiment introducing hysteresis, the value of each pixel PXH of the high gain image is compared with two threshold values THL and THH, with THH <THL (in the case of a negative VX signal slope) and the following formulas are applied:

If PXH <THH, PXC = PXG (3) If PXH> THL, PXC = PXH (4) If THH <PXH <THL, PXC = S1 (i-1, PXH, PXG) (5) with S1 (i-1 , PXH, PXG) is the state of the switch S1 determined for the pixel PXC in the previous composite image (i-1).

According to an embodiment introducing simple averaging, the following formulas are applied:

If PXH <THH, PXC = PXG (6) If PXH> THL, PXC = PXH (7)

If THH <PXH <THL, PXC = (PXH + PXG) / 2 (8)

According to an embodiment introducing weighted averaging, the following formulas are applied:

If PXH <THH, PXC = PXG (9)

If PXH> THL, PXC = PXH (10)

If THH <PXH <THL, PXC = (ax PXH + bx PXG) (1 1) where a and b are coefficients such that a + b = 1. The interval between THH and THL can be divided into several segments, by example of the same width, the values of the coefficients a and b varying from one segment to another. Thus, for example if we consider three segments of width (THL-THH) / 3, the coefficients (a, b), can be respectively equal to (0.75, 0.25) for the first segment starting from THH, (0.5 , 0.5) for the middle segment, and (0.25, 0.75) for the third segment on the THL threshold side.

FIG. 11 represents a PCC pixel processing circuit for generating a pixel of a composite image formed by the fusion of a high gain image and a low gain image, in the case where several measurement points are used to form the high gain image and the low gain image. The circuit PCC1 differs from the circuit PCC in that the comparator CP compares to a threshold value TD1, not the value of the pixel with high gain PXH, but a value such as the number of PNS points measured unsaturated. The threshold value TD1 can be set to a value such as 7 or 8, or to a lower value such as 4 or 5, depending on whether the least squares method or the Fowler method is used to determine the slope of each pixel signal VX.

Other embodiments can be obtained by transposing the embodiments represented by formulas (3) to (12), in the case where the comparisons are made, not with the pixel value PXH, but with the number PNS of points unsaturated measured, which is then compared to two separate threshold values.

FIG. 12 illustrates a mode of fusion of the images generated by the imaging device IS. The imaging device IS successively constructs, from the pixel signals supplied by the pixel circuits PX, PX1, PX2, PX3, images with high gain IMH and images with low gain IML. In camera mode, the imaging device provides a stream of high IMH gain images alternated with low IML gain images. According to one embodiment, the images of each pair of successive images (IMH, IML) or (IML, IMH) are combined for example using the PCC circuit to obtain a composite image IM1, IM2 ... IM10. When each IMH, IML image is combined with the previous image and the following image, to form two composite images IMi, IMi + 1, a flow of composite images is obtained having the same image rate as the flow of high and low gain images IMH, IML.

According to one embodiment, the device IS is configured to merge together several of the successive composite images IM1 -IM10. According to one example, the device IS provides an IMH, IML image or an IM1-IM10 composite image at the image rate of 600 images per second, and generates an IM image from each successive group distinct from 10 composite images. Thus, IM images can be generated with an image rate of 60 images per second.

It will be clear to a person skilled in the art that the present invention is capable of various variant embodiments and various applications. In particular, the invention is not limited to the pixel circuits described, but can be implemented with other pixel circuits, provided that these pixel circuits are configured to operate with different gains which can be selected. .

Nor is the invention limited to pixel circuits that can apply two different gains. Thus, the gain values used for the high and low gain modes can be adjusted for example as a function of an average illumination value of the observed scene. Furthermore, provision may be made to generate three or more images at different gains and to combine these images to form each composite image.

The respective durations of the integration phases are also not necessarily identical, and can be adjusted relative to each other, for example for the same overall integration time, to obtain better image quality, especially depending on the lighting of the scene.

Claims

1. Method for controlling an imaging device, comprising steps consisting in:

generate photo-currents by photosensitive elements (EP) of pixel circuits in an imaging device (IS) comprising a plurality of pixel circuits (PX, PX1, PX2, PX3),

select in each pixel circuit a first capacity (C1, C3), during a first integration phase (EXH, EXH1, EXP),

during the first integration phase, integrate, in each pixel circuit, an electric charge resulting from the photo-current in the first capacity of the pixel circuit,

activate a control signal (SH) to measure in each pixel circuit, a first pixel signal (PH), as a function of the electric charge integrated in the first capacitor, during or at the end of the first integration phase,

during the first integration phase or a second integration phase, form a first image (IMH) from the first measured pixel signals, select a second capacitance in each pixel circuit (C1 + C2, C2, C3 + C4 , C4), during the second integration phase (EXL, EXL1, EXP), the first and second capacities corresponding to distinct respective gain values, during the second integration phase, integrate, in each pixel circuit, the electric charge resulting from the photo-current in the second capacity of the pixel circuit,

activate the control signal (SH) to measure in each pixel circuit a second pixel signal (PL), as a function of the electric charge integrated in the second capacitor, during or at the end of the second integration phase, during the second integration phase, or a subsequent integration phase, forming a second image (IML) from the second measured pixel signals, and forming a composite image (IM1 -IM10) by combining the first image with the second image .

2. Method according to claim 1, in which the first and the second integration phase have identical durations.

3. Method according to claim 1 or 2, comprising steps consisting in:

successively carry out the first integration phase (EXH, EXH1, EXP) by selecting the first capacity, alternating with the second integration phase (EXL, EXL1, EXP) by selecting the second capacity, to obtain a first flow of firsts images (IMH) alternating with second images (IML), and

forming a second stream of composite images (IM1 -IM10) by combining each second image of the first stream with a first adjacent previous image in the first stream, to generate a first composite image, and by combining each second image of the first stream with a first adjacent next image in the first stream, to generate a second composite image.

4. Method according to claim 3, comprising steps of dividing the second stream into groups of several successive composite images (IM1 -IM10), and of merging the composite images of each group into a single image (IMG) respectively which is supplied in output from the imaging device (IS).

5. Method according to one of claims 1 to 4, comprising, for each pixel circuit (PX, PX1 -PX3), steps consisting in:

perform several pixel signal measurements (PH1 -PHn, PL1 -PLn), during each of the first and second integration phases (EXH1, EXL1), and

determine the first and second pixel values (PXH, PXL) from signal slopes defined by the pixel signal measurements during the first and second integration phases, respectively.

6. Method according to one of claims 1 to 4, comprising, for each pixel circuit (PX1, PX3), steps consisting in: perform, during each of the first and second integration phases (EXP), several pixel signal measurements (PH1 -PHn, PL1 -PLn), by alternately selecting the first capacitance (C1, C3) and the second capacitance (C2, C4 ), determine a first pixel value (PXH) from a signal slope defined by the pixel signal measurements (PH1 -PHn) carried out with the first capacitance, the first pixel value being used to form the first image (IMH), and

determining a second pixel value (PXL) from a signal slope defined by the pixel signal measurements (PL1 -PLn) made with the second capacitance, the second pixel value being used to form the second image (IML ).

7. Method according to one of claims 5 and 6, in which the signal slope (VX) is calculated by eliminating saturated pixel measurements, the slope being used to determine each of the first and second pixel values (PXL, PXH ) at the end of the first and second integration phases, respectively.

8. Method according to one of claims 1 to 7, wherein the first gain value is greater than the second gain value, the formation of the composite image (IM1 -IM10) comprising, for each pixel circuit (PX , PX1 -PX3), steps consisting in:

determine if the first pixel signal (PH, PXH) corresponds to a saturated value,

if the first pixel signal does not correspond to a saturated value, use the first pixel signal to form a corresponding pixel in the composite image, and if the first pixel signal corresponds to a saturated value, form the corresponding pixel of the composite image using the second pixel signal (PL, PXL), or else steps consisting in:

compare the first pixel signal (PH, PXH) to a first saturated value and to a second saturated value, the second saturated value corresponding to greater saturation than the first saturated value,

if the first pixel signal is less saturated than the first saturated value, use the first pixel signal, to form a corresponding pixel of the composite image,

if the first pixel signal is more saturated than the second saturated value, use the second pixel signal (PL, PXL) to form the corresponding pixel of the composite image,

if the first pixel signal is more saturated than the first saturated value and less saturated than the second saturated value, form the corresponding pixel of the composite image by one of the following methods:

using the first pixel signal if for a previous image the first pixel signal was used, and the second pixel signal if for the previous image the second pixel signal was used,

using an average of the first pixel signal and the second pixel signal.

9. Method according to one of claims 5 to 7, in which the first gain value is greater than the second gain value, the formation of the composite image (IM1 -IM10) comprising, for each pixel circuit (PX , PX1 -PX3), steps consisting in:

determining a number of saturated pixel measurements obtained to determine the first pixel signal (PH, PXH),

if the number of saturated pixel measurements is less than a first threshold value, use the first pixel signal, to form a corresponding pixel of the composite image, and

if the number of saturated pixel measurements is greater than the first threshold value, form the corresponding pixel of the composite image using the second pixel signal (PL, PXL),

or steps consisting in:

compare the number of saturated pixel measurements with the first threshold value and a second threshold value,

if the number of saturated pixel measurements is less than the first threshold value, use the first pixel signal, to form a corresponding pixel of the composite image,

if the number of saturated pixel measurements is greater than the second value threshold, use the second pixel signal (PL, PXL) to form the pixel

corresponding to the composite image,

if the number of saturated pixel measurements is between the first threshold value and the second threshold value, form the corresponding pixel of the composite image by one of the following methods:

using an average of the first pixel signal and the second pixel signal.

10. Method according to one of claims 1 to 9, comprising steps of initialization of each pixel circuit (PX, PX1 -PX3), to remove electrical charges accumulated in the selected capacity, before each of the first and second phases integration, or only before each first integration phase.

1 1. Imaging device comprising:

a matrix (PXA) of pixel circuits (PX, PX1, PX2, PX3) arranged in a plurality of lines and in a plurality of columns, each pixel circuit comprising a read circuit (SHC, TS) controlled by a single signal (SH) to trigger the measurement of a pixel signal (PH, PL), as a function of an electric charge integrated into a capacity of the pixel circuit, and

control circuits configured to implement the method according to one of claims 1 to 10.

12. Device according to claim 11, in which each pixel circuit (PX, PX1, PX2, PX3) has an architecture of the CTIA or SF type.

13. Device according to claim 1 1 or 12, in which the second capacitance (C1 + C2, C2, C3 + C4, C4) of each pixel circuit (PX, PX1, PX2, PX3) is formed by the combination of two capacitors (C1 + C2, C3 + C4), or is formed by a capacitor (C2, C4) distinct from a capacitor (C1, C3) forming the first capacitor.