AU2018387689B2 - Method and system for imaging at high and low light levels - Google Patents

Method and system for imaging at high and low light levels Download PDF

Info

Publication number
AU2018387689B2
AU2018387689B2 AU2018387689A AU2018387689A AU2018387689B2 AU 2018387689 B2 AU2018387689 B2 AU 2018387689B2 AU 2018387689 A AU2018387689 A AU 2018387689A AU 2018387689 A AU2018387689 A AU 2018387689A AU 2018387689 B2 AU2018387689 B2 AU 2018387689B2
Authority
AU
Australia
Prior art keywords
imaging mode
imaging
photon
counting
mode
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
AU2018387689A
Other versions
AU2018387689A1 (en
Inventor
Baya BENNAI
Yves Courcol
Thierry Midavaine
Xavier Normandin
Ludovic Perruchot
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Thales SA
Original Assignee
Thales SA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Thales SA filed Critical Thales SA
Publication of AU2018387689A1 publication Critical patent/AU2018387689A1/en
Application granted granted Critical
Publication of AU2018387689B2 publication Critical patent/AU2018387689B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J1/00Photometry, e.g. photographic exposure meter
    • G01J1/42Photometry, e.g. photographic exposure meter using electric radiation detectors
    • G01J1/4228Photometry, e.g. photographic exposure meter using electric radiation detectors arrangements with two or more detectors, e.g. for sensitivity compensation
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J1/00Photometry, e.g. photographic exposure meter
    • G01J1/02Details
    • G01J1/0228Control of working procedures; Failure detection; Spectral bandwidth calculation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/40Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
    • H04N25/42Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by switching between different modes of operation using different resolutions or aspect ratios, e.g. switching between interlaced and non-interlaced mode
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/50Control of the SSIS exposure
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J1/00Photometry, e.g. photographic exposure meter
    • G01J1/42Photometry, e.g. photographic exposure meter using electric radiation detectors
    • G01J1/44Electric circuits
    • G01J2001/4413Type
    • G01J2001/442Single-photon detection or photon counting
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/30Transforming light or analogous information into electric information
    • H04N5/33Transforming infrared radiation

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • General Physics & Mathematics (AREA)
  • Studio Devices (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)
  • Solid State Image Pick-Up Elements (AREA)
  • Liquid Crystal Display Device Control (AREA)

Abstract

The invention relates to a method for imaging a scene using an imaging system making it possible to obtain an image of the scene, the imaging system comprising a device for acquiring frames according to acquisition parameters, and a unit for processing the acquired frames. The method comprises the steps: A) selection of an imaging mode from among a conventional imaging mode, a natural photon-counting imaging mode and a forced photon-counting imaging mode, and, depending on the selected imaging mode, determination by the processing unit of corresponding acquisition parameters, B) acquisition of at least one frame by the acquisition device parameterized with said acquisition parameters, and transmission of the frames that were acquired to the processing unit in order to obtain an image, the image being obtained at the end of the following sub-steps if the selected imaging mode is the natural photon-counting imaging mode or the forced photon-counting imaging mode: -binarization of the frames that were acquired and -summing the binarized frames to obtain an image, C) estimation of the quality of the image obtained, D) D1) depending on the quality of the image obtained, determining a new imaging mode selected among the conventional imaging mode, the natural photon-counting imaging mode, and the forced photon-counting imaging mode, D2) repeating steps A, B, C and D with the new imaging mode selected as the imaging mode.

Description

METHOD AND SYSTEM FOR IMAGING AT HIGH AND LOW LIGHT LEVELS
The field of the invention is that of imaging and more particularly imaging at low light levels. In the visible range, the day vision capability of a human decreases with decreasing light levels. In a night-time environment, the various sky conditions, mainly affected by the phase of the Moon and cloud cover, allow an illumination at ground level of between 1 lux and 0.1 mlux which is also qualified as levels of night (according to the Aero 790 40 standard) ranging from night 1 (starting at 1 lux) to night 5 (ending at 0.1 mlux), respectively. The level of 0.1 mlux constitutes an ultimate level of illumination to be reached and enhanced for visionics. Increasing the image quality at level 5 night constitutes an operational advantage. However, in low light conditions, only very few photons are received from the observed scene. As the received flux decreases, the signal-to-noise ratio (SNR) decreases until it becomes insufficient for vision. The signal is then drowned out by the noise generated by the detector and the image from the detector is unusable or highly degraded. Thus, the sensitivity of and noise from the detector limit performance for imaging at very low light levels. Generally, low-light-level imagers are optimized by increasing the area of the pixel and by increasing the integration time as far as is possible. Currently, improving night performance may also involve the use of technologies with low read and dark noise and for example using an amplifying medium introducing multiplication gains onto the signal. Mention may be made of: " ICMOS: image-intensifier tube (11 tube) (with MCP, for microchannel plate) optically coupled to a CMOS array; " EBCMOS: photocathode + CMOS array (bombarded by photoelectrons) integrated in a tube; " EMCCD: CCD array with an electron multiplier which provides amplification via avalanche on the multiplexed signal; m APD (avalanche photodiode) array either in linear mode (gain multiplier) or in Geiger counter mode (triggering of a current pulse on generation of a photoelectron). For scientific applications such as astronomy or biomedical imaging, where it is known that very few photons will be received from what it is desired to observe, photon detection and photon-counting imaging is used; this technique was first introduced many years ago with the use of photomultipliers in time signal processing. Photon-counting imaging is based on the principle that the number of photons arriving at a pixel array detector over a given exposure time is a stochastic process. It is characterized by a Poisson distribution with mean pp and standard deviation pp. According to the mean number of photons per pixel pp, it is possible to calculate the probability P of np photons being detected over the exposure time tex (accounted for by pp). This distribution is defined by the relationship:
P (np)=e p. pn!
The distribution in figure 1 shows the Poisson distribution for various mean values of photons per pixel. Thus, for example, by using the Poisson distribution, for a mean flux of 0.15 photon per pixel, 86% of the pixels see no photons, 13% see one and 1% see more than one photon. Thus, in this particular case of the Poisson distribution with a mean of 0.15 photon per pixel (or less), then a statistical exploitation of the received signal may be constrained by knowing a priori that either no photon (0 photons) is received or only one is received in 99% of cases per period of exposure time. It is therefore possible to estimate the signal in a binary manner, being wrong only in less than 1% of cases for this example in which the average flux is lower than or equal to 0.15 photon per pixel. For higher average fluxes there will be an increasingly high error rate with confusion in situations where the pixel counts one photon when it has collected two or more. The above reasoning for the reception of photons by a pixel also applies to photoelectrons generated by the photoelectric detection process implemented in the detector. The analog signal at the output of the detector fluctuates and is dominated by the read noise and the dark noise for a pixel which has not generated a photoelectron, and the multiplicative noise for the pixels which have converted one or more photoelectrons. However, a priori knowledge of the number of photons received, less than 0.15 on average, constrains the output to two states: either zero photons were received or one was received, in 99% of cases. The fluctuating analog signal may therefore be replaced by binarizing an elementary frame or an elementary acquisition of the pixel as 0 or 1. This may be done for example by means of a thresholding algorithm: below a certain threshold, it is considered that zero photons have been received, the analog fluctuation of the initial signal being dominated by the various types of noise from the detector rather than by the true amplitude linked to the incidence of zero photons. Above the threshold, it is considered that the detection of one and only one photon has indeed taken place and this "unity" amplitude replaces the analog amplitude linked to the fluctuation of the read and multiplicative noise of the signal. However, binarizing the frame using this method leads to residual noise linked to the error in the photon count rate. This residual noise may be characterized by a counting error probability which may still be estimated and bounded using noise statistics and the Poisson distribution. This probability is broken down into the probability of counting one for zero incident photons, the probability of counting zero for one incident photon and the probability of counting one for two or more incident photons. Finally, the last step in photon-counting imaging consists in summing a plurality of binary elementary frames to reconstruct an image corresponding to the output image rendering rate, for example, for a video output rate of 25 Hz, namely 40 ms accumulation time of elementary frames obtained at higher rate. Summing the various binarized frames makes it possible to reconstruct an image with various levels of gray. The number of discernible shades of gray is no longer dominated by the dispersion of multiplication gains from the signal amplification process which is the source of the snow-like effect which dominates the image and its shades. It is important to note that if the scene flux increases above an average of 0.15 photon per elementary frame, then this constrained mode with zero or one photons is exited and there are an increasing number of cases where the pixel sees two or more photons. However, in that case, the amplification noise factor of the detector no longer makes it possible to estimate the exact number of photons received with a good degree of confidence. In other words, it is not possible to know whether one, two, three or more photons have been received. Information is lost. This is a substantial limitation of photon-counting imaging with currently available charge multiplication detectors. It is important to note that this amplification noise factor differs for the different types of detectors. In SPAD (single-photon avalanche diode) or Geiger-mode avalanche photodiode technologies, the absorption of a photon generates an enormous avalanche process. The current generated by the avalanche is detected by an electronic circuit which allows the photon to be detected. However, after such a process, a dead time is required for charge removal and therefore prevents a new photon from being detected during this removal time. It is not this type of technology which is preferred and presented here. For certain applications in astronomy (using for example a camera from Nv or from First Light Imaging) or in biomedical imaging, photon counting imaging is highly suitable because it is known that only a few photons will be received. However, for applications in which any night-time scene is viewed, it is not known in advance what lighting conditions might be encountered and what scene dynamic range will be encountered in the video image (spatially and temporally). If the scene flux is high when using photon counting imaging, then it is outside of the range of light levels which allows the signal from each pixel in the frame to be binarized; information is therefore lost. However, in this range, conventional imaging may provide sufficient quality because of the noise from the receiver being lower than Inp (standard deviation for a mean level of np (for example np > 10 photons per frame). However, if the scene flux is too low when conventional imaging is being used, noise from the detector will then dominate and the image will be unusable. Consequently, there remains to this day a need for a high dynamic-range imaging method and system, suitable for unforeseen lighting conditions which may vary from a high scene flux to a low scene flux. Any discussion of documents, acts, materials, devices, articles or the like which has been included in the present specification is not to be taken as an admission that any or all of these matters form part of the prior art base or were common general knowledge in the field relevant to the present disclosure as it existed before the priority date of each of the appended claims.
The imaging system embodying the invention comprises several imaging modes, with at least: m a conventional imaging mode used when the scene flux is high. This mode and the application conditions for this mode assume a high photon flux, higher than the square of the read noise from the detector and the dark current thereof. In this case, the detector will generate an image limited by photon noise. If the detector allows it (as in EMCCDs), the use of a signal amplification or multiplication mode is not necessary and in this case the S/N is typically close to the root of the photon flux accumulated per pixel and per frame. If the photon flux decreases and becomes close to the dark current or to the square of the read noise, then an amplification or multiplication process may be performed on the signal in order to limit the deterioration of the signal to-noise ratio. m a "natural" photon-counting imaging mode, allowing imaging at very low light levels. At very low flux, detectors are limited by the read noise from the receiver. In this case, it is worth using a photoelectron amplification or multiplication device. Because of this, in some detectors the multiplied photoelectric signal comes to dominate over the read noise of the array or even the dark current. However, the multiplication factor is stochastic and adds noise referred to as an amplification noise factor which will negatively affect the S/N. It has been shown that if this flux is lower than 0.15 photoelectron per pixel and per frame then threshold logic makes it possible to constrain the analog signal delivered by every pixel to 0 or 1. In this case, it is in what is defined as the "natural" photon-counting imaging mode. This mode allows the image quality, which can be defined by its S/N, to be improved and a greater dynamic range of levels of gray to be rendered. When the scene flux is intermediate, i.e. decreases without reaching a threshold for a very low light level suitable for "natural" photon counting, then the imaging system can be forced to switch to a specific "forced" photon-counting imaging mode, placing it under conditions allowing it to be such that it is constrained to zero or one photon per pixel for a frame. In this case, it is worth examining whether it makes sense to activate the signal amplification or multiplication function to decrease the influence of the read noise and the dark current of the sensor as seen in the conventional imaging mode. It is then in multiplied or amplified conventional imaging mode. However, in this multiplication mode there will be a deterioration in the S/N and in the dynamic range of the image with decreasing light level. It is then possible, paradoxically, to decrease the photon flux per pixel and per frame to enter a mode similar to natural photon counting in order to improve the image quality via its S/N and its dynamic range with respect to the multiplied mode. This defines the forced photon-counting imaging mode. As a first approximation, it may be considered that: - when the scene flux is lower than 0.15 photoelectron per pixel and per frame, the imaging mode is natural photon-counting mode; - when the scene flux is higher than a few (for example five) photoelectrons per pixel and per frame, the imaging mode is conventional mode; - when the scene flux is intermediate, the imaging mode is forced photon counting mode. The system is made to switch to an imaging mode by controlling certain parameters such as: m the integration time in order to have, for example, a mean of 0.15 photon/pixel/frame or less, if photon-counting imaging mode is chosen. This value of 0.15 may be adjustable and depends on the binarization algorithm chosen; it is possible to have for example 0.2 photon/pixel/frame by tolerating for example a slightly larger counting error; m the acquisition rate; m the numerical aperture of the optical system; m the optical attenuation; m the reception spectral band in order for example to improve spectral discrimination; m the pixel pitch or its fill factor if the technology allows it, for example in order to improve the angular resolution or the MTF; m etc. More specifically, one subject of the invention is a method for imaging a scene using an imaging system making it possible to obtain an image of the scene, the imaging system comprising a device for acquiring frames according to acquisition parameters, and a unit for processing the acquired frames, characterized in that the method comprises the following steps: A) selection of an imaging mode from among a conventional imaging mode, a natural photon-counting imaging mode and a forced photon counting imaging mode, and, depending on the selected imaging mode, determination by the processing unit of corresponding acquisitionparameters; B) acquisition of at least one frame by the acquisition device parametrized with said acquisition parameters, and transmission of the frames that were acquired to the processing unit in order to obtain an image, the image being obtained at the end of the following sub steps if the selected imaging mode is the natural photon-counting imaging mode or the forced photon-counting imaging mode: - binarization of the frames that were acquired and - summing the binarized frames to obtain an image; C) estimation of the quality of the image obtained; D) D1) depending on the quality of the image obtained, determining one or two new imaging modes selected from among the conventional imaging mode, the natural photon-counting imaging mode and the forced photon-counting imaging mode; D2) repeating steps A, B, C and D with the new imaging mode selected as the imaging mode. According to one embodiment, step D1) is carried out by comparing (first comparison) the quality of the image obtained with a first predetermined quality corresponding to the conventional imaging mode: - if the comparison (first comparison) is favorable, steps A, B, C and D are reiterated with the conventional imaging mode as the selected imaging mode; - otherwise the quality of the image obtained is compared (second comparison) with a second predetermined quality corresponding to the natural photon-counting imaging mode; and o if the comparison (second comparison) is favorable, steps A, B, C and D are reiterated with the natural photon-counting imaging mode as the selected imaging mode; o otherwise steps A, B, C and D are reiterated with the forced photon-counting imaging mode as the selected imaging mode. As recalled in the preamble, without a photon-counting mode, low light-level imagers are optimized, for very low fluxes, by increasing the area of the pixel and by increasing the integration time as far as is possible. According to the invention, introducing the (natural and forced) photon counting mode makes it possible to promote a decrease in pixel size and/or to decrease the integration time by increasing the frame rate so as to use this mode in a greater scene dynamic range. The parameters of the acquisition device are typically integration time and/or rate and/or optical system aperture and/or spectral band variation and/or optical attenuation variation parameters. The acquisition parameters in the conventional imaging mode and the parameters in the natural photon-counting imaging mode typically have fixed values, and the parameters in the forced photon-counting imaging mode typically have values that vary from one iteration to another in order to optimize the image quality criterion on each iteration. According to one feature, the method comprises a step of spatially dividing each frame into sub-frames and the steps for the imaging modes are applied to each sub-frame. The estimate of the quality of the image obtained may be determined by computation or by an operator. Additionally, such a method may advantageously be used in active imaging, i.e. with laser illumination. In this case, the parameters of the acquisition device may advantageously lead to decreasing the laser illumination or increasing the range thereof. Thus, according to another feature, the method comprises a prior step of illuminating the scene using an illumination device synchronized with the acquisition device and the illumination device and the acquisition device have the same spectral band. In this case, illumination parameters may also be defined in step A. The illumination parameters are for example illumination duration and/or power parameters of the illumination device. Another subject of the invention is a system for imaging a scene which comprises a device for acquiring frames of the scene, optionally a device for illuminating the scene, and a processing unit connected to the acquisition device and to the optional illumination device, which are configured to implement the imaging method such as described. Thus, instead of being subject to the scene flux received, the imaging system adjusts its operating parameters, such as its acquisition rate for example, so as to be in the imaging mode exhibiting the best performance. Throughout this specification the word "comprise", or variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated element, integer or step, or group of elements, integers or steps, but not the exclusion of any other element, integer or step, or group of elements, integers or steps. Other features and advantages of the invention will become apparent on reading the detailed description that follows, provided by way of non-limiting example and with reference to the appended drawings, in which: figure 1 schematically shows examples of Poisson distributions, in this instance histograms of photon fluxes acquired on an array, for different mean values pp; figure 2a schematically shows a first example of the imaging system according to the invention, and figure 2b schematically shows a second example of the imaging system according to the invention, with an active imaging system; figure 3 is an example of a flowchart showing different steps in the method according to the invention. An example of the imaging system 100 embodying the invention is described with reference to figure 2a. It comprises: " a device 1 for acquiring frames of the scene, comprising: o an optical device 12 for forming images on a focal plane; and o in the focal plane, a detection array 11 which generates the acquisition of frames; and " a processing unit 2 connected to the acquisition device 1. The imaging system may operate in a plurality of imaging modes, with at least: - a conventional imaging mode used when the scene flux is high;
- what is called a natural photon-counting imaging mode, allowing imaging at very low light levels, i.e. when the scene flux is low; - what is called a forced photon-counting imaging mode, allowing imaging when the scene flux is intermediate between high and low. Thus, when the scene flux is high, conventional imaging is retained. When the scene flux is low, a switch is made to natural photon counting imaging. However, when the scene flux is intermediate, then the imaging system is forced to switch to a specific forced photon-counting imaging mode, placing the imaging system under conditions allowing it to be such that it is constrained to zero or one photon per pixel for a frame: the imaging system is constrained by acting on one or more of its different parameters (integration time, rate, aperture, illumination for active imaging, etc.) in order to be able to be in the particular mode in which photon counting may be carried out advantageously from the point of view of image quality rendered. The imaging system is forced to switch to one or the other imaging mode by controlling the parameters of the acquisition device such as: - the acquisition rate of the detection array 11, or the acquisition period; - the integration time of the detection array 11; - the aperture of the image-forming device 12; - the optical attenuation or the spectral band variation by selecting a filter 13 from among filters with different attenuations or different spectral bandwidths, respectively. For example and preferably, it is possible to: m adjust the frame acquisition rate; m control the integration time in order to have a mean of 0.15 photon/pixel/frame or less, if photon-counting imaging mode is chosen. This value of 0.15 is dependent on the binarization function selected and on the error rate allowed in the rendered image. It is also possible to act on other parameters (for example: aperture, variable optical attenuation, illumination of the scene in active imaging, etc.) to artificially limit the flux from the scene and thus force the system to be in the photon-counting imaging mode. These parameters are controlled by the processing unit 2.
Thus, instead of being subject to the scene flux received, the imaging system according to the invention adjusts its operating parameters, such as its acquisition rate for example, so as to be in the imaging mode exhibiting the best performance. Switching between the various imaging modes is done according to the scene flux received or more precisely according to the quality of the image obtained with one or the other imaging mode, this quality being linked to the scene flux received. It may be done automatically by being carried out by the processing unit 2, or manually. It may be done for the entire detection array 11 (4 for each frame acquired) or only for part (4 sub-frame or window), or, if the array allows it, per pixel, which makes it possible to adjust the imaging mode to the flux received as precisely as possible. If the scene flux is relatively low, then the imaging system adjusts its parameters so as to remain in a photon-counting imaging mode. If the scene flux is too high and the system can no longer be forced to be in the specific photon-counting mode or if performance is better in conventional imaging mode, then a switch is made back to a conventional imaging mode in which photons are accumulated over an integration time equal to that of the final image (40 ms in the case of a 25 Hz video for example). In low-light-level imaging, to increase sensitivity or image quality, it is conventional to increase the integration time per frame and/or to increase pixel area and/or to increase the reception spectral band. According to the invention, to use photon-counting mode, the parameters of the acquisition device are controlled by favoring an increase in the frame rate and thus a decrease in the integration time, and a limiting of or even decrease in the spectral band. The method according to the invention also makes it possible to use a detector array the size of the pixels of which is decreased, even if it means using binning modes (as in CCDs) which allow summing of the charges on adjacent pixels (2x2, 3x3 or 4x4) for example to effectively increase the size of the pixels. Steps in an example of the imaging method according to the invention will be described with reference to figure 3.
A) Determination, by the processing unit 2, of the acquisition parameters of the acquisition device 1 depending on the selected imaging mode. The parameters of the conventional imaging mode and those of the natural counting imaging mode are predetermined and have predetermined values. Those of the parameters of the natural counting imaging mode correspond to the ultimate conditions of use of the acquisition device, corresponding for example to 0.15 photon/pixel or less for each frame. The parameters of the forced counting imaging mode are variable: the parameters may vary from one image to the next and their values are optimized from one image to the next so as to optimize the image quality criterion from one image to the next. They are generally optimized by the processing unit within a range of values, proceeding from one iteration to the next. These ranges of values lie between the values of the parameters in conventional mode and those of the parameters in natural counting mode. B) Acquisition of at least one frame by the acquisition device parametrized according to the currently used imaging mode and transmission of these frames to the processing unit as they are acquired. If the currently used imaging mode (selected in step A) is the conventional mode, an image is obtained on the basis of these frames by the processing unit in a conventional manner (acquisition of at least one frame and summing of the frames to obtain the final image). If the currently used imaging mode (selected in step A) is the natural counting or forced counting imaging mode, the following sub-steps are carried out by the processing unit: - Binarization of the acquired frames; - Summing the binarized frames to form an image. For the binarization of the acquired frames, a threshold binarization function may be used (1 is assigned to a pixel when the amplitude measured on this pixel is higher than a predetermined threshold, otherwise 0), but other binarization functions may be considered as described in the following publications: * - E. Lantz et al "Multi-imaging and Bayesian estimation for photon counting with EMCCDs" Mon. Not. R. Astron. Soc, 386, 2262-2270 (2008)
* - K.B.W. Harpsoe et al "Bayesian photon counting with EMCCDs", A&A, 537 (2012). At the output of the processing unit, the image obtained according to one or the other mode is sent for example to a display device. C) Estimation by the processing unit 2 of the quality of the image obtained. D) Comparison (first comparison) with a first predetermined image quality (quality 1) corresponding to an image quality in conventional mode, by the processing unit. If the comparison (1st comparison) is favorable with respect to the conventional imaging mode (the quality of the image obtained is for example given by the level of illumination of the scene measured in the visible spectral range between 100 000 lux and 0.01 lux, or less depending on the optical configuration and the performance of the detector), steps A, B, C and D are reiterated with the conventional imaging mode (which may be the currently used mode) selected. This illumination level criterion may also be implemented at the pupil entrance or on the focal plane, it may be measured in W/m2 or in photons/s/m2 in the spectral range of the filters 13, for example in the near IR or even in the SWIR. If the comparison (first comparison) is unfavorable with respect to the conventional imaging mode, the quality of the image obtained is compared (second comparison) with a second predetermined image quality (quality 2), corresponding to an image quality in natural counting mode, by means of the processing unit. If the comparison (second comparison) is favorable with respect to the natural counting mode (the quality of the image obtained is for example between illuminations of 1 mlux and 1 plux), steps A, B, C and D are reiterated with the natural counting mode (which may be the currently used mode) selected, otherwise (= the comparison (second comparison) is therefore favorable with respect to the forced counting mode with a quality of the image obtained for example for illuminations between 1 mlux and 10 mlux)), steps A, B, C and D are reiterated in forced counting mode (which may be the currently used mode) with therefore an optimization of the acquisition parameters in step A. When quality 1 and/or potentially quality 2 are not predetermined, it is possible to acquire new frames in two imaging modes (conventional and forced counting, or forced counting and natural counting) and the imaging mode which gives the best quality criterion is selected from the two. This amounts to defining these quality criteria relatively. As an example of quality criteria, a level of illumination incident on the scene or at the pupil entrance or in the focal plane has been given; it is of course possible to take other criteria known to those skilled in the art such as the (temporal or spatial) signal-to-noise ratio, or the dynamic range in levels of gray rendered. The initial imaging mode may be any mode. This initial mode is engaged without knowing the level of illumination of the scene flux received. In the example of figure 3, the conventional mode has been selected as the initial mode represented by a dashed arrow. However, if a (natural or forced) photon-counting imaging mode is chosen, the tests of comparing with qualities 1 and 2 are of course reversed in order to retain the acquisition parameters for as long as it is not necessary to change them. In other words, the method comprises the following steps: A) Selection of an imaging mode from among a conventional imaging mode, a natural photon-counting imaging mode and a forced photon counting imaging mode, and determination by the processing unit 2 of the acquisition parameters of the acquisition device 1 depending on the selected imaging mode. The parameters of the conventional imaging mode and those of the natural counting imaging mode are predetermined and have predetermined values. Those of the parameters of the natural counting imaging mode correspond to the ultimate conditions of use of the acquisition device, corresponding for example to 0.15 photon/pixel or less for each frame. The parameters of the forced counting imaging mode are variable: the parameters may vary from one image to the next and their values are optimized from one image to the next so as to optimize the image quality criterion from one image to the next. They are generally optimized by the processing unit within a range of values, proceeding from one iteration to the next. These ranges of values lie between the values of the parameters in conventional mode and those of the parameters in natural counting mode. B) Acquisition of at least one frame by the acquisition device parametrized according to the currently used imaging mode and transmission of these frames to the processing unit as they are acquired.
If the currently used imaging mode (selected in step A) is the conventional mode, an image is obtained on the basis of these frames by the processing unit in a conventional manner (acquisition of at least one frame and summing of the frames to obtain the final image). If the currently used imaging mode (selected in step A) is the natural counting or forced counting imaging mode, the following sub-steps are carried out by the processing unit: - binarization of the acquired frames; - summing the binarized frames to form an image. At the output of the processing unit, the image obtained according to one or the other mode is sent for example to a display device. C) Estimation by the processing unit 2 of the quality of the image obtained. D) D1) depending on the quality of the image obtained, determining one (or two as mentioned above) new imaging mode selected from among the conventional imaging mode, the natural photon-counting imaging mode, and the forced photon-counting imaging mode; D2) repeating steps A, B, C and D with the (or each) new imaging mode selected as the imaging mode. According to one variant, prior to the image-obtaining steps, the acquired frames are spatially divided into sub-frames or windows, this spatial division being able to go so far as to consider a fraction of the pixels in each frame (window, rows, columns or even given pixels). The steps in the different imaging modes are then applied to each sub-frame. This imaging method is in particular well suited to active imaging which then comprises a prior step of illuminating the scene using an illumination device (laser, LED, lamp) 200 shown in figure 2b which is generally synchronized with the acquisition device 1. The spectral band of the acquisition device will then be restricted to that of the illumination device and thus only the photons of the scene reflected in this band will be counted. It is also possible to decrease the illumination durations or power of the illumination device. The spectral band, the illumination durations or power are then illumination parameters also controlled by the processing unit 2.

Claims (11)

1. A method for imaging a scene using an imaging system making it possible to obtain an image of the scene, the imaging system comprising a device for acquiring frames according to acquisition parameters, and a unit for processing the acquired frames, wherein the method comprises the following steps: A) selection of an imaging mode from among a conventional imaging mode, a natural photon-counting imaging mode and a forced photon counting imaging mode, and, depending on the selected imaging mode, determination by the processing unit of corresponding acquisitionparameters; B) acquisition of at least one frame by the acquisition device parametrized with said acquisition parameters, and transmission of the frames that were acquired to the processing unit in order to obtain an image, the image being obtained at the end of the following sub steps if the selected imaging mode is the natural photon-counting imaging mode or the forced photon-counting imaging mode: - binarization of the frames that were acquired and - summing the binarized frames to obtain an image; C) estimation of the quality of the image obtained; D) D1) depending on the quality of the image obtained, determining one or two new imaging modes selected from among the conventional imaging mode, the natural photon-counting imaging mode or the forced photon-counting imaging mode; D2) repeating steps A, B, C and D with the new imaging mode selected as the imaging mode.
2. The method for imaging a scene as claimed in the preceding claim, wherein the new imaging mode selected in step D1) is obtained by making a first comparison comparing the quality of the image obtained with a first predetermined quality corresponding to the conventional imaging mode: - if the first comparison is favorable, the new imaging mode selected is the conventional imaging mode;
- otherwise a second comparison is performed comparing the quality of the image obtained with a second predetermined quality corresponding to the natural photon-counting imaging mode; and o if the second comparison is favorable, the new imaging mode selected is the natural photon-counting imaging mode; o otherwise the new imaging mode selected is the forced photon counting imaging mode, the quality of the image for first and second comparison being defined by at least one criteria based on image parameters.
3. The imaging method as claimed in either of the preceding claims, wherein the acquisition parameters are acquisition rate and/or integration time and/or aperture and/or spectral band variation and/or optical attenuation variation parameters.
4. The imaging method as claimed in any one of the preceding claims, wherein acquisition parameters in the conventional imaging mode and the parameters in the natural photon-counting imaging mode have fixed values, and in that the acquisition parameters in the forced photon-counting imaging mode have values that vary from one iteration to another.
5. The imaging method as claimed in any one of the preceding claims, wherein it comprises a step of spatially dividing each frame into sub frames and in that the steps for the imaging modes are applied to each sub-frame.
6. The imaging method as claimed in any one of the preceding claims, wherein the estimate of the quality of the image obtained is determined by computation or by an operator.
7. The imaging method as claimed in any one of the preceding claims, wherein it comprises a prior step of illuminating the scene using an illumination device synchronized with the acquisition device and in that the illumination device and the acquisition device have the same spectral band.
8. The imaging method as claimed in the preceding claim, wherein the illumination device has illumination parameters defined in step A.
9. The imaging method as claimed in the preceding claim, wherein the illumination parameters are illumination duration and/or power parameters of the illumination device.
10. A system for imaging a scene which comprises a device for acquiring frames and a unit for processing the acquired frames connected to the acquisition device, which are configured to implement the imaging method as claimed in any one of claims 1 to 6.
11.A system for imaging a scene which comprises a device for illuminating the scene, a device for acquiring frames and a unit for processing the acquired frames connected to the illumination device and to the acquisition device, which are configured to implement the imaging method as claimed in one of claims 7 to 9.
REPLACEMENT SHEET (RULE 26)
REPLACEMENT SHEET (RULE 26)
REPLACEMENT SHEET (RULE 26)
AU2018387689A 2017-12-19 2018-12-17 Method and system for imaging at high and low light levels Active AU2018387689B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
FR1701327 2017-12-19
FR1701327A FR3075544B1 (en) 2017-12-19 2017-12-19 HIGH AND LOW LEVEL IMAGING METHOD AND SYSTEM
PCT/EP2018/085234 WO2019121528A1 (en) 2017-12-19 2018-12-17 Method and system for imaging at high and low light levels

Publications (2)

Publication Number Publication Date
AU2018387689A1 AU2018387689A1 (en) 2020-07-09
AU2018387689B2 true AU2018387689B2 (en) 2022-10-20

Family

ID=62017303

Family Applications (1)

Application Number Title Priority Date Filing Date
AU2018387689A Active AU2018387689B2 (en) 2017-12-19 2018-12-17 Method and system for imaging at high and low light levels

Country Status (9)

Country Link
EP (1) EP3729796B1 (en)
AU (1) AU2018387689B2 (en)
CA (1) CA3086228A1 (en)
DK (1) DK3729796T3 (en)
ES (1) ES2909411T3 (en)
FR (1) FR3075544B1 (en)
RU (1) RU2765429C2 (en)
WO (1) WO2019121528A1 (en)
ZA (1) ZA202003682B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040094720A1 (en) * 2002-09-05 2004-05-20 Ofer Dagan Direct detection of high-energy single photons
EP2822270A1 (en) * 2012-02-27 2015-01-07 Sony Corporation Imaging element and electronic equipment
US20170208269A1 (en) * 2009-08-28 2017-07-20 Sony Corporation Imaging device and camera system including sense circuits to make binary decision

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6801258B1 (en) * 1998-03-16 2004-10-05 California Institute Of Technology CMOS integration sensor with fully differential column readout circuit for light adaptive imaging
US7091466B2 (en) * 2003-12-19 2006-08-15 Micron Technology, Inc. Apparatus and method for pixel binning in an image sensor
US9160912B2 (en) * 2012-06-08 2015-10-13 Apple Inc. System and method for automatic image capture control in digital imaging
US9661243B2 (en) * 2013-03-15 2017-05-23 Forza Silicon Corporation CMOS image sensor with column-wise selective charge-domain binning
US9686485B2 (en) * 2014-05-30 2017-06-20 Apple Inc. Pixel binning in an image sensor

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040094720A1 (en) * 2002-09-05 2004-05-20 Ofer Dagan Direct detection of high-energy single photons
US20170208269A1 (en) * 2009-08-28 2017-07-20 Sony Corporation Imaging device and camera system including sense circuits to make binary decision
EP2822270A1 (en) * 2012-02-27 2015-01-07 Sony Corporation Imaging element and electronic equipment

Also Published As

Publication number Publication date
RU2020120164A (en) 2022-01-20
FR3075544A1 (en) 2019-06-21
ES2909411T3 (en) 2022-05-06
RU2020120164A3 (en) 2022-01-20
EP3729796A1 (en) 2020-10-28
WO2019121528A1 (en) 2019-06-27
AU2018387689A1 (en) 2020-07-09
ZA202003682B (en) 2022-03-30
FR3075544B1 (en) 2020-01-17
RU2765429C2 (en) 2022-01-31
DK3729796T3 (en) 2022-05-09
EP3729796B1 (en) 2022-03-09
CA3086228A1 (en) 2019-06-27

Similar Documents

Publication Publication Date Title
US10616512B2 (en) Systems, methods, and media for high dynamic range imaging using dead-time-limited single photon detectors
CN108291961B (en) Solid-state imaging device, distance measuring device, and distance measuring method
Ingle et al. High flux passive imaging with single-photon sensors
JP5740413B2 (en) Demodulation sensor with separate pixel array and storage array
US8610808B2 (en) Color CMOS imager with single photon counting capability
Kurtz et al. Measuring diffuse, direct, and global irradiance using a sky imager
Wang et al. Photon counting based on solar-blind ultraviolet intensified complementary metal-oxide-semiconductor (ICMOS) for corona detection
EP3519855A1 (en) System for determining a distance to an object
Daigle et al. The darkest EMCCD ever
US10425589B2 (en) Adaptive XDR via reset and mean signal values
AU2018387689B2 (en) Method and system for imaging at high and low light levels
JP2019165447A (en) Solid-state imaging apparatus and imaging system
JP2022537117A (en) Method and system for extending image dynamic range using pixel-by-pixel encoding of pixel parameters
US20120033099A1 (en) Photo-detector and method for detecting an optical radiation
Mochizuki et al. Separation of multi-path components in sweep-less time-of-flight depth imaging with a temporally-compressive multi-aperture image sensor
US20220120873A1 (en) Time-of-Flight Device and 3D Optical Detector
EP4189636A1 (en) Method and system for imaging a scene in space
US20190098264A1 (en) Image Intensified Color Camera
Ralph et al. Smart imaging in the infrared
RU2657454C1 (en) Method for forming video signal in “ring” photosensor and server for computer system of panoramic observation in conditions of complex lighting and / or complex brightness of objects
RU2657456C1 (en) Method of forming a video signal in a “ring”; photosensor for computer system of panoramic television observation under conditions of complex lighting and / or complex brightness of objects
RU2657458C1 (en) Method of forming a video signal in a “ring”; photosensor for computer system of panoramic television observation under conditions of complex lighting and / or complex brightness of objects
RU2657459C1 (en) Method of forming a video signal in a “ring”; photosensor for computer system of panoramic television observation under conditions of complex lighting and / or complex brightness of objects
WO2022113028A1 (en) Methods and systems for infrared sensing
Accardo Tracring algorithm for star sensors using CMOS devices

Legal Events

Date Code Title Description
FGA Letters patent sealed or granted (standard patent)