US11417127B2 - Method for early observation of colonies of microorganisms - Google Patents

Method for early observation of colonies of microorganisms Download PDF

Info

Publication number
US11417127B2
US11417127B2 US16/845,123 US202016845123A US11417127B2 US 11417127 B2 US11417127 B2 US 11417127B2 US 202016845123 A US202016845123 A US 202016845123A US 11417127 B2 US11417127 B2 US 11417127B2
Authority
US
United States
Prior art keywords
image
sample
culture medium
image sensor
microorganisms
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US16/845,123
Other versions
US20200327306A1 (en
Inventor
Thomas Bordy
Cedric ALLIER
Pierre Marcoux
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Commissariat a lEnergie Atomique et aux Energies Alternatives CEA
Original Assignee
Commissariat a lEnergie Atomique et aux Energies Alternatives CEA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Commissariat a lEnergie Atomique et aux Energies Alternatives CEA filed Critical Commissariat a lEnergie Atomique et aux Energies Alternatives CEA
Publication of US20200327306A1 publication Critical patent/US20200327306A1/en
Assigned to COMMISSARIAT A L'ENERGIE ATOMIQUE ET AUX ENERGIES ALTERNATIVES reassignment COMMISSARIAT A L'ENERGIE ATOMIQUE ET AUX ENERGIES ALTERNATIVES ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: BORDY, THOMAS, MARCOUX, PIERRE, ALLIER, CEDRIC
Application granted granted Critical
Publication of US11417127B2 publication Critical patent/US11417127B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • C—CHEMISTRY; METALLURGY
    • C12—BIOCHEMISTRY; BEER; SPIRITS; WINE; VINEGAR; MICROBIOLOGY; ENZYMOLOGY; MUTATION OR GENETIC ENGINEERING
    • C12Q—MEASURING OR TESTING PROCESSES INVOLVING ENZYMES, NUCLEIC ACIDS OR MICROORGANISMS; COMPOSITIONS OR TEST PAPERS THEREFOR; PROCESSES OF PREPARING SUCH COMPOSITIONS; CONDITION-RESPONSIVE CONTROL IN MICROBIOLOGICAL OR ENZYMOLOGICAL PROCESSES
    • C12Q1/00—Measuring or testing processes involving enzymes, nucleic acids or microorganisms; Compositions therefor; Processes of preparing such compositions
    • C12Q1/02—Measuring or testing processes involving enzymes, nucleic acids or microorganisms; Compositions therefor; Processes of preparing such compositions involving viable microorganisms
    • C12Q1/04—Determining presence or kind of microorganism; Use of selective media for testing antibiotics or bacteriocides; Compositions containing a chemical indicator therefor
    • C12Q1/06—Quantitative determination
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/06—Investigating concentration of particle suspensions
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N15/14—Optical investigation techniques, e.g. flow cytometry
    • G01N15/1429—Signal processing
    • G01N15/1433—Signal processing using image recognition
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N15/14—Optical investigation techniques, e.g. flow cytometry
    • G01N15/1434—Optical arrangements
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N21/47—Scattering, i.e. diffuse reflection
    • G01N21/4788—Diffraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/693—Acquisition
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/01—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials specially adapted for biological cells, e.g. blood cells
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/06—Investigating concentration of particle suspensions
    • G01N15/075—Investigating concentration of particle suspensions by optical means
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/06—Investigating concentration of particle suspensions
    • G01N2015/0687—Investigating concentration of particle suspensions in solutions, e.g. non volatile residue
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N2015/1006—Investigating individual particles for cytology
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N15/14—Optical investigation techniques, e.g. flow cytometry
    • G01N15/1434—Optical arrangements
    • G01N2015/144—Imaging characterised by its optical setup
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N15/14—Optical investigation techniques, e.g. flow cytometry
    • G01N15/1434—Optical arrangements
    • G01N2015/1454—Optical arrangements using phase shift or interference, e.g. for improving contrast
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03H—HOLOGRAPHIC PROCESSES OR APPARATUS
    • G03H1/00—Holographic processes or apparatus using light, infrared or ultraviolet waves for obtaining holograms or for obtaining an image from them; Details peculiar thereto
    • G03H1/04—Processes or apparatus for producing holograms
    • G03H1/0443—Digital holography, i.e. recording holograms with digital recording means
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03H—HOLOGRAPHIC PROCESSES OR APPARATUS
    • G03H1/00—Holographic processes or apparatus using light, infrared or ultraviolet waves for obtaining holograms or for obtaining an image from them; Details peculiar thereto
    • G03H1/0005—Adaptation of holography to specific applications
    • G03H2001/005—Adaptation of holography to specific applications in microscopy, e.g. digital holographic microscope [DHM]
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03H—HOLOGRAPHIC PROCESSES OR APPARATUS
    • G03H1/00—Holographic processes or apparatus using light, infrared or ultraviolet waves for obtaining holograms or for obtaining an image from them; Details peculiar thereto
    • G03H1/04—Processes or apparatus for producing holograms
    • G03H1/0443—Digital holography, i.e. recording holograms with digital recording means
    • G03H2001/0447—In-line recording arrangement
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03H—HOLOGRAPHIC PROCESSES OR APPARATUS
    • G03H1/00—Holographic processes or apparatus using light, infrared or ultraviolet waves for obtaining holograms or for obtaining an image from them; Details peculiar thereto
    • G03H1/04—Processes or apparatus for producing holograms
    • G03H1/0443—Digital holography, i.e. recording holograms with digital recording means
    • G03H2001/045—Fourier or lensless Fourier arrangement

Definitions

  • the technical field of the invention is the observation of colonies of microorganisms, and in particular colonies developing in a nontransparent culture medium.
  • Observation of bacterial colonies by imaging is a technique that has been known for a long time in the field of microbiology, for monitoring the development of microorganisms or cells. It is possible to monitor the development of colonies, for example bacterial colonies, and count them, in a Petri dish. The form of the colonies provides information on the type of microorganism. Furthermore, by combining the use of different culture media, which either allow or do not allow colonies to develop, it is possible to identify the type of colony-forming microorganism.
  • Characterization of microorganisms on Petri dishes is still a reference method, often used in the field of microbiology, diagnostics, but also in the field of agriculture and food production or cosmetics.
  • the main drawback of this method is that it is slow, since it is generally necessary to wait several days to obtain a usable result.
  • Another drawback is that this method is difficult to automate, and requires experienced human operators.
  • U.S. Pat. No. 7,465,560 describes a method for characterizing a microorganism based on the exploitation of diffusion and diffraction of an incident laser beam by the microorganism.
  • the microorganism is arranged between a laser light source and an image sensor. Under the effect of illumination by the laser beam, an image is acquired on which diffraction patterns appear, the latter constituting a signature of the microorganism observed.
  • U.S. Pat. No. 8,787,633 describes a method having the same objective.
  • the holographic methods described above become inapplicable if the medium in which the microorganisms are disposed is opaque, colored or diffusing.
  • these methods use an image formed according to a so-called transmission configuration, in which the sample is arranged between a light source and an image sensor. Obtaining a usable image is subject to the use of a sufficiently transparent sample.
  • this method is not compatible with samples comprising a colored and/or diffusing culture medium, for example the medium known by the acronym COS (Columbia Sheep Blood) comprising a Columbia agar with sheep blood. It is also not applicable to an opaque, diffusing medium of the chocolate agar type. Now, culture media of this kind are often used in clinical diagnostics.
  • the documents WO2018122504 and WO2018122505 describe a method of bacterial identification according to a backscattering configuration: a light source illuminates a colony and an image sensor acquires an image based on radiation reflected by the bacterial colony. This method is very suitable for identifying colonies, but it does not allow efficient counting of the colonies distributed in a sample, except by scanning the sample.
  • the inventors have designed a particularly quick optical method for counting colonies of microorganisms, allowing early characterization of samples.
  • the invention relates to a method for observation of a sample, the sample comprising microorganisms immersed in a nontransparent culture medium, the culture medium being favorable to the development of the microorganisms, the sample being arranged between a light source and an image sensor, the method comprising:
  • the method being characterized in that the culture medium extends, parallel to the axis of propagation, to a thickness of less than 500 ⁇ m.
  • the culture medium may extend, parallel to the axis of propagation, to a thickness of less than 250 ⁇ m or less than 100 ⁇ m.
  • the culture medium extends between an upper face and a lower face, perpendicular or approximately perpendicular to the axis of propagation. As they develop, the microorganisms form colonies, at least one colony forming a light channel extending from the upper face to the lower face, through the culture medium, in such a way that at least one colony forms a light spot on the image acquired by the image sensor.
  • the method may comprise:
  • Light spot means a point zone, comprising for example some tens or some hundreds of pixels, whose intensity is greater than that of the pixels adjacent to the point zone.
  • the method may comprise:
  • Identification of a colony means determination of the species of microorganisms forming the colony.
  • the image acquired comprises at least one diffraction pattern, associated with a colony of microorganisms.
  • the method then comprises:
  • the method comprises:
  • the neural network may have been parameterized beforehand in a learning phase, using samples comprising microorganisms whose position is known, and whose species is preferably known.
  • the method comprises:
  • no image forming lens is arranged between the sample and the image sensor.
  • the method being such that during image acquisition:
  • Image-forming optical system means an objective or a lens.
  • the method being such that during image acquisition the lower face of the sample corresponds to the object plane and the detection plane corresponds to the image plane.
  • FIGS. 1A and 1B show an example of a device for implementing the invention, according to a lensless imaging configuration.
  • FIG. 1C shows a device of the invention in which an optical system extends between the sample and the image sensor.
  • FIG. 2 shows parts of images of a sample acquired at different time points after an initial time point.
  • FIGS. 3A and 3B show images of a sample acquired at an initial time point and 4 hours after the initial time point, respectively.
  • FIG. 4 shows the variation of the optical attenuation of an agar as a function of its thickness.
  • FIG. 5A shows schematically an architecture of a convolutional neural network.
  • FIG. 5B is an input thumbnail used during neural network training.
  • FIG. 5C is an output thumbnail used during neural network training.
  • FIG. 6A is an image comprising diffraction patterns.
  • FIG. 6B represents detection of diffraction patterns from the image in FIG. 6A .
  • FIGS. 1A and 1B show an example of a device for applying a method according to the invention.
  • a light source 11 is configured for emitting a light wave 12 , called incident light wave, propagating in the direction of a sample 10 , along an axis of propagation Z.
  • the light wave is emitted according to an illumination spectral band ⁇ .
  • the sample 10 comprises microorganisms 10 i that we wish to detect, in order to count them or characterize them, for example identifying them.
  • the microorganisms are immersed in a medium 10 m , forming a culture medium that is favorable to the development of the microorganisms, and in particular to colonies of microorganisms.
  • the medium 10 m comprises nutrients allowing development of the microorganisms.
  • “Microorganism” means notably a yeast, a bacterium, a spore, a fungus or a cell, whether it is a eukaryotic or a prokaryotic cell, or a microalga.
  • the method does not require microorganisms staining, such as viability staining.
  • the sample does not comprise a viability stain.
  • the concentration may for example be less than 1000 colonies of microorganisms per mm 2 .
  • the sample is confined in a confinement chamber 15 .
  • the latter is held, between the light source 11 and an image sensor 16 , by a sample holder 10 s.
  • the distance D between the light source 11 and the confinement chamber 15 is preferably greater than 1 cm. It is preferably between 2 and 30 cm.
  • the light source viewed by the sample, is considered to be a point source. This signifies that its diameter (or its diagonal) is preferably less than a tenth, better still less than a hundredth of the distance between the confinement chamber 15 and the light source.
  • the light source 11 is a light-emitting diode. It is generally combined with a diaphragm 18 , or spatial filter.
  • the aperture of the diaphragm is typically between 5 ⁇ m and 1 mm, preferably between 50 ⁇ m and 500 ⁇ m.
  • the diaphragm is supplied by Thorlabs under reference P150S and its diameter is 150 ⁇ m.
  • the diaphragm may be replaced with an optical fiber, a first end of which is placed facing the light source 11 and a second end is placed opposite the sample 10 .
  • the device shown in FIGS. 1A and 1B also comprises a diffuser 17 , arranged between the light source 11 and the diaphragm 18 .
  • the function of said diffuser is to distribute the light beam produced by a unit light source 11 according to a cone of angle ⁇ .
  • the diffusion angle ⁇ varies between 10° and 80°.
  • the use of a diffuser is described in WO2016078946.
  • the light source may be a laser source, such a laser diode. In this case, it is not useful to combine it with a spatial filter or a diffuser.
  • the emission spectral band ⁇ of the incident light wave 12 has a width less than 100 nm.
  • “Spectral band width” means the full width at half maximum of said spectral band.
  • the sample 10 is arranged between the light source 11 and the aforementioned image sensor 16 .
  • the latter extends preferably parallel, or approximately parallel to a plane over which the sample extends.
  • the term “approximately parallel” signifies that the two elements need not be strictly parallel, an angular tolerance of some degrees, less than 20° or 10° being allowed.
  • the sample extends over a plane XY, perpendicular to the axis of propagation Z.
  • the sample extends between an upper face 10 sup , located opposite the light source 11 , and a lower face 10 inf , located opposite the image sensor 16 .
  • the upper and lower faces preferably extend perpendicularly or sensibly perpendicularly to the axis of propagation Z of the incident light wave 12 .
  • “Sensibly perpendicularly” means perpendicular, allowing an angular tolerance of ⁇ 20° or ⁇ 10°.
  • the image sensor 16 is able to form an image I of the sample 10 on a detection plane P.
  • a detection plane P In the example shown, it is an image sensor comprising a pixel matrix, of the CCD type or a CMOS.
  • the detection plane P preferably extends perpendicularly to the axis of propagation Z of the incident light wave 12 .
  • the image sensor 16 is connected to a processing unit 20 , configured for processing the images obtained by the image sensor.
  • the processing unit 20 is connected to a memory 22 , configured to allow the execution of operations of image processing.
  • the processing unit 20 is also connected to a screen 24 .
  • the distance d between the sample 10 and the pixel matrix of the image sensor 16 is preferably between 50 ⁇ m and 2 cm, preferably between 100 ⁇ m and 2 mm.
  • an image-forming optical system 19 may be arranged between the sample and the image sensor, as shown in FIG. 1C .
  • the optical system may be an objective and a lens.
  • the optical system 19 defines an object plane P o and an image plane P i .
  • the object plane is preferably located in the sample, and in particular at the level of the lower face 10 inf of the sample.
  • the image plane P i preferably coincides with the detection plane P.
  • the magnification conferred by the optical system 19 is preferably equal to 1, or less than 1.
  • the magnification of the optical system 19 may be greater than 1.
  • the configuration shown in FIG. 1C is a focused configuration, the optical system conjugating the sample, and preferably the lower face 10 i , to the image sensor 16 .
  • the image defocusing distance and/or the object defocusing distance are preferably less than 1 mm, or even less than 500 ⁇ m.
  • the culture medium is a liquid medium or an agar medium commonly used in microbiology.
  • the culture medium may be:
  • the microorganisms may be seeded in the culture medium at a depth or on the surface.
  • the thickness e of the sample corresponds to the thickness of the culture medium 10 m . It is less than 1 mm, and preferably less than 500 ⁇ m, and more preferably less than 250 ⁇ m, or even less than 100 ⁇ m.
  • the thickness e is generally above 10 or 20 ⁇ m. At this thickness, the culture medium 10 m becomes translucent. “Translucent” means that it allows propagation of light through its thickness, without being transparent. Relative to the prior art, the reduction in thickness e makes it possible to form an image of the sample according to a transmission configuration, the sample extending between the light source 11 and the image sensor 16 .
  • the reduction in thickness of the culture medium 10 m is far from obvious.
  • the methods of observation and counting of the prior art assume that the colonies have acquired a certain level of development, which assumes a certain length of time between seeding the culture medium and observing the bacterial colonies. This time is generally more than 1 day, or even more than several days. This justifies the use of a sufficiently thick culture medium, so as to contain an amount of nutrients necessary for proliferation of the microorganisms.
  • the methods of the prior art would not be usable, as the amount of nutrients stored in the culture medium would not allow a sufficient culture time.
  • the inventors have shown that by reducing the thickness of the culture medium, images can be formed in transmission. These images are usable and allow observation of the colonies at an early stage of development. Because of this, the culture time can be limited to a few hours, making it unnecessary to use a thick culture medium. The amount of nutrient necessary for the development of the colonies, prior to their characterization, is thus considerably reduced relative to the techniques of the prior art. Examples of observation of colonies are presented in conjunction with FIGS. 2, 3A and 3B .
  • the sample 10 is, in this example, contained in a confinement chamber 15 .
  • the confinement chamber 15 is preferably transparent. It comprises an upper wall 15 sup , a lower wall 15 inf and a peripheral wall 15 p .
  • the peripheral wall 15 p extends between the upper wall 15 sup and the lower wall 15 inf .
  • the peripheral wall is an annular wall extending around the axis of propagation Z.
  • the lower wall 15 inf and upper wall 15 sup are preferably perpendicular, or approximately perpendicular, to the axis of propagation Z.
  • the upper wall 15 sup and lower wall 15 inf are preferably arranged in contact with the upper 10 sup and lower 10 inf faces of the sample 10 , respectively.
  • the culture medium 10 m may comprise a certain content of dissolved oxygen, which allows, to a certain extent, development by the aerobic route.
  • FIG. 2 shows images acquired by the image sensor using a device according to a lensless imaging configuration, as shown in FIGS. 1A and 1B .
  • a sample comprising a culture medium 10 m of the horse blood agar type, obtained by mixing defibrinated horse blood with a TSA agar melted at 55° C. (trypticase soy agar supplied by VWR), the volume fraction of blood added being 5%, or 3 mL to 60 mL of TSA agar.
  • the agar was seeded with a 600 ⁇ L solution containing bacteria of the Escherichia coli type at a concentration of 950 CFU (Colony Forming Units)/mL. Seeding took place while the agar was still liquid.
  • the seeded agar was poured into a Geneframe chamber with thickness varying between 250 ⁇ m and 500 ⁇ m.
  • the light source comprises quadrants emitting in a red (635 nm), green (520 nm) and blue (435 nm) illumination spectral band, respectively.
  • the illumination spectral bands were used separately.
  • the sample was illuminated by means of the light source, in the 520 nm spectral band. An image of the sample was acquired every 10 minutes, using the image sensor.
  • the diffraction pattern is formed by interference between:
  • a diffraction pattern generally has a central spot, around which concentric rings extend.
  • Detection, on the image formed by the image sensor, of these diffraction patterns makes it possible to detect development of bacterial colonies, and perform a count of the latter, at a particularly early stage of development.
  • each colony may be associated with a diffraction pattern.
  • the morphology of the diffraction pattern may allow identification of each colony.
  • the morphology of the diffraction patterns depends on the type of microorganism forming the colony, as described in the prior art.
  • the presence of diffraction patterns on the image acquired by the image sensor makes it possible to obtain qualitative and quantitative information at a very early stage of development, for example less than 2 hours after seeding.
  • the diffraction patterns, as described above, may also be observed according to a defocused configuration, as described above.
  • the defocusing distance is then preferably less than 1 mm, or less than 500 ⁇ m, or even less than 200 ⁇ m or 100 ⁇ m.
  • the bacterial colonies Owing to the small thickness of the culture medium, the bacterial colonies, as they develop, extend from the upper face 10 sup of the sample to the lower face 10 inf of the latter. They then form a channel 10 c extending through the culture medium 10 m , the channel joining the upper face 10 sup to the lower face 10 inf .
  • the channel 10 c forms a light guide, allowing easy propagation of a part 12 ′′ i of the incident light wave, propagating locally at the level of a colony 10 i through the light guide formed by the channel 10 c .
  • the exposure light wave 14 arriving at the image sensor 16 , comprises:
  • each colony forms an intense spot on the image. This allows particularly easy detection and counting of the colonies formed in the sample.
  • FIGS. 3A and 3B were acquired with a lensless configuration, as described in FIGS. 1A and 1B . Similar images, or even with higher contrast and higher resolution, may be obtained with an optical system 19 as described in FIG. 1C , according to a focused configuration. In this case, the object plane P o of the optical system 19 merges with the lower face 10 inf of the sample, whereas the image plane P i of the optical system 19 merges with the detection plane P.
  • FIG. 4 shows the evolution of the light attenuation of the blood agar described above, as a function of the thickness. This figure was obtained using an unseeded blood agar, varying in thickness between 75 ⁇ m and 500 ⁇ m. The attenuation formed by the agar was evaluated relative to measurement of luminous intensity I o without agar between the light source and the image sensor.
  • the attenuation Att e corresponding to the thickness e, is obtained from the following expression, derived from the Beer Lambert law:
  • Att e - ln ⁇ ( I e I 0 ) ( 1 ) where I e is the luminous intensity measured in the presence of a thickness e of agar.
  • the measurements shown in FIG. 4 were carried out at a wavelength of 540 nm.
  • the invention may be carried out provided that the attenuation, as defined in connection with equation (1), is below 0.5, and preferably below 0.4 or even 0.3.
  • the inventors consider that the method allows detection and counting of colonies whose diameter is between 5 ⁇ m and 50 ⁇ m, or more.
  • the method allows particularly early detection of the colonies.
  • counting of diffraction patterns is carried out using an automatic recognition algorithm, of the neural network type. More precisely, the algorithm is able to employ a convolutional neural network.
  • the neural network comprises an input layer IN, formed from an image acquired by the image sensor, and an output layer OUT, corresponding to an image on which the colonies of microorganisms are detected. Between the input layer IN and the output layer OUT, the neural network comprises 20 layers L1, L2 . . . L20, with ranks between 1 (layer adjacent to the layer IN) and 20 (layer adjacent to the layer OUT). Each layer comprises 32 planes. A layer is obtained by convolution of the 32 planes of the layer of preceding rank by a convolution kernel of size 3 ⁇ 3. The layer IN is regarded as the layer of rank 0.
  • FIG. 5A shows schematically an architecture of such a network.
  • the convolutional neural network has been the subject of training, considering 1000 input thumbnails, each input thumbnail having a size of 121 ⁇ 121 pixels.
  • the input thumbnails were taken at random from a hologram image.
  • An input thumbnail of this kind is shown in FIG. 5B .
  • an output thumbnail was determined by an expert.
  • the output thumbnail shows the localization of the microorganisms.
  • An example of an output thumbnail is shown in FIG. 5C . Training made it possible to parameterize the neural network.
  • FIG. 6A is an example of an image acquired by the image sensor. Each diffraction pattern corresponds to colonies of bacteria of the Escherichia coli type. FIG. 6B shows automatic detection of the colonies, each colony detected being represented by an open circle.
  • a neural network may allow for detection, counting and identification of the microorganisms.
  • the invention may be used for carrying out checking and counting of bacteria, as an aid in medical diagnostics, or in the field of environmental monitoring, or else control of industrial processes, for example in the field of food processing or cosmetics.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Analytical Chemistry (AREA)
  • Immunology (AREA)
  • Biochemistry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Organic Chemistry (AREA)
  • Biomedical Technology (AREA)
  • Pathology (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Zoology (AREA)
  • Wood Science & Technology (AREA)
  • Software Systems (AREA)
  • Proteomics, Peptides & Aminoacids (AREA)
  • Dispersion Chemistry (AREA)
  • Biotechnology (AREA)
  • Genetics & Genomics (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Microbiology (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Biophysics (AREA)
  • Toxicology (AREA)
  • Signal Processing (AREA)
  • Measuring Or Testing Involving Enzymes Or Micro-Organisms (AREA)
  • Apparatus Associated With Microorganisms And Enzymes (AREA)

Abstract

A method for observation of a sample, the sample comprising microorganisms immersed in a nontransparent culture medium, the culture medium being favorable to the development of the microorganisms, and the sample being arranged between a light source and an image sensor, includes illuminating the sample with the light source, the light source emitting light propagating along an axis of propagation; acquiring an image of the sample by the image sensor; and, from the image acquired, characterizing the microorganisms. Light travels along the microorganisms through the culture medium to the sensor.

Description

TECHNICAL FIELD
The technical field of the invention is the observation of colonies of microorganisms, and in particular colonies developing in a nontransparent culture medium.
PRIOR ART
Observation of bacterial colonies by imaging is a technique that has been known for a long time in the field of microbiology, for monitoring the development of microorganisms or cells. It is possible to monitor the development of colonies, for example bacterial colonies, and count them, in a Petri dish. The form of the colonies provides information on the type of microorganism. Furthermore, by combining the use of different culture media, which either allow or do not allow colonies to develop, it is possible to identify the type of colony-forming microorganism.
Characterization of microorganisms on Petri dishes is still a reference method, often used in the field of microbiology, diagnostics, but also in the field of agriculture and food production or cosmetics. The main drawback of this method is that it is slow, since it is generally necessary to wait several days to obtain a usable result. Another drawback is that this method is difficult to automate, and requires experienced human operators.
Recently, holographic methods have made it possible to count and characterize microorganisms, and they represent promising alternatives to the existing techniques, allowing better automation. For example, U.S. Pat. No. 7,465,560 describes a method for characterizing a microorganism based on the exploitation of diffusion and diffraction of an incident laser beam by the microorganism. The microorganism is arranged between a laser light source and an image sensor. Under the effect of illumination by the laser beam, an image is acquired on which diffraction patterns appear, the latter constituting a signature of the microorganism observed. U.S. Pat. No. 8,787,633 describes a method having the same objective.
Moreover, the work by Meada Y. “Colony fingerprint for discrimination of microbial species based on lensless imaging of microcolonies”, PLoS ONE 12(4) (2017), describes a method for observation of bacterial colonies according to a lensless imaging configuration. The colonies develop in LB Agar (Lysogeny Broth Agar) with a thickness of 600 μm. This agar is transparent, which allows correct observation of the colonies.
However, the holographic methods described above become inapplicable if the medium in which the microorganisms are disposed is opaque, colored or diffusing. In fact, these methods use an image formed according to a so-called transmission configuration, in which the sample is arranged between a light source and an image sensor. Obtaining a usable image is subject to the use of a sufficiently transparent sample. Thus, this method is not compatible with samples comprising a colored and/or diffusing culture medium, for example the medium known by the acronym COS (Columbia Sheep Blood) comprising a Columbia agar with sheep blood. It is also not applicable to an opaque, diffusing medium of the chocolate agar type. Now, culture media of this kind are often used in clinical diagnostics.
The documents WO2018122504 and WO2018122505 describe a method of bacterial identification according to a backscattering configuration: a light source illuminates a colony and an image sensor acquires an image based on radiation reflected by the bacterial colony. This method is very suitable for identifying colonies, but it does not allow efficient counting of the colonies distributed in a sample, except by scanning the sample.
The inventors have designed a particularly quick optical method for counting colonies of microorganisms, allowing early characterization of samples.
DISCLOSURE OF THE INVENTION
The invention relates to a method for observation of a sample, the sample comprising microorganisms immersed in a nontransparent culture medium, the culture medium being favorable to the development of the microorganisms, the sample being arranged between a light source and an image sensor, the method comprising:
a) illuminating the sample with the light source, the light source emitting an incident light wave propagating along an axis of propagation;
b) acquiring an image of the sample by the image sensor; and
c) from the image acquired, detecting the microorganisms;
the method being characterized in that the culture medium extends, parallel to the axis of propagation, to a thickness of less than 500 μm.
The culture medium may extend, parallel to the axis of propagation, to a thickness of less than 250 μm or less than 100 μm.
According to one embodiment, the culture medium extends between an upper face and a lower face, perpendicular or approximately perpendicular to the axis of propagation. As they develop, the microorganisms form colonies, at least one colony forming a light channel extending from the upper face to the lower face, through the culture medium, in such a way that at least one colony forms a light spot on the image acquired by the image sensor. According to this embodiment, the method may comprise:
    • counting light spots formed on the image acquired by the image sensor; and
    • estimating the number of colonies in the sample, as a function of the number of light spots counted on the image.
“Light spot” means a point zone, comprising for example some tens or some hundreds of pixels, whose intensity is greater than that of the pixels adjacent to the point zone.
The method may comprise:
    • performing a morphological analysis of at least one light spot formed on the image acquired by the image sensor; and
    • identifying the colony that produced the light spot, based on the morphological analysis.
“Identification of a colony” means determination of the species of microorganisms forming the colony.
According to one embodiment, the image acquired comprises at least one diffraction pattern, associated with a colony of microorganisms. The method then comprises:
    • counting each diffraction pattern, so as to estimate a quantity of colonies in the sample;
    • and/or morphological analysis of at least one diffraction pattern, as well as identification of the colony associated with the diffraction pattern, based on the morphological analysis.
According to one embodiment, the method comprises:
    • employing a neural network, based on the image acquired, for detecting the diffraction patterns; and
    • counting the diffraction patterns thus detected.
The neural network may have been parameterized beforehand in a learning phase, using samples comprising microorganisms whose position is known, and whose species is preferably known.
According to one embodiment, the method comprises:
    • acquisition of a first image, at a first time point, the image comprising at least one diffraction pattern associated with a colony of microorganisms; and
    • acquisition of a second image, at a second time point, subsequent to the first time point, the second image comprising at least one light spot associated with the colony of microorganisms.
According to one embodiment:
    • the culture medium extends in a confinement chamber;
    • the culture medium extends between two opposite faces, perpendicular or approximately perpendicular to the axis of propagation: and
    • the confinement chamber is in contact with the culture medium at the level of the two opposite faces.
According to one embodiment, no image forming lens is arranged between the sample and the image sensor.
According to one embodiment,
    • an image-forming optical system is arranged between the sample and the image sensor, the optical system defining an object plane and an image plane; and
    • the image sensor defines a detection plane;
the method being such that during image acquisition:
    • the sample is displaced relative to the object plane by an object defocusing distance;
    • and/or the detection plane is displaced, relative to the image plane, by an image defocusing distance.
“Image-forming optical system” means an objective or a lens.
According to one embodiment,
    • an image-forming optical system is arranged between the sample and the image sensor, the optical system defining an object plane and an image plane; and
    • the image sensor defines a detection plane;
the method being such that during image acquisition the lower face of the sample corresponds to the object plane and the detection plane corresponds to the image plane.
The invention will be better understood on reading the account of embodiment examples presented hereunder in conjunction with the figures listed below.
FIGURES
FIGS. 1A and 1B show an example of a device for implementing the invention, according to a lensless imaging configuration.
FIG. 1C shows a device of the invention in which an optical system extends between the sample and the image sensor.
FIG. 2 shows parts of images of a sample acquired at different time points after an initial time point.
FIGS. 3A and 3B show images of a sample acquired at an initial time point and 4 hours after the initial time point, respectively.
FIG. 4 shows the variation of the optical attenuation of an agar as a function of its thickness.
FIG. 5A shows schematically an architecture of a convolutional neural network.
FIG. 5B is an input thumbnail used during neural network training.
FIG. 5C is an output thumbnail used during neural network training.
FIG. 6A is an image comprising diffraction patterns.
FIG. 6B represents detection of diffraction patterns from the image in FIG. 6A.
 DESCRIPTION OF PARTICULAR EMBODIMENTS
FIGS. 1A and 1B show an example of a device for applying a method according to the invention. A light source 11 is configured for emitting a light wave 12, called incident light wave, propagating in the direction of a sample 10, along an axis of propagation Z. The light wave is emitted according to an illumination spectral band Δλ.
The sample 10 comprises microorganisms 10 i that we wish to detect, in order to count them or characterize them, for example identifying them. The microorganisms are immersed in a medium 10 m, forming a culture medium that is favorable to the development of the microorganisms, and in particular to colonies of microorganisms. The medium 10 m comprises nutrients allowing development of the microorganisms. “Microorganism” means notably a yeast, a bacterium, a spore, a fungus or a cell, whether it is a eukaryotic or a prokaryotic cell, or a microalga.
The method does not require microorganisms staining, such as viability staining. In a preferred embodiment, the sample does not comprise a viability stain.
When the microorganisms form colonies, the concentration may for example be less than 1000 colonies of microorganisms per mm2. The sample is confined in a confinement chamber 15. The latter is held, between the light source 11 and an image sensor 16, by a sample holder 10 s.
The distance D between the light source 11 and the confinement chamber 15 is preferably greater than 1 cm. It is preferably between 2 and 30 cm. Advantageously, the light source, viewed by the sample, is considered to be a point source. This signifies that its diameter (or its diagonal) is preferably less than a tenth, better still less than a hundredth of the distance between the confinement chamber 15 and the light source. In FIGS. 1A and 1B, the light source 11 is a light-emitting diode. It is generally combined with a diaphragm 18, or spatial filter. The aperture of the diaphragm is typically between 5 μm and 1 mm, preferably between 50 μm and 500 μm. In this example, the diaphragm is supplied by Thorlabs under reference P150S and its diameter is 150 μm. The diaphragm may be replaced with an optical fiber, a first end of which is placed facing the light source 11 and a second end is placed opposite the sample 10. The device shown in FIGS. 1A and 1B also comprises a diffuser 17, arranged between the light source 11 and the diaphragm 18. The use of said diffuser makes it possible to overcome constraints on centering of the light source 11 relative to the aperture of the diaphragm 18. The function of said diffuser is to distribute the light beam produced by a unit light source 11 according to a cone of angle α. Preferably, the diffusion angle α varies between 10° and 80°. The use of a diffuser is described in WO2016078946. Alternatively, the light source may be a laser source, such a laser diode. In this case, it is not useful to combine it with a spatial filter or a diffuser.
Preferably, the emission spectral band Δλ of the incident light wave 12 has a width less than 100 nm. “Spectral band width” means the full width at half maximum of said spectral band.
The sample 10 is arranged between the light source 11 and the aforementioned image sensor 16. The latter extends preferably parallel, or approximately parallel to a plane over which the sample extends. The term “approximately parallel” signifies that the two elements need not be strictly parallel, an angular tolerance of some degrees, less than 20° or 10° being allowed. In this example, the sample extends over a plane XY, perpendicular to the axis of propagation Z.
The sample extends between an upper face 10 sup, located opposite the light source 11, and a lower face 10 inf, located opposite the image sensor 16. The upper and lower faces preferably extend perpendicularly or sensibly perpendicularly to the axis of propagation Z of the incident light wave 12. “Sensibly perpendicularly” means perpendicular, allowing an angular tolerance of ±20° or ±10°.
The image sensor 16 is able to form an image I of the sample 10 on a detection plane P. In the example shown, it is an image sensor comprising a pixel matrix, of the CCD type or a CMOS. The detection plane P preferably extends perpendicularly to the axis of propagation Z of the incident light wave 12.
The image sensor 16 is connected to a processing unit 20, configured for processing the images obtained by the image sensor. The processing unit 20 is connected to a memory 22, configured to allow the execution of operations of image processing. In this example, the processing unit 20 is also connected to a screen 24.
In the configuration shown in FIGS. 1A and 1B, it will be noted that there is no magnifying or image-forming lens between the image sensor 16 and the sample 10. This does not preclude the optional presence of focusing microlenses at the level of each pixel of the image sensor 16, these latter not having the function of magnifying the image acquired by the image sensor, their function being to optimize the efficacy of detection. In such a configuration, called lensless imaging, the distance d between the sample 10 and the pixel matrix of the image sensor 16 is preferably between 50 μm and 2 cm, preferably between 100 μm and 2 mm.
In another configuration, an image-forming optical system 19 may be arranged between the sample and the image sensor, as shown in FIG. 1C. In this configuration, the optical system may be an objective and a lens. The optical system 19 defines an object plane Po and an image plane Pi. The object plane is preferably located in the sample, and in particular at the level of the lower face 10 inf of the sample. The image plane Pi preferably coincides with the detection plane P. In order to allow observation of a large area of the sample, the magnification conferred by the optical system 19 is preferably equal to 1, or less than 1. When, however, observation of details of the sample is preferred, the magnification of the optical system 19 may be greater than 1. The configuration shown in FIG. 1C is a focused configuration, the optical system conjugating the sample, and preferably the lower face 10 i, to the image sensor 16.
The advantage of the focused configuration is explained in conjunction with FIG. 3B.
According to another configuration, called defocused:
    • the object plane Po is defocused relative to the sample, by an object defocusing distance;
    • and/or the image plane Pi is defocused relative to the detection plane formed by the image sensor, by an image defocusing distance.
The image defocusing distance and/or the object defocusing distance are preferably less than 1 mm, or even less than 500 μm.
The advantage of a defocused configuration is explained in conjunction with FIG. 2.
The culture medium is a liquid medium or an agar medium commonly used in microbiology. For example, but not limiting, the culture medium may be:
    • a Chapman medium, favorable to the development of halophilic and halotolerant microorganisms, for example, or bacteria of the types Staphylococcus, or Micrococcus, or Enterococcus, or Bacillus;
    • a Hektoen medium, favorable to the development of Salmonellae or Shigellae;
    • a Salmonella-Shigella agar, favorable to the isolation of pathogenic enterobacteria;
    • an Eosin Methylene Blue (EMB) medium, favorable to the development of Gram-negative bacteria;
    • a MacConkey medium, favorable to the development of Gram-negative bacilli, coliform bacteria, or of Salmonella Shigella;
    • a CLED (Cystine Lactose Electrolyte Deficient) medium, commonly used in the study of bacteria present in urine, both Gram+ and Gram−;
    • a BCP (Bromocresol Purple) medium, commonly used for detecting and isolating enterobacteria;
    • a Baird Parker medium, favorable to the identification of bacteria of the Staphylococcus aureus type;
    • a BEA (Bile Esculin Agar) medium, favorable to the identification of bacteria of the Streptococcus type;
a blood agar (Columbia), favorable to the development of bacteria of the Streptococcus type;
    • a cooked blood agar (chocolate);
    • a kaolin agar;
    • a Slanetz agar, usual for identifying bacteria of the Enterococcus type;
    • a cetrimide agar, allowing isolation of bacteria of the Pseudomonas type;
    • a Sabouraud agar, for isolating and identifying saprophytic or pathogenic yeasts or molds;
    • a CIN (Cefsulodin Irgasan Novobiocin) agar, allowing isolation of bacteria of the Yersinia enterocolitica type;
    • a milk agar;
    • a starch agar;
    • an egg agar;
    • a Mossel agar;
    • a Drigalski agar;
    • a TSN (Tryptone Sulfite Neomycin) agar;
    • a TCBS (Thiosulfate-citrate-bile salt-sucrose) agar;
    • a lactose agar with deoxycholate;
    • a Muller Hinton medium;
    • an ordinary nutrient agar (Trypticase Soy Agar for example);
    • a Todd Hewitt medium;
    • a TTC (Triphenyl Tetrazolium Chloride) tergitol medium;
    • a Hajna-Kligler medium;
    • a Lysine Iron medium;
    • a meat-liver medium;
    • a Falkow medium;
    • a Möller medium;
    • a King medium A or a King medium B;
    • a Rappaport medium;
    • an Esculin agar;
    • a Lowenstein-Jensen agar;
    • a BLBVB broth (“bouillon lactosé bilié au vert brilliant”): brilliant green bile lactose broth.
      This is not an exhaustive list.
The microorganisms may be seeded in the culture medium at a depth or on the surface.
Most of the media used in clinical microbiology are nontransparent. When they are used in conventional Petri dishes, their thickness is of the order of 5 mm, which makes them opaque. Owing to their opacity, as indicated in connection with the prior art, it is not conceivable to acquire a usable image of a sample according to the configurations shown in FIGS. 1A to 1C. Thus, an important element of the invention is that the thickness e of the sample, defined parallel to the axis of propagation Z, is far less than in the methods of the prior art. The thickness of the sample corresponds to the thickness of the culture medium 10 m. It is less than 1 mm, and preferably less than 500 μm, and more preferably less than 250 μm, or even less than 100 μm. The thickness e is generally above 10 or 20 μm. At this thickness, the culture medium 10 m becomes translucent. “Translucent” means that it allows propagation of light through its thickness, without being transparent. Relative to the prior art, the reduction in thickness e makes it possible to form an image of the sample according to a transmission configuration, the sample extending between the light source 11 and the image sensor 16.
It should be noted that the reduction in thickness of the culture medium 10 m is far from obvious. In fact, the methods of observation and counting of the prior art assume that the colonies have acquired a certain level of development, which assumes a certain length of time between seeding the culture medium and observing the bacterial colonies. This time is generally more than 1 day, or even more than several days. This justifies the use of a sufficiently thick culture medium, so as to contain an amount of nutrients necessary for proliferation of the microorganisms. On reducing the thickness of the culture medium, the methods of the prior art would not be usable, as the amount of nutrients stored in the culture medium would not allow a sufficient culture time.
The inventors have shown that by reducing the thickness of the culture medium, images can be formed in transmission. These images are usable and allow observation of the colonies at an early stage of development. Because of this, the culture time can be limited to a few hours, making it unnecessary to use a thick culture medium. The amount of nutrient necessary for the development of the colonies, prior to their characterization, is thus considerably reduced relative to the techniques of the prior art. Examples of observation of colonies are presented in conjunction with FIGS. 2, 3A and 3B.
The sample 10 is, in this example, contained in a confinement chamber 15. The confinement chamber 15 is preferably transparent. It comprises an upper wall 15 sup, a lower wall 15 inf and a peripheral wall 15 p. The peripheral wall 15 p extends between the upper wall 15 sup and the lower wall 15 inf. Preferably, the peripheral wall is an annular wall extending around the axis of propagation Z. The lower wall 15 inf and upper wall 15 sup are preferably perpendicular, or approximately perpendicular, to the axis of propagation Z. The upper wall 15 sup and lower wall 15 inf are preferably arranged in contact with the upper 10 sup and lower 10 inf faces of the sample 10, respectively. The culture medium 10 m may comprise a certain content of dissolved oxygen, which allows, to a certain extent, development by the aerobic route.
Confinement of the sample by the lower and upper walls makes it possible to keep the sample isolated relative to the environment, which limits the risks of contamination of the environment or of the sample.
FIG. 2 shows images acquired by the image sensor using a device according to a lensless imaging configuration, as shown in FIGS. 1A and 1B. We used a sample comprising a culture medium 10 m of the horse blood agar type, obtained by mixing defibrinated horse blood with a TSA agar melted at 55° C. (trypticase soy agar supplied by VWR), the volume fraction of blood added being 5%, or 3 mL to 60 mL of TSA agar. The agar was seeded with a 600 μL solution containing bacteria of the Escherichia coli type at a concentration of 950 CFU (Colony Forming Units)/mL. Seeding took place while the agar was still liquid. The seeded agar was poured into a Geneframe chamber with thickness varying between 250 μm and 500 μm.
The other test parameters were as follows:
    • image sensor: CMOS monochrome—pixel size 1.67 μm—active surface 30 mm2;
    • light source: four-quadrant LED, CREE MCE-Color;
    • light source—image sensor distance: 5 cm;
    • sample—image sensor distance: 1 mm;
    • diffuser: Luminit 40°; and
    • diaphragm: Thorlabs 150 μm.
The light source comprises quadrants emitting in a red (635 nm), green (520 nm) and blue (435 nm) illumination spectral band, respectively. The illumination spectral bands were used separately.
The sample was illuminated by means of the light source, in the 520 nm spectral band. An image of the sample was acquired every 10 minutes, using the image sensor. FIG. 2 shows one and the same region of interest of 20 images acquired successively between t=0 min and t=190 min. The time point t=0 min corresponds to an initial time point, considered to coincide with the pouring of the agar into the confinement chamber 15.
Between t=100 min and t=140 min, formation of a diffraction pattern is observed at the center of each image. The diffraction pattern is difficult to distinguish at t=100 min and becomes more and more discernible, in particular at t=130 min and t=140 min. The diffraction pattern is formed by interference between:
    • a part 12′ of the illumination light wave 12 emitted by the light source, the part 12′ corresponding to the part of the light wave transmitted by the sample, i.e. not absorbed by the latter; and
    • a diffraction wave 13, generated by a bacterial colony. The diffraction wave becomes more and more visible as the volume of the bacterial colony develops.
The superposition of the transmitted part 12′ of the illumination light wave, and of the diffraction wave 13, forms an exposure light wave 14 propagating to the image sensor, and allowing detection of the colony. An exposure light wave of this kind is shown in FIG. 1A. A diffraction pattern generally has a central spot, around which concentric rings extend.
Detection, on the image formed by the image sensor, of these diffraction patterns makes it possible to detect development of bacterial colonies, and perform a count of the latter, at a particularly early stage of development. In fact, each colony may be associated with a diffraction pattern. Furthermore, the morphology of the diffraction pattern may allow identification of each colony. In fact, the morphology of the diffraction patterns depends on the type of microorganism forming the colony, as described in the prior art. Thus, the presence of diffraction patterns on the image acquired by the image sensor makes it possible to obtain qualitative and quantitative information at a very early stage of development, for example less than 2 hours after seeding.
The diffraction patterns, as described above, may also be observed according to a defocused configuration, as described above. The defocusing distance is then preferably less than 1 mm, or less than 500 μm, or even less than 200 μm or 100 μm.
When the colony develops, a remarkable phenomenon occurs, as can be seen on the thumbnails corresponding to the time points between t=150 min to 190 min. In fact, formation of intense light points is observed on the image acquired by the image sensor. These intense point zones are well delimited, and make it possible to perform particularly easy detection and counting of the colonies. The inventors explain the presence of the intense point zones by the development of the colonies, as shown in FIG. 1B. Owing to the small thickness of the culture medium, the bacterial colonies, as they develop, extend from the upper face 10 sup of the sample to the lower face 10 inf of the latter. They then form a channel 10 c extending through the culture medium 10 m, the channel joining the upper face 10 sup to the lower face 10 inf. The channel 10 c forms a light guide, allowing easy propagation of a part 12″i of the incident light wave, propagating locally at the level of a colony 10 i through the light guide formed by the channel 10 c.
Thus, after a certain stage of development of the colonies, the exposure light wave 14, arriving at the image sensor 16, comprises:
    • a first part 12′ of the illumination light wave 12 transmitted by the agar forming the sample: and
    • a second part 12″i of the illumination light wave 12, transmitted locally by each channel 10 c, at the level of colonies 10 i, and forming intense point zones on the image, or light spots.
This results in formation of a particularly contrasted image, or each colony forms an intense spot on the image. This allows particularly easy detection and counting of the colonies formed in the sample.
FIGS. 3A and 3B show examples of images of the sample described above, formed at t=min (time point considered as corresponding to pouring of the agar into the sample) and at t=4 h, respectively. The image formed at t=4 h is sufficiently contrasted to count the colonies.
Thus the method allows:
    • at a very early stage, formation of diffraction patterns, evidence of the presence of bacterial colonies in the agar, and allowing a first count, or even identification of the microorganisms forming each colony. By comparing two images acquired at two different time points, separated by an interval of at least ½ generation time, it is possible to distinguish the diffraction patterns obtained from growing microcolonies, from diffraction patterns obtained from sterile nonbiological objects that may be found conventionally in agar media (dust, precipitates), and
    • at an early stage, i.e. in a few hours, typically in less than 6 h, formation of a contrasted image, formed by point light spots, each point light spot corresponding to a colony. This allows easy, reliable counting of the colonies. It is considered that the shape of the spots may depend on the type of microorganism. Thus, a morphological analysis of the image may allow qualitative information to be obtained, relating to the type of microorganism developing in the culture medium.
FIGS. 3A and 3B were acquired with a lensless configuration, as described in FIGS. 1A and 1B. Similar images, or even with higher contrast and higher resolution, may be obtained with an optical system 19 as described in FIG. 1C, according to a focused configuration. In this case, the object plane Po of the optical system 19 merges with the lower face 10 inf of the sample, whereas the image plane Pi of the optical system 19 merges with the detection plane P.
FIG. 4 shows the evolution of the light attenuation of the blood agar described above, as a function of the thickness. This figure was obtained using an unseeded blood agar, varying in thickness between 75 μm and 500 μm. The attenuation formed by the agar was evaluated relative to measurement of luminous intensity Io without agar between the light source and the image sensor.
The attenuation Atte, corresponding to the thickness e, is obtained from the following expression, derived from the Beer Lambert law:
Att e = - ln ⁡ ( I e I 0 ) ( 1 )
where Ie is the luminous intensity measured in the presence of a thickness e of agar.
The measurements shown in FIG. 4 were carried out at a wavelength of 540 nm.
Comparative measurements were undertaken using clear agar, of the TSA type. The attenuation shows little dependence on the thickness, and remains close to 0.05, for thicknesses between 100 and 500 μm.
The invention may be carried out provided that the attenuation, as defined in connection with equation (1), is below 0.5, and preferably below 0.4 or even 0.3.
The inventors consider that the method allows detection and counting of colonies whose diameter is between 5 μm and 50 μm, or more. The method allows particularly early detection of the colonies.
According to one embodiment, counting of diffraction patterns, such as those shown in FIG. 2, is carried out using an automatic recognition algorithm, of the neural network type. More precisely, the algorithm is able to employ a convolutional neural network. The neural network comprises an input layer IN, formed from an image acquired by the image sensor, and an output layer OUT, corresponding to an image on which the colonies of microorganisms are detected. Between the input layer IN and the output layer OUT, the neural network comprises 20 layers L1, L2 . . . L20, with ranks between 1 (layer adjacent to the layer IN) and 20 (layer adjacent to the layer OUT). Each layer comprises 32 planes. A layer is obtained by convolution of the 32 planes of the layer of preceding rank by a convolution kernel of size 3×3. The layer IN is regarded as the layer of rank 0. FIG. 5A shows schematically an architecture of such a network.
The convolutional neural network has been the subject of training, considering 1000 input thumbnails, each input thumbnail having a size of 121×121 pixels. The input thumbnails were taken at random from a hologram image. An input thumbnail of this kind is shown in FIG. 5B. For each input thumbnail, an output thumbnail was determined by an expert. The output thumbnail shows the localization of the microorganisms. An example of an output thumbnail is shown in FIG. 5C. Training made it possible to parameterize the neural network.
Following training, the neural network was used for detecting colonies of microorganisms automatically. FIG. 6A is an example of an image acquired by the image sensor. Each diffraction pattern corresponds to colonies of bacteria of the Escherichia coli type. FIG. 6B shows automatic detection of the colonies, each colony detected being represented by an open circle. Thus, the use of a neural network allows for detection, and counting, as well as localization, of colonies, moreover at an early stage of development.
Furthermore, by carrying out training on different species of microorganisms, employment of a neural network may allow for detection, counting and identification of the microorganisms.
The invention may be used for carrying out checking and counting of bacteria, as an aid in medical diagnostics, or in the field of environmental monitoring, or else control of industrial processes, for example in the field of food processing or cosmetics.

Claims (21)

The invention claimed is:
1. A method for observation of a sample, the sample comprising colonies of microorganisms introduced into an opaque culture medium, the culture medium comprising nutrients allowing development of the microorganisms, the sample being arranged between a light source and an image sensor, the method comprising:
a) illuminating the sample with the light source, the light source emitting an incident light wave propagating along an axis of propagation;
b) acquiring an image of the sample by the image sensor; and
c) from the image acquired, detecting microorganisms;
wherein:
the culture medium extends, parallel to the axis of propagation, to a thickness of less than 500 μm;
the culture medium extends between an upper face and a lower face, perpendicular or sensibly perpendicular to the axis of propagation; and
as they develop, the microorganisms form colonies, at least one colony forming a light channel extending from the upper face to the lower face through the culture medium, so that light propagates through the light channel to the image sensor and forms a light spot on the image acquired by the image sensor having an intensity greater than an intensity of an area on the image sensor corresponding to the culture medium.
2. The method according to claim 1, wherein the culture medium extends, parallel to the axis of propagation, to a thickness of less than 250 μm.
3. The method according to claim 1, comprising:
counting light spots formed on the image acquired by the image sensor; and
estimating a number of colonies in the sample, as a function of a number of light spots counted on the image.
4. The method according to claim 1, comprising:
performing a morphological analysis of at least one light spot formed on the image acquired by the image sensor; and
identifying the colony that produced the light spot, based on the morphological analysis.
5. The method according to claim 1, comprising:
acquiring a first image, at a first time point, the image comprising at least one diffraction pattern associated with a colony of microorganisms; and
acquiring a second image, at a second time point, subsequent to the first time point, the second image comprising at least one light spot associated with the colony of microorganisms.
6. The method according to claim 1, wherein:
the culture medium extends in a confinement chamber;
the culture medium extends between two opposite faces, perpendicular or approximately perpendicular to the axis of propagation; and
the confinement chamber is in contact with the culture medium at the level of the two opposite faces.
7. The method according to claim 1, wherein no image-forming lens is arranged between the sample and the image sensor.
8. The method according to claim 1 wherein:
an image-forming optical system is arranged between the sample and the image sensor, the optical system defining an object plane and an image plane; and
the image sensor defines a detection plane;
wherein during image acquisition, at least one of:
the sample is shifted relative to the object plane by an object defocusing distance; and
the detection plane is shifted, relative to the image plane, by an image defocusing distance.
9. The method according to claim 1, wherein
an image-forming optical system is arranged between the sample and the image sensor, the optical system defining an object plane and an image plane; and
the image sensor defines a detection plane;
wherein during image acquisition, the lower face of the sample corresponds to the object plane and the detection plane corresponds to the image plane.
10. The method according to claim 1, wherein the microorganisms are not stained.
11. The method according to claim 1, wherein the culture medium extends, parallel to the axis of propagation, to a thickness of less than 100 μm.
12. A method for observation of a sample, the sample comprising colonies of microorganisms introduced into an opaque culture medium, the culture medium comprising nutrients allowing development of the microorganisms, the sample being arranged between a light source and an image sensor, the method comprising:
a) illuminating the sample with the light source, the light source emitting an incident light wave propagating along an axis of propagation;
b) acquiring an image of the sample using the image sensor; and
c) detecting microorganisms using the image;
wherein:
the culture medium comprises an upper face and a lower face extending in a direction crossing to the axis of propagation; and
the microorganisms form at least one colony extending through the culture medium from the upper face to the lower face allowing light to propagate to the image sensor to form a light spot on the image sensor having an intensity greater than an intensity of an area on the image sensor corresponding to the culture medium.
13. The method according to claim 12, wherein the culture medium extends, parallel to the axis of propagation, to a thickness of less than 500 μm.
14. The method according to claim 12, wherein the culture medium extends, parallel to the axis of propagation, to a thickness sufficient to allow the at least one colony to extend from the upper face to the lower face.
15. The method according to claim 12, comprising:
counting light spots formed on the image acquired by the image sensor; and
estimating a number of colonies in the sample, as a function of a number of light spots counted on the image.
16. The method according to claim 12, comprising:
performing a morphological analysis of at least one light spot formed on the image acquired by the image sensor; and
identifying the colony that produced the light spot, based on the morphological analysis.
17. The method according to claim 12, comprising:
acquiring a first image, at a first time point, the image comprising at least one diffraction pattern associated with a colony of microorganisms; and
acquiring a second image, at a second time point, subsequent to the first time point, the second image comprising at least one light spot associated with the colony of microorganisms.
18. The method according to claim 12, comprising:
disposing the culture medium in a confinement chamber.
19. The method according to claim 12, comprising:
the at least one colony forming a light channel; and
propagating the light through the channel to the image sensor.
20. A method for observation of a sample, the sample comprising colonies of microorganisms introduced into a culture medium, the culture medium comprising nutrients allowing development of the microorganisms, the sample being arranged between a light source and an image sensor, the method comprising:
a) illuminating the sample with the light source, the light source emitting an incident light wave propagating along an axis of propagation;
b) acquiring an image of the sample using the image sensor; and
c) detecting microorganisms using the image;
wherein:
the culture medium comprises an upper face and a lower face extending in a direction crossing to the axis of propagation; and
the microorganisms form at least one colony comprising a channel through the culture medium from the upper face to the lower face allowing light to propagate through the channel to the image sensor at a higher intensity than light transmitted through the culture medium.
21. The method according to claim 20, wherein the culture medium is opaque.
US16/845,123 2019-04-12 2020-04-10 Method for early observation of colonies of microorganisms Active US11417127B2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FR1903964 2019-04-12
FR1903964A FR3094988A1 (en) 2019-04-12 2019-04-12 Method of early observation of colonies of microorganisms

Publications (2)

Publication Number Publication Date
US20200327306A1 US20200327306A1 (en) 2020-10-15
US11417127B2 true US11417127B2 (en) 2022-08-16

Family

ID=67875570

Family Applications (1)

Application Number Title Priority Date Filing Date
US16/845,123 Active US11417127B2 (en) 2019-04-12 2020-04-10 Method for early observation of colonies of microorganisms

Country Status (3)

Country Link
US (1) US11417127B2 (en)
EP (1) EP3722439B1 (en)
FR (1) FR3094988A1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102020132312A1 (en) 2020-12-04 2022-06-09 Ludwig-Maximilians-Universität München (Körperschaft des öffentlichen Rechts) Device for recording microscopy images
FR3118465A1 (en) 2020-12-27 2022-07-01 Commissariat à l'Energie Atomique et aux Energies Alternatives method for determining the presence of bacteria in or on an agar component

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030082516A1 (en) * 2001-09-06 2003-05-01 Don Straus Rapid detection of replicating cells
US20060172370A1 (en) * 2004-11-30 2006-08-03 Hirleman Edwin D Jr System and method for rapid detection and characterization of bacterial colonies using forward light scattering
US20120044340A1 (en) * 2010-08-18 2012-02-23 Sony Corporation Microscope control device and optical distortion correction method
US20120218379A1 (en) * 2009-10-20 2012-08-30 The Regents Of The University Of California Incoherent lensfree cell holography and microscopy on a chip
US20130258091A1 (en) * 2010-12-14 2013-10-03 The Regents Of The University Of California Method and device for holographic opto-fluidic microscopy
FR3054037A1 (en) 2016-07-13 2018-01-19 Commissariat A L'energie Atomique Et Aux Energies Alternatives DEVICE FOR OBSERVING A SAMPLE
US20180211095A1 (en) * 2017-01-23 2018-07-26 Interscience Process and Device for Colony Counting
US20180285624A1 (en) 2015-04-21 2018-10-04 Joseph Paul Robinson Culture Detection and Measurement Over Time
US20180299374A1 (en) * 2015-10-12 2018-10-18 The Regents Of The University Of California Spectroscopy imaging and analysis of live cells
WO2018215337A1 (en) 2017-05-22 2018-11-29 Commissariat à l'énergie atomique et aux énergies alternatives Method for analysing microorganisms

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8787633B2 (en) 2007-01-16 2014-07-22 Purdue Research Foundation System and method of organism identification
FR3028951B1 (en) 2014-11-21 2017-01-06 Commissariat Energie Atomique LENS-FREE IMAGING SYSTEM COMPRISING DIODE, DIAPHRAGM AND DIFFUSER BETWEEN DIODE AND DIAPHRAGM
FR3061300B1 (en) 2016-12-26 2020-06-12 Commissariat A L'energie Atomique Et Aux Energies Alternatives METHOD OF OBSERVING AN OBJECT
FR3061301B1 (en) 2016-12-26 2020-09-04 Commissariat Energie Atomique OBJECT OBSERVATION PROCESS

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030082516A1 (en) * 2001-09-06 2003-05-01 Don Straus Rapid detection of replicating cells
US20060172370A1 (en) * 2004-11-30 2006-08-03 Hirleman Edwin D Jr System and method for rapid detection and characterization of bacterial colonies using forward light scattering
US7465560B2 (en) * 2004-11-30 2008-12-16 Purdue Research Foundation System and method for rapid detection and characterization of bacterial colonies using forward light scattering
US20120218379A1 (en) * 2009-10-20 2012-08-30 The Regents Of The University Of California Incoherent lensfree cell holography and microscopy on a chip
US20120044340A1 (en) * 2010-08-18 2012-02-23 Sony Corporation Microscope control device and optical distortion correction method
US20130258091A1 (en) * 2010-12-14 2013-10-03 The Regents Of The University Of California Method and device for holographic opto-fluidic microscopy
US20180285624A1 (en) 2015-04-21 2018-10-04 Joseph Paul Robinson Culture Detection and Measurement Over Time
US20180299374A1 (en) * 2015-10-12 2018-10-18 The Regents Of The University Of California Spectroscopy imaging and analysis of live cells
FR3054037A1 (en) 2016-07-13 2018-01-19 Commissariat A L'energie Atomique Et Aux Energies Alternatives DEVICE FOR OBSERVING A SAMPLE
US20180211095A1 (en) * 2017-01-23 2018-07-26 Interscience Process and Device for Colony Counting
WO2018215337A1 (en) 2017-05-22 2018-11-29 Commissariat à l'énergie atomique et aux énergies alternatives Method for analysing microorganisms

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Yoshiaki Maeda et al. "Colony fingerprint for discrimination of microbial species based on lensless imaging of microcolonies." Division of Biotechnology and Life Science, Institute of Engineering, Tokyo University of Agriculture and Technology, Tokyo, Japan. Malcom Co., Tokyo, Japan. Apr. 3, 2017.

Also Published As

Publication number Publication date
EP3722439B1 (en) 2023-05-10
EP3722439A1 (en) 2020-10-14
US20200327306A1 (en) 2020-10-15
FR3094988A1 (en) 2020-10-16

Similar Documents

Publication Publication Date Title
US9822389B2 (en) Method for the characterization of microorganisms on solid or semi-solid media
US6905838B1 (en) Method and device for characterizing a culture liquid
US10900011B2 (en) Test apparatus
Croxatto et al. Towards automated detection, semi-quantification and identification of microbial growth in clinical bacteriology: a proof of concept
KR20020034171A (en) Apparatus and method for detecting, quantifying and characterizing microorganisms
US11417127B2 (en) Method for early observation of colonies of microorganisms
KR102811379B1 (en) Method and system for identifying gram types of bacteria
US20060172370A1 (en) System and method for rapid detection and characterization of bacterial colonies using forward light scattering
KR102811424B1 (en) How to identify yeast or bacteria
US20170044588A1 (en) Method for detecting micro-colonies growing on a membrane or an agarose medium of a sample and a sterility testing apparatus
JP6487420B2 (en) Organic observation method and related system
US20240344106A1 (en) Rapid antimicrobial susceptibility testing by video-based object scattering intensity detection
JP2017521069A (en) Method for detecting the presence or absence of biological particles
US20240019365A1 (en) Method for analysing a biological sample comprising determination of the spatial distribution of biomass along the optical axis
US20130224773A1 (en) Method for Rapid Growth, Detection and Identification of Live Microorganisms Immobilized on Permeable Membranes by Antibodies
JP2021093951A (en) Microbe determination system
Fröhling et al. A method for viability testing of Pectobacterium carotovorum in postharvest processing by means of flow cytometry
CA2187406A1 (en) Incubation control for contaminants and process for using the same in an antibiogram directly produced from sampling
KR102848394B1 (en) Method for identify the type of bacteria
Iyengar et al. Measuring autofluorescence spectral signatures for detecting antibiotic-resistant bacteria using Thermofisher’s Bigfoot spectral flow cytometer
WO2022071238A1 (en) Method for evaluating evaluation target
Sharp et al. AI and interpretive algorithms
EP3221689A1 (en) Process for observing a plurality of samples and associated devices

Legal Events

Date Code Title Description
FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

AS Assignment

Owner name: COMMISSARIAT A L'ENERGIE ATOMIQUE ET AUX ENERGIES ALTERNATIVES, FRANCE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BORDY, THOMAS;ALLIER, CEDRIC;MARCOUX, PIERRE;SIGNING DATES FROM 20200717 TO 20200720;REEL/FRAME:055029/0481

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: ADVISORY ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4