Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/60—Type of objects
  • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
  • G06V20/695—Preprocessing, e.g. image segmentation
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
  • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
  • G01N15/10—Investigating individual particles
  • G01N15/1012—Calibrating particle analysers; References therefor
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
  • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
  • G01N15/10—Investigating individual particles
  • G01N15/14—Optical investigation techniques, e.g. flow cytometry
  • G01N15/1429—Signal processing
  • G01N15/1433—Signal processing using image recognition
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
  • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
  • G01N15/10—Investigating individual particles
  • G01N15/14—Optical investigation techniques, e.g. flow cytometry
  • G01N15/1434—Optical arrangements
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
  • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
  • G01N15/10—Investigating individual particles
  • G01N15/14—Optical investigation techniques, e.g. flow cytometry
  • G01N15/1456—Optical investigation techniques, e.g. flow cytometry without spatial resolution of the texture or inner structure of the particle, e.g. processing of pulse signals
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/60—Type of objects
  • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
  • G06V20/693—Acquisition
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/60—Type of objects
  • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
  • G06V20/698—Matching; Classification
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
  • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
  • G01N15/10—Investigating individual particles
  • G01N2015/1006—Investigating individual particles for cytology

Definitions

• This invention relates to methods and systems for image data processing and more particularly relates to an apparatus system and method for image normalization using a Gaussian residual of fit selection criteria.
• Imaging using detectors such as charged coupled device (CCD) detectors is employed in several currently-available instruments in biotechnology applications. Many of the commercially available systems are configured to image target human (or other animal) cells. For multiplexed applications in which CCD detectors are used to measure fluorescent emission of cells, the position of the cells and the fluorescent emission within the image may be used to characterize the cells.
• CCD charged coupled device
• the method includes acquiring a two-dimensional image of a plurality of particles, where the plurality of particles comprises a plurality of calibration particles.
• the method may include the step of identifying a calibration particle by correlating a portion of the image corresponding to the calibration particle to a mathematical model.
• the method may include measuring an intensity of the calibration particle and utilizing the intensity of the calibration particle to normalize the intensity of the image.
• the calibration particle is internally dyed.
• the method may also include identifying a plurality of calibration particles, where the plurality of calibration particles are distributed into a plurality of regions of the two- dimensional image.
• the method may include utilizing an intensity of the plurality of calibration particles to normalize an intensity of the plurality of regions.
• the method may include utilizing the intensity of the calibration particle to normalize the intensity of a second two-dimensional image of the plurality of particles.
• the second two-dimensional image may be a classification image.
• the mathematical model may be a Gaussian mathematical model. In some embodiments, the mathematical model may be a quadratic mathematical model.
• measuring the intensity of the calibration particle may include detecting a peak of the calibration particle.
• measuring the intensity of the calibration particle may include integrating an area of the image around a center of the calibration particle.
• the method may include subtracting a background signal from the two-dimensional image before identifying the calibration particle.
• tangible computer-readable media are also presented.
• the tangible computer- readable media may include instructions, that when executed by a computer, cause the computer to perform the methods described herein.
• Coupled is defined as connected, although not necessarily directly, and not necessarily mechanically.
• a step of a method or an element of a device that "comprises,” “has,” “includes” or “contains” one or more features possesses those one or more features, but is not limited to possessing only those one or more features.
• a device or structure that is configured in a certain way is configured in at least that way, but may also be configured in ways that are not listed.
• FIG. 1 is a flow chart illustrating one embodiment of a method for normalizing an image.
• FIG. 2 is a three-dimensional representation of a two-dimensional image of a calibration particle.
• FIG. 3 is a cross-sectional view of images of two particles.
• FIG. 4 is a representation of a two-dimensional image partitioned into nine regions for normalization.
• the types of particles that are compatible with the systems and methods described herein include particles with fluorescent materials attached to, or associated with, the surface of the particles.
• These types of particles in which fluorescent dyes or fluorescent particles are coupled directly to the surface of the particles in order to provide the classification fluorescence (i.e., fluorescence emission measured and used for determining an identity of a particle or the subset to which a particle belongs), are illustrated and described in U.S. Patent Nos. 6,268,222 to Chandler et al. and 6,649,414 to Chandler et al., which are incorporated by reference as if fully set forth herein.
• the types of particles that can be used in the methods and systems described herein also include particles having one or more fluorochromes or fluorescent dyes incorporated into the core of the particles.
• calibration particles may be internally and uniformly dyed.
• a calibration particle may be internally dyed with a plurality of dyes.
• Particles that can be used in the methods and systems described herein further include particles that in of themselves will exhibit one or more fluorescent signals upon exposure to one or more appropriate light sources.
• particles may be manufactured such that upon excitation the particles exhibit multiple fluorescent signals, each of which may be used separately or in combination to determine an identity of the particles.
• image data processing may include classification of the particles, particularly for a multi-analyte fluid, as well as a determination of the amount of analyte bound to the particles. Since a reporter signal, which represents the amount of analyte bound to the particle, is typically unknown during operations, specially dyed particles, which not only emit fluorescence in the classification wavelength(s) or wavelength band(s) but also in the reporter wavelength or wavelength band, may be used for the processes described herein.
• the methods described herein generally include analyzing one or more images of particles and processing data measured from the images to determine one or more characteristics of the particles.
• the processing of data may be used to determine normalized numerical values representing the magnitude of fluorescence emission of the particles at multiple detection wavelengths in multiple regions of an image.
• Subsequent processing of the one or more characteristics of the particles such as using one or more of the numerical values to determine a token ID representing the multiplex subset to which the particles belong and/or a reporter value representing a presence and/or a quantity of analyte bound to the surface of the particles, can be performed according to the methods described in U.S. Patent Nos.
• FIG. 1 illustrates one embodiment of a method 100 for image normalization using a mathematical model (e.g. Gaussian) residual of fit selection criteria.
• the method 100 begins with the step 102 of acquiring a two-dimensional image of a plurality of particles.
• the image may be taken, for example, with a CCD sensor.
• multiple images may be taken.
• the plurality of particles may include a plurality of calibration particles.
• Calibration particles may be, for example, triple-dyed particles.
• the dyes may be evenly distributed throughout the calibration particles, which may result in an evenly distributed fluorescence in the reporter channel when the calibration particles are illuminated with an excitation light source.
• all particles will have evenly distributed fluorescence in the classification channels.
• only calibration particles will have evenly distributed fluorescence in the reporter channel.
• Evenly distributed fluorescence may cause a Gaussian distribution of light in an image corresponding to the particle due to the spherical shape of a particle.
• Step 104 recites identifying a calibration particle by correlating a portion of the image that corresponds to the calibration particle to a mathematical formula.
• This step may include some sub-components.
• the method may first include subtracting out a background signal from the image and then detecting the peaks in the image that correspond to individual particles. Once the location of the particles are known, the method may include performing a numerical fit to the image pixels around the detected peaks.
• the numerical fit may be a Gaussian fit of the equation in the form of: [0029]
• the fit process determines the parameters a and b that best fit the image of the particle.
• the fit may be done in sub-pixel resolution, such as by interpolating pixels to increase the resolution of the image used to perform the fit.
• the residual of the fit may be measured and if the residual is above a predetermined value (a tolerance), the particle may be rejected as not being a calibration particle.
• Calibration particles may have Gaussian profiles due to being internally dyed.
• assay particles which may have fluorescence solely on the surface of the particle, may not have a Gaussian distribution.
• calibration particles may be identified by their Gaussian profiles.
• the profiles of calibration particles is described generally as Gaussian, in some embodiments, the mathematical formula to perform the fit may be quadratic, for example. Different formulas may reduce the processing required to determine the fit at the expense of reduced accuracy in detecting calibration particles.
• additional steps such as discarding outlier particles, may be used to increase the performance of the system.
• Method 100 also includes the step 106 of measuring an intensity of the calibration particle.
• the intensity of the particle may be measured by measuring the peak of the measured signal, or it may be measured by integrating the pixels within a particular radius of the measured peak. Additionally, the intensity may be measured by first determining a sub-pixel image of the particle, such as through interpolation, and then integrating the sub-pixel image around a peak of the particle.
• a calibration particle may have a known size and amount of fluorescent material that are established when the calibration particles are manufactured.
• the amount of different dyes used to manufacture the calibration particles can be carefully controlled to ensure a known amount and even distribution of fluorescence.
• Method 100 also includes the step 108 of using the intensity of the calibration particle to normalize the two-dimensional image. Because the expected amount of fluorescence of the calibration particle is known, that amount of fluorescence may be used to normalize the measured amount of fluorescence intensity. Moreover, this process may be repeated for a plurality of calibration particles distributed throughout the 2-D image to normalize different areas of the image. Because calibration particles can be interspersed with assay particles, the image may be normalized without having to take a separate image with calibration particles alone. Thus, the throughput may be increased while maintaining normalized intensity of multiple images. The lack of uniformity of image intensity may be caused by light source non-uniformity, lens nonuniformity, or movement of the imaging plane, for example. The methods described herein may be able to simultaneously normalize for a plurality of causes of non-uniform light measurements.
• the normalization of the image intensity may be used when multiple images are taken of the same set of particles. For example, two separate images may be taken of classification channel and one in a reporter channel.
• the calibration particles may show up in all three images and may be used to normalize all three images.
• FIG. 2 describes a portion of a two-dimensional image where the intensity of measured light 202 from a particle is shown on the z-axis.
• the measured light 202 is Gaussian in form. That information may be used to identify a particle as a calibration particle as described above.
• FIG. 3 shows example profiles of two different particles.
• Curve 302 corresponds to a Gaussian curve.
• the particle whose image corresponds to curve 302 may be identified as a calibration particle.
• curve 304 may correspond to a an assay particle (non-calibration particle).
• the detected fluorescence may come from material distributed on the surface of the particle. As such, the distribution of light will not be Gaussian and the particle can be identified as not being a calibration particle.
• These curves correspond to a cross section of the image shown in FIG 2.
• FIG. 4 shows one example of how calibration particles 420 may be used to normalize an image.
• the image 400 is partitioned into nine regions (402, 404, 406, 408, 410, 412, 414, 416, and 418). Each region has one or more calibration particles 420.
• the calibration particles may be first identified as being calibration particles by the profile of their image. After the calibration particles are identified, the intensity of the calibration particles 420 may be measured as described above. That measured intensity may then be used to normalize measurements of non- calibration particles (not shown) in the image 400.
• Differences in image intensity may be compensated through this normalization.
• the example in FIG. 4 shows an image that is partitioned into nine regions, the partitions may be as few as one (the entire image is normalized uniformly, or may be unlimited. In the latter situation, a mathematical formula representing the normalization may be constructed. For reasons of explanation only, the mathematical formula representing the normalization may resemble a topographical map that shows how amount of normalization applies varies by location on the image.
• Some embodiments include a tangible computer-readable medium that includes computer-readable code that, when executed by a computer, causes a computer to perform at least one embodiment of the present methods.
• the tangible computer-readable medium may be, for example, a CD-ROM, a DVD-ROM, a flash drive, a hard drive or any other physical storage device.
• a tangible computer-readable medium is created.
• the method may include recording the computer readable medium with computer readable code that, when executed by a computer, causes the computer to perform at least one embodiment of the present methods. Recording the computer readable medium may include, for example, burning data onto a CD-ROM or a DVD-ROM, or otherwise populating a physical storage device with the data.

Landscapes

• Chemical & Material Sciences (AREA)
• Engineering & Computer Science (AREA)
• General Health & Medical Sciences (AREA)
• Physics & Mathematics (AREA)
• Health & Medical Sciences (AREA)
• Life Sciences & Earth Sciences (AREA)
• General Physics & Mathematics (AREA)
• Pathology (AREA)
• Biochemistry (AREA)
• Immunology (AREA)
• Analytical Chemistry (AREA)
• Dispersion Chemistry (AREA)
• Multimedia (AREA)
• Molecular Biology (AREA)
• Biomedical Technology (AREA)
• Theoretical Computer Science (AREA)
• Signal Processing (AREA)
• Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Image Analysis (AREA)
• Image Processing (AREA)

Priority Applications (5)

Applications Claiming Priority (2)

Publications (2)

Family

ID=49755971

Family Applications (1)

Country Status (6)

Cited By (1)

Families Citing this family (4)

Citations (11)

Family Cites Families (10)

• 2013
  • 2013-06-14 WO PCT/US2013/046035 patent/WO2013188857A2/en not_active Ceased
  • 2013-06-14 JP JP2015517472A patent/JP6105061B2/ja active Active
  • 2013-06-14 EP EP13804717.0A patent/EP2861969B1/en active Active
  • 2013-06-14 CA CA2876903A patent/CA2876903C/en active Active
  • 2013-06-14 US US13/918,540 patent/US9245169B2/en active Active
  • 2013-06-14 CN CN201380031436.0A patent/CN104541152B/zh active Active
• 2016
  • 2016-01-22 US US15/004,485 patent/US9984279B2/en active Active
• 2018
  • 2018-05-23 US US15/987,359 patent/US10733418B2/en active Active

Patent Citations (13)

Also Published As

Similar Documents

Legal Events