WO2021003369A1 - Cytomètre de calcul à modulation magnétique et procédés d'utilisation - Google Patents

Cytomètre de calcul à modulation magnétique et procédés d'utilisation Download PDF

Info

Publication number
WO2021003369A1
WO2021003369A1 PCT/US2020/040664 US2020040664W WO2021003369A1 WO 2021003369 A1 WO2021003369 A1 WO 2021003369A1 US 2020040664 W US2020040664 W US 2020040664W WO 2021003369 A1 WO2021003369 A1 WO 2021003369A1
Authority
WO
WIPO (PCT)
Prior art keywords
sample
cells
objects
target object
sample holder
Prior art date
Application number
PCT/US2020/040664
Other languages
English (en)
Inventor
Aydogan Ozcan
Aniruddha RAY
Yibo Zhang
Dino Di Carlo
Original Assignee
The Regents Of The University Of California
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by The Regents Of The University Of California filed Critical The Regents Of The University Of California
Priority to US17/621,979 priority Critical patent/US20220260481A1/en
Publication of WO2021003369A1 publication Critical patent/WO2021003369A1/fr

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N15/14—Optical investigation techniques, e.g. flow cytometry
    • G01N15/1429—Signal processing
    • G01N15/1433—Signal processing using image recognition
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B03—SEPARATION OF SOLID MATERIALS USING LIQUIDS OR USING PNEUMATIC TABLES OR JIGS; MAGNETIC OR ELECTROSTATIC SEPARATION OF SOLID MATERIALS FROM SOLID MATERIALS OR FLUIDS; SEPARATION BY HIGH-VOLTAGE ELECTRIC FIELDS
    • B03C—MAGNETIC OR ELECTROSTATIC SEPARATION OF SOLID MATERIALS FROM SOLID MATERIALS OR FLUIDS; SEPARATION BY HIGH-VOLTAGE ELECTRIC FIELDS
    • B03C1/00—Magnetic separation
    • B03C1/02—Magnetic separation acting directly on the substance being separated
    • B03C1/30—Combinations with other devices, not otherwise provided for
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B03—SEPARATION OF SOLID MATERIALS USING LIQUIDS OR USING PNEUMATIC TABLES OR JIGS; MAGNETIC OR ELECTROSTATIC SEPARATION OF SOLID MATERIALS FROM SOLID MATERIALS OR FLUIDS; SEPARATION BY HIGH-VOLTAGE ELECTRIC FIELDS
    • B03C—MAGNETIC OR ELECTROSTATIC SEPARATION OF SOLID MATERIALS FROM SOLID MATERIALS OR FLUIDS; SEPARATION BY HIGH-VOLTAGE ELECTRIC FIELDS
    • B03C1/00—Magnetic separation
    • B03C1/005—Pretreatment specially adapted for magnetic separation
    • B03C1/01—Pretreatment specially adapted for magnetic separation by addition of magnetic adjuvants
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B03—SEPARATION OF SOLID MATERIALS USING LIQUIDS OR USING PNEUMATIC TABLES OR JIGS; MAGNETIC OR ELECTROSTATIC SEPARATION OF SOLID MATERIALS FROM SOLID MATERIALS OR FLUIDS; SEPARATION BY HIGH-VOLTAGE ELECTRIC FIELDS
    • B03C—MAGNETIC OR ELECTROSTATIC SEPARATION OF SOLID MATERIALS FROM SOLID MATERIALS OR FLUIDS; SEPARATION BY HIGH-VOLTAGE ELECTRIC FIELDS
    • B03C1/00—Magnetic separation
    • B03C1/02—Magnetic separation acting directly on the substance being separated
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B03—SEPARATION OF SOLID MATERIALS USING LIQUIDS OR USING PNEUMATIC TABLES OR JIGS; MAGNETIC OR ELECTROSTATIC SEPARATION OF SOLID MATERIALS FROM SOLID MATERIALS OR FLUIDS; SEPARATION BY HIGH-VOLTAGE ELECTRIC FIELDS
    • B03C—MAGNETIC OR ELECTROSTATIC SEPARATION OF SOLID MATERIALS FROM SOLID MATERIALS OR FLUIDS; SEPARATION BY HIGH-VOLTAGE ELECTRIC FIELDS
    • B03C1/00—Magnetic separation
    • B03C1/02—Magnetic separation acting directly on the substance being separated
    • B03C1/025—High gradient magnetic separators
    • B03C1/031—Component parts; Auxiliary operations
    • B03C1/033—Component parts; Auxiliary operations characterised by the magnetic circuit
    • B03C1/0332—Component parts; Auxiliary operations characterised by the magnetic circuit using permanent magnets
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B03—SEPARATION OF SOLID MATERIALS USING LIQUIDS OR USING PNEUMATIC TABLES OR JIGS; MAGNETIC OR ELECTROSTATIC SEPARATION OF SOLID MATERIALS FROM SOLID MATERIALS OR FLUIDS; SEPARATION BY HIGH-VOLTAGE ELECTRIC FIELDS
    • B03C—MAGNETIC OR ELECTROSTATIC SEPARATION OF SOLID MATERIALS FROM SOLID MATERIALS OR FLUIDS; SEPARATION BY HIGH-VOLTAGE ELECTRIC FIELDS
    • B03C1/00—Magnetic separation
    • B03C1/02—Magnetic separation acting directly on the substance being separated
    • B03C1/28—Magnetic plugs and dipsticks
    • B03C1/288—Magnetic plugs and dipsticks disposed at the outer circumference of a recipient
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N15/14—Optical investigation techniques, e.g. flow cytometry
    • G01N15/1434—Optical arrangements
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03H—HOLOGRAPHIC PROCESSES OR APPARATUS
    • G03H1/00—Holographic processes or apparatus using light, infrared or ultraviolet waves for obtaining holograms or for obtaining an image from them; Details peculiar thereto
    • G03H1/0005—Adaptation of holography to specific applications
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/147—Details of sensors, e.g. sensor lenses
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/693—Acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/698—Matching; Classification
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B01—PHYSICAL OR CHEMICAL PROCESSES OR APPARATUS IN GENERAL
    • B01D—SEPARATION
    • B01D35/00—Filtering devices having features not specifically covered by groups B01D24/00 - B01D33/00, or for applications not specifically covered by groups B01D24/00 - B01D33/00; Auxiliary devices for filtration; Filter housing constructions
    • B01D35/06—Filters making use of electricity or magnetism
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B03—SEPARATION OF SOLID MATERIALS USING LIQUIDS OR USING PNEUMATIC TABLES OR JIGS; MAGNETIC OR ELECTROSTATIC SEPARATION OF SOLID MATERIALS FROM SOLID MATERIALS OR FLUIDS; SEPARATION BY HIGH-VOLTAGE ELECTRIC FIELDS
    • B03C—MAGNETIC OR ELECTROSTATIC SEPARATION OF SOLID MATERIALS FROM SOLID MATERIALS OR FLUIDS; SEPARATION BY HIGH-VOLTAGE ELECTRIC FIELDS
    • B03C2201/00—Details of magnetic or electrostatic separation
    • B03C2201/18—Magnetic separation whereby the particles are suspended in a liquid
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B03—SEPARATION OF SOLID MATERIALS USING LIQUIDS OR USING PNEUMATIC TABLES OR JIGS; MAGNETIC OR ELECTROSTATIC SEPARATION OF SOLID MATERIALS FROM SOLID MATERIALS OR FLUIDS; SEPARATION BY HIGH-VOLTAGE ELECTRIC FIELDS
    • B03C—MAGNETIC OR ELECTROSTATIC SEPARATION OF SOLID MATERIALS FROM SOLID MATERIALS OR FLUIDS; SEPARATION BY HIGH-VOLTAGE ELECTRIC FIELDS
    • B03C2201/00—Details of magnetic or electrostatic separation
    • B03C2201/26—Details of magnetic or electrostatic separation for use in medical applications
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/01—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials specially adapted for biological cells, e.g. blood cells
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N2015/1006—Investigating individual particles for cytology
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N15/14—Optical investigation techniques, e.g. flow cytometry
    • G01N15/1434—Optical arrangements
    • G01N2015/144—Imaging characterised by its optical setup
    • G01N2015/1445—Three-dimensional imaging, imaging in different image planes, e.g. under different angles or at different depths, e.g. by a relative motion of sample and detector, for instance by tomography
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N15/14—Optical investigation techniques, e.g. flow cytometry
    • G01N15/1434—Optical arrangements
    • G01N2015/1452—Adjustment of focus; Alignment
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N15/14—Optical investigation techniques, e.g. flow cytometry
    • G01N15/1434—Optical arrangements
    • G01N2015/1454—Optical arrangements using phase shift or interference, e.g. for improving contrast
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03H—HOLOGRAPHIC PROCESSES OR APPARATUS
    • G03H1/00—Holographic processes or apparatus using light, infrared or ultraviolet waves for obtaining holograms or for obtaining an image from them; Details peculiar thereto
    • G03H1/0005—Adaptation of holography to specific applications
    • G03H2001/005—Adaptation of holography to specific applications in microscopy, e.g. digital holographic microscope [DHM]
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03H—HOLOGRAPHIC PROCESSES OR APPARATUS
    • G03H2222/00—Light sources or light beam properties
    • G03H2222/10—Spectral composition
    • G03H2222/12—Single or narrow bandwidth source, e.g. laser, light emitting diode [LED]

Definitions

  • the technical field generally to cytometer devices used to detect objects such as rare cells within bodily fluids such as rare cancer cells within blood. More specifically, the field of the invention relates to a computation cytometer based on magnetically modulated lensless speckle imaging, which introduces oscillatory motion to the magnetic particle/bead- conjugated rare cells of interest through a periodic magnetic force, and uses lensless time- resolved holographic speckle imaging to rapidly detect the target cells in three-dimensions (3D).
  • 3D three-dimensions
  • Rare cell detection aims to identify enough low-abundant cells within a vast majority of background cells, which typically requires the processing of large volumes of biological sample.
  • the detection and enumeration of these rare cells is vital for disease diagnostics, evaluation of disease progression, and characterization of immune response.
  • circulating fetal cells presented in maternal blood are recognized as a source of fetal genomic DNA, and their isolation is crucial for the implementation of routine prenatal diagnostic testing.
  • antigen-specific T cells in peripheral blood play a central role in mediating immune response and the formation of immunological memory, which could lead to the prediction of immune protection and diagnosis of immune-related diseases.
  • Circulating endothelial cells with a mature phenotype are increased in patients with certain types of cancer and several pathological conditions, indicating their potential as disease markers.
  • Circulating tumor cells CTCs
  • CTCs Circulating tumor cells
  • hematopoietic stem and progenitor cells which reside predominantly in bone marrow with low numbers, also found in peripheral blood, possess the unique capacity for self-renewal and multilineage differentiation and their trafficking in blood may connect to disease processes.
  • a computational cytometer that uses speckle imaging with lensless chip-scale microscopy that can be employed for specific and sensitive detection of rare cells in blood with low cost and high throughput.
  • This cell detection and cytometry platform and technique is based on magnetically modulated lensless speckle imaging, which specifically labels rare cells of interest using magnetic particles attached to surface markers of interest and generates periodic and well-controlled motion on target cells by alternating an external magnetic field that is applied to a large sample volume.
  • the holographic diffraction and the resulting speckle or diffraction patterns of the moving cells are then captured using a compact and cost-effective on-chip lensless imager and are computationally analyzed by a deep learning-based algorithm to rapidly detect and accurately identify the rare cells of interest in a high-throughput manner based on their unique spatio- temporal features.
  • the computational cytometer uses a magnetically modulated speckle imaging module that includes a lensless in-line holographic microscope and two oppositely positioned electromagnets.
  • the lensless microscope contains a laser diode (650 nm wavelength) to illuminate the sample from ⁇ 5-10 cm above the sample, and a complementary metal–oxide–semiconductor (CMOS) image sensor is placed ⁇ 1 mm below the sample for acquisition of a high-frame-rate video to monitor the spatio-temporal evolution of the sample containing the target cells of interest.
  • CMOS complementary metal–oxide–semiconductor
  • the optical design has a unit magnification, and the field of view (FOV) of a single image is equal to the active area of the image sensor (which can be around 10-30 mm 2 using standard CMOS imagers employed in digital cameras and mobile phones).
  • FOV field of view
  • target cells are enriched using a magnetic separation operation and loaded inside a capillary tube for imaging.
  • the imaging module is mounted onto a linear translation stage and is translated along the direction of the sample tube to capture a holographic video for each section of the sample tube.
  • the electromagnets are supplied with alternating current with a 180° phase difference between the electromagnets to exert an alternating pulling force to the magnetic bead-conjugated cells in the sample, which causes them to oscillate at the same frequency as the driving current.
  • Rods made of permalloy were designed and utilized to enhance the magnetic force at the sample location by ⁇ 40-fold.
  • the holographic diffraction patterns that are cast by the magnetically modulated target cells are captured using the image sensor and are transferred to a computing device such as a laptop computer.
  • a computational motion analysis (CMA) algorithm executed by software on the computing device and a densely connected pseudo-3D convolutional neural network structure (P3D CNN) (also executed by the computing device) then analyze the holographic image sequence that contains the 3D dynamic information from the oscillating cells, which allows rapid and specific detection of the target cells.
  • CMA computational motion analysis
  • P3D CNN densely connected pseudo-3D convolutional neural network structure
  • the tested prototype was used to screen ⁇ 0.942 mL of fluid sample, corresponding to ⁇ 1.177 mL of whole blood sample before enrichment, in ⁇ 7 min, while the components cost only around $750 (excluding the function generator, power supply and laptop computer) and weighing ⁇ 2.1 kg.
  • the platform with a single imaging channel can be expanded to parallel imaging channels by mounting several imaging modules onto the same linear stage, as shown in FIG.1A.
  • the performance of this platform was tested by detecting a model rare cell system of spiked MCF7 cancer cells in human blood.
  • the platform described herein has a limit of detection (LoD) of 10 cells per mL of whole blood using only a single imaging channel. Because the current LoD is mainly limited by the screening volume, the LoD can be further improved by including additional parallel imaging channels (i.e., capillaries or sample holders) and increasing the sample volume that is screened.
  • a cytometer device includes one or more optically transparent sample holders configured to hold a volume of sample therein containing one or more objects therein with at least some of the one or more objects containing magnetic particles bound or conjugated thereto.
  • a moveable scanning head (or multiple scanning heads in some embodiments) is disposed adjacent to the one or more optically transparent sample holders, the moveable scanning head having a lensless imaging module that includes one or more illumination sources configured to illuminate the sample holder from a first side and an image sensor disposed on a second side of the sample holder, the image sensor configured to capture a plurality diffraction patterns created by one or more objects within the sample volume.
  • the moveable scanning head further includes first and second electromagnets located laterally adjacent to the lensless imaging module.
  • Optional permalloy rods are associated with each electromagnet to increase the magnetic field strength on the sample.
  • a translation stage is coupled to the moveable scanning head and configured to move the moveable scanning head along different regions of the optically transparent sample holder. This enables images or movies to be obtained of the sample while the magnetic field is applied to sample volume.
  • the cytometer further includes, in one embodiment, a computing device operatively connected to the cytometer device and configured to receive a plurality of images or video obtained by the image sensor.
  • the computing device executes or runs image processing software configured to identify candidate objects of interest and classify the objects of interest as a target object of interest or not a target object of interest.
  • the image processing software performs drift correction prior to identifying candidate objects of interest.
  • the image processing software inputs a plurality of images or video to a trained neural network to classify the objects of interest.
  • the computing device may also be used to run and/or operate aspects of the cytometer device. This includes activation of the one or more light sources, capturing, transferring and/or storing image files, movement of the scanning head(s), operation and actuation of the electromagnetic, etc.
  • a method of identifying one or more target objects among non-target objects within a sample includes conjugating the one or more target objects with one or more magnetic particles; loading an optically transparent sample holder with a sample containing the conjugated target object(s) and non-target objects; applying an alternating magnetic field to the sample holder containing the sample; illuminating the optically transparent sample holder with illumination from one or more light sources and capturing a plurality of images or video of diffraction patterns generated by the target object(s) and non- target objects within the sample while the alternating magnetic field is applied; subjecting the plurality of images or video to image processing to identify candidate target object(s) and generate a plurality of images or video of the candidate target object(s) that are input to trained neural network or machine learning algorithm that outputs the classification of the candidate target object(s) as a target object or non-target object.
  • FIG.1A illustrates a perspective view of the computational cytometer.
  • FIG.1A further illustrates a magnetically modulated lensless imaging module (inset) and includes one or more illumination sources and an image sensor and two electromagnets driven by alternating current with opposite phase.
  • the lensless imaging module and the two electromagnetics may be contained in a moveable scanning head as illustrated.
  • the fluid sample that contains magnetic bead-conjugated cells of interest (or other objects) is loaded into a capillary tube or other sample holder.
  • the imaging module along with the two electromagnets is mounted to a translation stage to scan along the capillary tube to record holographic images of each section of the tube.
  • FIG.1B illustrates another perspective view of the computation cytometer along with a computing device (e.g., a laptop computer) that is used to control the device and acquire image data.
  • a computing device e.g., a laptop computer
  • a function generator and a power supply together with custom-designed circuitry are used to provide the power and driving current for the translation stage and electromagnets.
  • FIG.1B shows in inset an image of the two electromagnets and respective rods that are located on either side of the sample holder (e.g., capillary tube). The scanning head is removed to better illustrate the configuration of the electromagnets adjacent to the capillary tube.
  • FIG.1C schematically illustrates a lensless imaging module that is used to generate images or movies that are processed by the computing device via image processing software (FIG.1B). Also illustrated is an object of interest (e.g., cell) that is conjugated to a plurality of magnetic particles (e.g., beads).
  • object of interest e.g., cell
  • magnetic particles e.g., beads
  • FIG.2 illustrates sample preparation and imaging procedures according to one embodiment.
  • the sample preparation time before scanning is approximately 1 hour, where the first 30 min is passive incubation that does not require supervision.
  • FIGS.4A-4K illustrate the steps or operations of computational detection of rare cells according to one embodiment.
  • FIGS.4A-4C illustrate the preliminary screening for the whole FOV to detect candidates for target cells (MCF7).
  • MCF7 target cells
  • CMA computational motion analysis
  • FIGS.4D-4G are zoomed-in preliminary processing for the example region marked with1 in FIGS.4B-4C.
  • FIGS.4H-4K illustrate a classification process for two cell candidates marked with1 and2 in FIG.4C.
  • the axial location for each cell candidate was determined by autofocusing (FIG.4H).
  • a video was formed for each cell candidate by propagating each frame to the in-focus plane (FIG.4I).
  • the classification was performed by the trained neural network which was a densely connected P3D convolutional neural network (FIG.4J).
  • FIG.4K illustrates the final result for each candidate object (true or false determination).
  • FIG.5 illustrates the quantification of the limit of detection (LoD) of the computational cytometer based on magnetically modulated lensless speckle imaging for the detection of MCF7 cells in whole blood.
  • the axes are a hybrid of logarithmic and linear scales to permit 0 cells/mL to be shown in the same plot.
  • the solid data points represent one- time testing results of a single trained P3D CNN.
  • the error bars represent the respective standard deviation of the three repeated tests at each spiked target cell concentration.
  • the circle data points represent the averaged testing results using five P3D CNNs that were individually trained on a different subset of data.
  • the error bars represent the standard deviation resulting from the detections of the five individual networks; for each trained network, three detected concentrations are averaged at each spiked concentration.
  • FIG.6 illustrates the structure of the trained neural network according to one embodiment.
  • the neural network was a densely connected P3D CNN.
  • Network consists of convolutional layers, series of dense blocks, a fully-connected layer, and a softmax layer.
  • each dense spatial-temporal convolution block was constructed by introducing skip connections between input and output of convolutional layers in channel dimension, where red (R) represents the input of the dense block, green (G) and blue (B) represent the output of spatial and temporal convolutional layers, respectively, and yellow (Y) represents the output of the entire block.
  • FIGS.7A-7D illustrate COMSOL simulation of magnetic force field generated by electromagnets with respective permalloy relay.
  • FIG.7A is a 3D schematic of the permalloy relays relative to the electromagnets.
  • FIGS.7B-7D illustrate simulation of the relative (unitless) magnitude of the magnetic force field that is generated by a single electromagnet with (FIG.7D) or without (FIG.7C) permalloy relay, as a function of the spatial position.
  • the relay significantly increases the magnetic force field for a given axial distance from the electromagnet (FIG.7D).
  • FIG.8 illustrates the effect of using computational drift correction to remove false positives. Without drift correction, due to the drifting of the medium, particles that do not oscillate in response to the changing magnetic force field may generate contrast in the 2D contrast map, which reduces the effectiveness of the computational motion analysis.
  • the sub- pixel drift correction step removes most of the“false positive” contrast.
  • FIG.9 illustrates the receiver operating characteristic (ROC) curve of the trained P3D CNN classifier. The curve is generated by varying the decision threshold for positive classification using the validation dataset. The area under the curve (AUC) is 0.9678.
  • ROC receiver operating characteristic
  • FIGS.1A-1C illustrate a cytometer device 10 according to one embodiment.
  • the cytometer device 10 is located in a housing or enclosure 12 and contains the various components of the cytometer device 10.
  • the cytometer device 10 itself contains one or more optically transparent sample holder(s) 14 that holds a volume of sample that is to be scanned or interrogated.
  • the optically transparent sample holder 14 may include a capillary, tube, flow cell, or microfluidic channel, for example.
  • the optically transparent sample holder 14 is loaded with a sample obtained from a biological fluid.
  • the optically transparent sample holder 14 may also contain a viscous fluid (e.g., methyl cellulose solution) along with the sample.
  • the sample may include cells, bacteria, protozoa, multi-cellular organisms and the like.
  • the sample may be a biological sample or an environmental sample. In one preferred embodiment, the sample contains cells and is used to identify certain types of cells as explained herein.
  • FIG.1A illustrates an embodiment of the cytometer device 10 that contains multiple moveable scanning heads 16. This embodiment allows parallel processing of larger sample sizes or even different samples at the same time. While three such scanning heads 16 are illustrated (along with corresponding sample holders 14) there may be fewer or more scanning heads 16. For example, there may be 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 or more scanning heads 16.
  • the optically transparent sample holder 14 may pass through all or a portion of the scanning head(s) 16.
  • the moveable scanning head(s) 16 includes several components.
  • the scanning head 16 includes a lensless imaging module 18 that is used to illuminate the sample within the sample holder 14 with light and obtain a plurality of diffraction pattern images or a movie 100 over a period of time of objects 90 (FIGS.1C, 4I, 6) contained in the sample in the sample holder 14.
  • the objects 90 may include beads or cells.
  • Each scanning head 16 is associate with its own sample holder 14.
  • Certain objects 90 are target objects that are desired to be identified or classified. These target objects 90 are to be distinguished from non-target objects 90.
  • the target objects 90 comprise cells of a particular type or phenotype, morphology, shape, size, or genotype.
  • the target objects 90 may comprise cancer cells or certain type of cancer cells such as circulating tumor cells (CTCs).
  • CTCs circulating tumor cells
  • the target objects 90 are conjugated to one or more magnetic particles 92.
  • Particles 92 may include magnetic beads or the like.
  • the one or more magnetic particles 92 may be made from superparamagnetic particles.
  • An example of such particles 92 include Dynabeads® (Invitrogen, Carlsbad, California, USA). The presence of the conjugated or bound magnetic particle(s) 92 in response to the externally applied magnetic field is what allows for the
  • the lensless imaging module 18 includes one or more illumination sources 20 configured to illuminate the sample from a first side of the sample holder 14 and an image sensor 22 (e.g., CMOS sensor) disposed on a second side of the sample holder 14.
  • the one or more illumination sources 20 include a laser diode although light emitting diodes (LEDs) may also be used.
  • the one or more illumination sources 20 may be driven by on-board driver circuitry (not shown) located in the scanning head 16, for example.
  • the one or more illumination sources 20 include a laser diode (650-nm wavelength, AML-N056-650001-01, Arima Lasers Corp., Taoyuan, Taiwan) for illumination, which has an output power of ⁇ 1 mW.
  • the one or more illumination sources 20 emit light onto the top of the sample holder 14 while the image sensor 22 captures time series of speckle pattern images 100 from the bottom of the sample holder 14.
  • the image sensor 22 is configured to capture a time series of speckle pattern images 100 created by the one or more objects 90 within the volume of sample.
  • the moveable scanning head(s) 16 further include, in one embodiment, first and second electromagnets 24, 26 located laterally adjacent to the lensless imaging module 18. That is to say, first and second electromagnets 24, 26 are located on either side of the lensless imaging module 18.
  • optional permalloy (nickel-iron magnetic alloy) rods 28 are used with each of the electromagnets 24, 26 to enhance or relay the magnetic force on the objects 90.
  • more than two electromagnets 24, 26 may be used with the moveable scanning head(s) 16.
  • the plurality of electromagnets 24, 26 are driven using dedicated circuitry and/or a function generator 30 (FIG.1B). As an alternative to the function generator 30 an oscillator circuit built form a timer integrated circuit may be used. In one embodiment, the plurality of electromagnets 24, 26 are driven with alternating current with a 180° phase shift. Thus, while one electromagnet (e.g., electromagnet 24) is energized at an on or high state, the other electromagnetic (e.g., electromagnet 26) is energized at on off or low state. This is seen, for example, in the inset of FIG.1A). This provides an alternating pulling force on the target objects 90 that are bound to the magnetic particle(s) 92.
  • a function generator 30 As an alternative to the function generator 30 an oscillator circuit built form a timer integrated circuit may be used.
  • the plurality of electromagnets 24, 26 are driven with alternating current with a 180° phase shift.
  • one electromagnet e.g., electromagnet 24
  • the frequency at which the plurality of electromagnets 24, 26 are driven at may vary between about .01 Hz and about 100 kHz, although the platform was optimized for cancer cells at around 1 Hz.
  • Different object 90 and sample conditions e.g., number of magnetic particles 92, size of objects 90, fluid in which the object is contained, frictional or viscous forces, etc. affect the frequency required to generate effective oscillations of the objects 90.
  • the cytometer device 10 further includes a translation stage 32 mechanically coupled to the moveable scanning head(s) 16 and configured to move the moveable scanning head(s) 16 along different regions of the optically transparent sample holder 14.
  • the translation stage 32 may be a linear translation stage 32 that moves the scanning head(s) 16 to different regions on the sample holder 14.
  • the translation stage 32 is mounted on rails 34 on either side of the enclosure or housing 12. The translation stage 32 is thus allowed to move or slide on the rails 34 in the forward and backwards direction of arrow A.
  • the translation stage 32 in one embodiment, is moved via a belt 36 that is driven by a stepper motor 38.
  • the belt 36 is secured to the translation stage 32 and passes over pulleys 40 (front and back with only front illustrated) that is mounted on a support rod 42.
  • the stepper motor 38 moves in one direction to advance the translation stage 32 while the stepper motor 38 moves in the opposing direction to retract the translation stage 32. While the stepper motor 38 and belt 36 is described, other drive mechanisms known to those skilled in the art may be used to move the translation stage 32.
  • the key is that the translation stage 32 can move back-and-forth to move the scanning head(s) relative the sample holder 14. Note that the fluid containing the objects 90 within the sample holder 14 is substantially stationary during the scanning process. There may be slight movements or currents of fluid caused by heating or other effects as described herein which can be addressed using a drift correction procedure as explained herein.
  • the stepper motor 38 is operated by a driver/drive circuitry (not shown) that is controlled via a microcontroller 44.
  • the microcontroller 44 interfaces with a computing 46 that is used to control the operation of the cytometer device 10 as well as process the images/videos 100 that are acquired by the image sensor 22.
  • the computing device 46 may include a laptop as illustrated but it may also include, for example, a personal computer, tablet PC, mobile phone, or remote computer such as a server or the like. In some embodiments, various tasks or operations may be divided between multiple computing devices 46. For example, one computing device 46 may be used to control the cytometer device 10 and acquire the images/videos 100.
  • Another computing device 46 may run the trained neural network 112 and results may be returned to the controlling computer device 46 (or another computing device 46 entirely). Of course, these tasks may be consolidated into a single computing device 46.
  • the computing device 46 includes image processing software 110 that is executed thereon or thereby that it used to process the images/videos obtained from the image sensor.
  • the computing device 46 may also be integrated into the cytometer device 10 in some embodiments. Some computations or image processing may also take place within the microcontroller 44.
  • the image processing software 110 processes the raw images 100 (i.e., image frames) that are acquired by the image sensor 22.
  • the image processing software 110 may first apply computation drift correction to mitigate for fluid shift within the sample holder 14. Heat from the image sensor 22 may cause localized movement of fluid within the sample holder 14 that causes the objects 90 contained therein to drift. This unwanted drift is corrected by the image processing software 110 that corrects the overall drift of the objects 90 between image frames.
  • the image processing software 110 uses a phase correlation method to estimate the relative translation between each frame in the holographic image sequence and uses 2D bilinear interpolation to remove the drift between frames. Details of the phase correlation method may be found in Reddy, B. S. et al., An FFT-based technique for translation, rotation, and scale-invariant image registration. IEEE Transactions on Image Processing 5, 1266–1271 (1996), which is incorporated herein by reference.
  • the image processing software 110 uses a high-pass filtered back-propagation step that calculates holographic images at different axial distances within the three- dimensional sample (see FIGS.4A-4C) using a high-pass-filtered angular spectrum propagation kernel as described herein (e.g., Eq. (1) herein).
  • Computer motion analysis (CMA) is then performed by the image processing software 110 to analyze the differences among the frames to enhance the three-dimensional contrast for periodically moving objects 90 that oscillate at the driving frequency (e.g., Eq. (2) herein). This essentially finds what oscillates while minimizing or ignoring non-oscillating objects 90.
  • a maximum intensity projection (converting 3D to 2D) and threshold-based detection to locate potential candidate objects 90.
  • An autofocusing operation is applied to each candidate object 90 in order to create an in-focus amplitude and phase video of each candidate object 90 (obtained from reconstructions).
  • the candidate objects 90 are then then classified (e.g., as positive, true, yes/negative, false, no) by the trained neural network 112 which is
  • P3D CNN densely connected pseudo-3D convolutional neural network
  • the detection technique capitalizes on the periodic oscillatory motion of the target objects 90 of interest (i.e., cells), with a large number of labeling magnetic particles 92, to specifically detect them with high throughput.
  • a pair of electromagnets 24, 26 were used to exert periodic and alternating magnetic force on the magnetic particles 92 bound to these cells of interest 90 (FIGS.1A-1C).
  • rods 28 were designed and machined that were made with magnetically soft permalloy, which were attached to the electromagnets 24, 26 to enhance the magnetic force at the sample location by ⁇ 40-fold with minimal magnetic hysteresis (see FIG.7B).
  • the labelled cells 90 due to the high viscosity of the methyl cellulose solution, the labelled cells 90 mainly demonstrated 3D rotational motion.
  • magnetic beads 92 typically form chains when they cluster under an external magnetic field, and these chains exhibit a swinging motion under the alternating magnetic field. This contrasts with the 3D rotational motion, i.e., the“rolling” motion associated with the bead-conjugated cells 90.
  • 3D rotational motion i.e., the“rolling” motion associated with the bead-conjugated cells 90.
  • the sample which contains the periodically oscillating target cells 90 and other types of unwanted background particles or objects 90, is illuminated with coherent light.
  • the interference pattern recorded by the CMOS image sensor 22 represents an in-line hologram of the target cells 90, which is partially obscured by the random speckle noise resulting from the background particles, including other unlabelled cells, cell debris and unbound magnetic particles 92. Recorded at 26.7 frames per second using the CMOS image sensor 22, these patterns exhibit spatio-temporal variations that are partially due to the controlled cell motion. This phenomenon is exploited for the rapid detection of magnetic-bead-conjugated rare cells 90 from a highly complex and noisy background.
  • FIGS.4A-4G show the detailed computational steps for the preliminary screening of cell candidates from a raw holographic image sequence.
  • a computational drift correction step mitigates the overall drift of the sample between frames.
  • a high-pass filtered back-propagation step using the angular spectrum method calculates the holographic images at different axial distances within the 3D sample.
  • a CMA step analyses the differences among the frames to enhance the 3D contrast for periodically moving objects 90 that oscillate at the driving frequency and employs time averaging to suppress the random speckle noise caused by background particles. This is then followed by a maximum intensity projection and threshold-based detection to locate potential cell candidates 90.
  • the cell candidates 90 that are detected in this preliminary screening step contain a large number of false positives, which mainly result from unbound magnetic beads 92 that form clusters under the external magnetic field. Therefore, another classification step was used (FIG.4H-4K) to improve the specificity of the final detection.
  • a densely connected P3D CNN structure was used for the trained neural network 112 to classify the holographic videos to exploit the spatial and temporal information encoded in the captured image sequence.
  • the densely connected P3D CNN structure is modified based on a recently proposed CNN structure by Qu et al. by adding dense connections. See e.g., Qiu et al., Learning spatio-temporal representation with pseudo-3d residual networks.
  • An autofocusing step is applied to each candidate object 90 to create an in-focus amplitude and phase video, which is then classified (as positive/negative) by a densely connected P3D CNN 112. These classification results are used to generate the final rare cell detection decisions and cell concentration measurements.
  • the CNN was trained and validated with manually labelled video clips generated from ten samples that were used solely for creating the training/validation datasets. This training needs to be performed only once for a given type of cell-bead conjugate.
  • This prepared sample was then loaded into the sample holder 14 (i.e., disposable capillary tube) to be screened by the computational cytometer 10. Because the capillary tube length is designed to be longer than the range of the motion of the translation stage 32 and because the capillary tube 14 was wider than the width of the CMOS sensor 22, the actual imaged volume per test (within the sample tube) is ⁇ 0.942 mL, which corresponds to ⁇ 1.177 mL of the blood sample before the enrichment process.
  • MCF7 concentrations of 0 mL -1 (negative control), 10 mL -1 , 100 mL -1 and 1000 mL -1 were tested, where three samples for each concentration were prepared and
  • FIG.5 shows the results of the blind testing of the technique using serial dilution experiments.
  • the data points correspond to a one-time testing result, where the error bars correspond to the standard deviations of the three detected concentrations at each spiked concentration.
  • the technique was able to consistently detect MCF7 cells 90 from 10 mL -1 samples, measuring a target cell concentration of 1.98 ⁇ 1.06 mL -1 .
  • the detection rate was approximately 20%.
  • the experimentally measured detection rate dropped to ⁇ 5% at a higher concentration of 1000 cells/mL.
  • the training of the deep neural network 112 inherently includes randomness, the repeatability of the network training process was further evaluated.
  • the training data was randomly and equally divided into five subsets, and five individual networks 112 were trained by assigning one different subset as the validation dataset and the combination of the remaining four subsets as the training dataset.
  • Each of the five networks was blindly tested to generate the serial dilution results.
  • the mean and standard deviation of the detected concentrations resulting from the five networks are shown in FIG.5. Overall, good consistency between the different network results is observed.
  • the under detection of the system is due to a combination of both systematic errors and random factors.
  • a major reason for under detection is the tuning of the classification network 112.
  • the classifier In the preliminary screening step, because there are typically a large number of false positive detections and a low number of true positive detections (since the target cells are quite rare), the classifier must be tuned to have an extremely low false positive rate (FPR) to have a low LoD.
  • FPR extremely low false positive rate
  • TPR true positive rate
  • Table 1 shows the concentrations of different types of cells and particles in the sample before and after the magnetic enrichment.
  • MCF7 cells were spiked into a whole blood sample at a concentration of 1.1 ⁇ 10 5 mL -1 , and enrichment was performed following the procedure reported in FIG.2. After the enrichment, the sample was loaded into a counting chamber and imaged by a 20 ⁇ 0.45NA benchtop microscope, and the particles/cells were manually counted.
  • concentration after the enrichment is normalized by a volume factor (i.e., the ratio between the volume before the enrichment and the volume after the enrichment).
  • MCF7 cells are known to form clusters and have thus been extensively used for preparing in vitro tumour models.
  • MCF7 cells were spiked at a concentration of 1.1 ⁇ 10 5 /mL (Table 1), it was observed that ⁇ 50% of the MCF7 cells formed clusters after enrichment.
  • the amount of clustering is expected to be lower at decreased MCF7 concentrations, which partially explains the reduced detection efficiency at higher cell concentrations. This clustering of cells not only reduces the overall number of target entities but may also exhibit changes in their oscillation patterns and may be misclassified by the classifier.
  • the computational cytometry technique may be applied for the detection of various types of rare cells 90 in blood or other bodily fluids using appropriately selected ligand- coated magnetic beads 92.
  • the first important advantage is its ability to detect target rare cells 90 without any additional modification such as labeling with fluorescent or radioactive compounds.
  • the same magnetic beads 92 that are used for capturing and isolation of target cells 90 from whole blood are also used for the purpose of periodic cell modulation and specific detection within a dense background. False positives are mitigated by identifying the controlled spatio-temporal patterns associated with the labeled target cells 90 through a trained deep neural network 112.
  • the technique Compared to existing approaches, the technique also has the advantages of a relatively low LoD, rapid detection and low cost, which makes it suitable for sensitive detection of rare cells 90 in resource-limited settings.
  • fluorescence imaging and Raman microscopy have been widely used to detect rare cells and have been shown to have very low LoDs (e.g., ⁇ 1 cell/mL), but they are typically limited by a high system cost and complexity.
  • the entire prototype of the computational cytometer 10 shown in FIGS.1A, 1B (excluding the function generator 30, power supply (not shown) and laptop computer 46) has a raw material cost of ⁇ $750.
  • This cost can be significantly reduced under large volume manufacturing, and currently it is mainly attributed to the image sensor 22 and frame-grabber ( ⁇ $550), the permalloy rod 28 ( ⁇ $70), and the electromagnets 24, 26 ( ⁇ $40), with the other components being much more inexpensive.
  • the power supply and function generator 30 can be replaced with cost-effective integrated circuit chips.
  • the power supply can be replaced with a 20 V power adapter (e.g., TR9KZ900T00- IMR6B, GlobTek, Inc., Northvale, NJ, USA) and a step-down converter (e.g.,
  • the function generator 30 can be replaced with an oscillator circuit built from a timer integrated circuit (e.g., NE555DR, Texas Instruments, Dallas, TX, USA). The total cost of these components would be less than $25.
  • the device 10 can be easily scaled up to include two or more parallel imaging channels to achieve a higher sample throughput, which is proportionate with the number of imaging channels.
  • MCF7 cell lines were purchased from ATCC (Manassas, Virginia, USA). Cells were plated with 10 mL of growth media in T75 flask (Corning Inc., New York, USA) at a concentration of 1 ⁇ 10 5 cells/mL. The growth media was composed of Dulbecco's Modified Eagle Medium (DMEM, Gibco ® , Life Technologies, Carlsbad, California, USA)
  • DMEM Dulbecco's Modified Eagle Medium
  • Cells 90 were grown in a humidified incubator at 37°C in a 5% CO 2 environment. Cells were harvested by treating them with 0.25% trypsin-edta (Gibco ® , Life Technologies, Carlsbad, California, USA) for 3 min 2-3 days after seeding depending on confluency. Then, cells 90 were pelleted by centrifuging for 3 min at 1200 RPM and resuspended in the growth media to a final concentration of 1 ⁇ 10 6 cells/mL.
  • FBS fetal bovine serum
  • penicillin-streptomycin Sigma-Aldrich Co., St. Louis, Missouri, USA.
  • Cells 90 were grown in a humidified incubator at 37°C in a 5% CO 2 environment. Cells were harvested by treating them with 0.25% trypsin-edta (Gibco ® , Life Technologies, Carlsbad, California, USA) for 3 min 2-3 days after seeding depending on confluency. Then, cells 90 were pelleted by centr
  • MCF7 cells 90 were serially diluted in Dulbecco's phosphate-buffered saline (DPBS, Sigma-Aldrich Co., St. Louis, Missouri, USA) at different concentrations (2 ⁇ 10 4 cells/mL, 2 ⁇ 10 3 cells/mL, and 2 ⁇ 10 2 cells/mL).
  • DPBS Dulbecco's phosphate-buffered saline
  • the dilution of MCF7 cells 90 in whole blood was prepared by mixing the cell solution with whole blood at a ratio of 1:19 (v/v). Most of the experiments were performed by mixing 200 mL of cell solution with 3.8 mL of whole blood. Healthy human whole blood (from anonymous and existing samples) was obtained from the UCLA Blood and Platelet Center.
  • Bead washing CELLection Epithelial Enrich Dynabeads® 92 (Invitrogen, Carlsbad, California, USA) were first resuspended in DPBS and vortexed for 30 sec. A magnet (DX08B-N52, K&J Magnetics, Inc., Pipersville, Pennsylvania, USA) was then used to separate the Dynabeads® 92 and the supernatant was discarded. This process was repeated three times, and the Dynabeads® 92 were resuspended in DPBS at the initial volume.
  • a magnet DX08B-N52, K&J Magnetics, Inc., Pipersville, Pennsylvania, USA
  • the ends of the capillary tube 14 were sealed with parafilm before the tube 14 was mounted onto the computational cytometer 10 for imaging and cell screening.
  • the device hardware consists of a scanning head 16 with a lensless imaging module 18, electromagnets 24, 26 and a translation stage 32.
  • the lensless imaging module 18 in FIGS.1A-1C contains a laser diode 20 (650-nm wavelength, AML-N056-650001-01, Arima Lasers Corp., Taoyuan, Taiwan) for illumination, which has an output power of ⁇ 1 mW.
  • the sample is loaded inside a capillary tube 14 with a rectangular cross section, which is placed ⁇ 7.6 cm below the light source 14.
  • a CMOS image sensor 22 (acA3800-14um, Basler, Ahrensburg, Germany) with a pixel size of 1.67 mm, which is placed below the glass tube 4 with a narrow gap ( ⁇ 1 mm), is used to capture the holographic speckle/diffraction patterns generated by the liquid sample.
  • two electromagnets 24, 26 Part #XRN-XP30 ⁇ 22, Xuan Rui Ning Co., Ltd., Leqing, Zhejiangzhou, China
  • custom-machined permalloy rods 28 are placed on either side of the glass tube 14.
  • An alternating driving current (square wave) from the function generator 30 is supplied to either of the electromagnets 24, 26, with a 180° phase shift between them, which creates alternative pulling force to the magnetic particles 92 within the sample.
  • the low level of the driving current is 0, and the high level of the driving current is ⁇ 500 mA.
  • the frequency is 1 Hz, which was experimentally optimized to maximize the signal corresponding to the magnetic bead-conjugated cancer cells 90.
  • the translation stage 32 (i.e., linear translation stage) is custom-built using off-the- shelf components.
  • a bipolar stepper motor 38 (No.324, Adafruit Industries LLC., New York, USA) with two timing pulleys 40 and a timing belt 36 is used to provide mechanical actuation, and the lensless imaging module 18 is guided by a pair of linear motion sliders and linear motion shafts 34 on either side of the scanning head.3D-printed plastic is used to construct the housing for the scanning head 16, and laser-cut acrylic is used to create the outer shell or enclosure 12 of the device 10.
  • the image acquisition procedure begins.
  • the translation stage 32 moves the scanning head 16 to a series of discrete positions along the glass tube 14. At each position, the stage stops 32, allowing the CMOS image sensor 22 to capture a sequence of 120 holograms at a frame rate of 26.7 fps before moving onto the next position.
  • the image data are saved to a solid-state drive (SSD) (which may be disposed in the computing device 46) for storage and further processing.
  • SSD solid-state drive
  • electromagnets 24.26 are relatively balanced.
  • the temperature of the CMOS image sensor 22 quickly rises when it is turned on, it tends to cause undesired flow inside the glass tube 14 due to convection.
  • a scanning pattern is engineered to reduce the local heating of the sample: if one denotes 1, 2,..., 32 as the indices of the spatially adjacent scanning positions, the scanning pattern follows 1, 9, 17, 25, 2, 10, 18, 26,.... This scanning pattern ensures that a given part of the sample cools down before the scanning head 16 moves back to its neighborhood.
  • the power to the image sensor 22 was also cut off during the transition between the two successive scanning positions, which was implemented by inserting a MOSFET-based switch into the power line of the USB cable.
  • the image processing procedure can be divided into two parts: (1) a preliminary screening step, which applies computational drift correction and MCF7 candidate detection to the entire FOV to locate target cell 90 candidates in 2D, and (2) a classification step, which refocuses the holographic image sequence to each individual MCF7 candidate 90 in its local area, generates an in-focus amplitude and phase video for each candidate, and classifies the corresponding video with a trained deep neural network 112. This procedure is further detailed below.
  • the sample fluid in the glass capillary tube 14 often drifts slowly throughout the duration of the image acquisition, which is due to e.g., the imperfect sealing at the ends of the tube and the convection due to the heat from the image sensor 22. Because the detection and classification of the target cells 90 are largely based on their periodic motion, the drifting problem should be corrected. Since the sample is embedded within a viscous methyl cellulose, minimal turbulent flow is observed, and the drifting motion within the imaged FOV is almost purely translational.
  • a phase correlation method was used to estimate the relative translation between each frame in the sequence with respect to a reference frame (chosen to be the middle frame in the holographic image sequence), and used 2D bilinear interpolation to remove the drift between frames (FIG.4B and FIG.4D).
  • the detection of the target cell candidates 90 plays a key role in automatically analyzing the sample, because it greatly narrows down the search space for the rare cells of interest and allows the subsequent deep learning-based classification to be applied to a limited number of holographic videos.
  • the lateral locations of the MCF7 candidate cells 90 are detected (FIG.4C).
  • Each frame of the raw hologram sequence was propagated to various axial distances throughout the sample volume using a high-pass-filtered angular spectrum propagation kernel, which can be written as:
  • HP( ⁇ ) denotes the high-pass filter
  • ⁇ ( ⁇ ) denotes angular spectrum propagation
  • a i denotes the i-th frame of the raw hologram sequence after the drift correction
  • zj denotes the j-th propagation (axial) distance.
  • the selected propagation distances ranged from 800 mm to 5000 mm with a step size of 100 mm to ensure coverage of all possible MCF7 candidates 90 within the sample tube.
  • a zoomed-in image of Bi(zj) corresponding to an example region is shown in FIG.4E.
  • C(x, y; z) is a 3D contrast map that has high values corresponding to the locations of periodic motion that matches the frequency of the external magnetic field.
  • An example of C is shown in FIG.4F.
  • xk and yk are the lateral centroid coordinates of the k-th potential target cell candidate.
  • Each video of the MCF7 candidate 90 was fed into a classification neural network 112 (FIG.6), which outputs the probability of having an MCF7 cell 90 in the corresponding video (FIGS.4J-4K).
  • a novel structure for the classification neural network 112 was designed, named densely connected P3D CNN, which is inspired by the Pseudo-3D Residual Network and the Densely Connected Convolutional Network.
  • the original P3D CNN used a mixture of three different designs of the P3D blocks to gain structural diversity, which resulted in a better performance.
  • each block consists of a 1 ⁇ 3 ⁇ 3 spatial convolutional layer (Conv s ), a 3 ⁇ 1 ⁇ 1 temporal convolutional layer (Conv t ), followed by a max pooling layer (Max).
  • a fully connected (FC) layer 58 with a 0.5 dropout rate and a softmax layer 60 are introduced, which output the class probability 62 (target rare cell 90 or not) for the corresponding input video.
  • a decision threshold is applied to the class probability output to generate the final positive/negative classification (FIG.4K), where the decision threshold is tuned based on the training/validation data to reduce the FPR.
  • the threshold may be based on cut-off of the class probability (e.g., if class probability is greater than .8 or 80% then this corresponds to true, otherwise false).
  • the negative training data came from only the 5 negative control experiments, where all the candidate videos from those experiments were used to construct the negative dataset.
  • the positive training data were manually labelled by two human annotators using 5 experiments spiked at 10 3 mL -1 , where only the video clips that were labelled as positive with high confidence by both annotators were selected to enter the positive training dataset, while all the others were discarded.
  • the training/validation datasets were randomly partitioned into a training set and a validation set with no overlap between the two.
  • the training set contained 1713 positive videos and 11324 negative videos.
  • the validation set contained 788 positive videos and 3622 negative videos.
  • the training dataset was further augmented by randomly mirroring and rotating the frames by 90°, 180° and 270°.
  • the convolutional layer weights were initialized using a truncated normal distribution, while the weights for the FC layer were initialized to zero.
  • Trainable parameters were optimized using an adaptive moment estimation (Adam) optimizer with a learning rate of 10 -4 and a batch size of 240.
  • the network converged after ⁇ 800-1000 epochs.
  • the network structure and hyperparameters were first optimized to achieve high sensitivity and specificity for the validation set.
  • a default decision threshold of 0.5 a sensitivity and specificity of 78.4% and 99.4%, respectively, were achieved for the validation set; a sensitivity and specificity of 77.3% and 99.5%, respectively, were achieved for the training set.
  • the decision threshold of the classifier was further tuned to avoid false positives. For this, the training and validation datasets were combined to increase the total number of examples, and the decision threshold (for positive classification) was gradually increased from 0.5 while monitoring the FPR for the combined
  • the decision threshold was tuned to a high level to suppress false positives, which in turn resulted in a very low TPR.
  • the decision threshold may be relaxed to a lower level, which also allows the TPR to be higher.
  • the current computer code which is not optimized, it takes ⁇ 80 s to pre- process the data within one FOV (corresponding to a volume of 14.7 mm 2 ⁇ 2 mm) for extracting the MCF7 cell candidates, corresponding to the preliminary screening step in FIGS.4A-4G.
  • For each detected cell candidate it takes ⁇ 5.5 s to generate the input video for network classification.
  • the network inference time for each input video is ⁇ 0.01 s. Based on these numbers, if there are, e.g., ⁇ 1,500 cell candidates per experiment, the total processing time using the current computer code would be ⁇ 3.0 hours.
  • the data processing time depends on various factors, including the computer hardware configuration, the cell concentration in the sample, the programming language and whether the code is optimized for the hardware.
  • relatively high-performance hardware an Intel Core i7 CPU, 64 GB of RAM, and an Nvidia GeForce GTX 1080Ti GPU
  • MATLAB MicroTextWorks, Natick, MA, USA
  • the code was not extensively optimized for improved speed.
  • a careful optimization of the GPU code should bring a significant speedup in computation time.
  • a custom-machined rod 42 made of permalloy (relative permeability m r ⁇ 100,000) was used to relay the force field and enhance the relative magnetic force on target cells by ⁇ 40 times.
  • a rod 42 was used for each electromagnet 24, 26.
  • FEM finite element method
  • the permalloy rod was modeled using Permendur.
  • a thick layer of air was added as a coaxial cylinder with a radius of 10 mm and a height of 30 mm.
  • the magnetic flux density inside the simulation space was simulated using the magnetic field module.
  • a coefficient form PDE module in the mathematics library was used to derive the relative magnetic force field.
  • the magnetic force that is received by superparamagnetic beads is given by:
  • V is the volume of the magnetic particle
  • c is the magnetic susceptibility
  • m0 is the magnetic permeability in a vacuum
  • B is the magnetic flux density
  • FIGS.7B-7D The simulation results are shown in FIGS.7B-7D.
  • the results in FIG.7B indicate that the relative magnetic force rapidly reduces as a function of the distance from the electromagnet. However, by using a permalloy rods 28, the relative magnetic force at the sample location is enhanced by ⁇ 40 times (FIGS.7B and 7D).
  • G1 and G2 are the high-pass filters in the spatial frequency domain, given by
  • G1 was used mainly to suppress the low- frequency interference patterns caused by the various interfaces in the light path, and G2 was used mainly to suppress the unwanted diffraction patterns due to the grooves in the capillary tubes, which is a manufacturing artifact.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Pathology (AREA)
  • Dispersion Chemistry (AREA)
  • Immunology (AREA)
  • Biochemistry (AREA)
  • Analytical Chemistry (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Vascular Medicine (AREA)
  • Databases & Information Systems (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Investigating Or Analysing Biological Materials (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un cytomètre de calcul fonctionnant à l'aide d'une imagerie par granularité sans lentille modulée magnétiquement, qui introduit un mouvement oscillatoire sur des cellules rares conjuguées à des billes magnétiques d'intérêt par l'intermédiaire d'une force magnétique périodique et qui fait intervenir une imagerie par granularité holographique à résolution temporelle sans lentille pour détecter rapidement les cellules cibles en trois dimensions La spécificité de détection est en outre améliorée par l'intermédiaire d'un classificateur fondé sur un apprentissage profond fondé sur un réseau neuronal convolutif pseudo-3D (P3D CNN) connecté densément, qui détecte automatiquement des cellules rares d'intérêt en fonction de leurs caractéristiques spatio-temporelles sous une force magnétique contrôlée. Ledit cytomètre de calcul compact, économique et à haut rendement peut être utilisé pour la détection et la quantification de cellules rares dans des fluides corporels pour une variété d'applications biomédicales.
PCT/US2020/040664 2019-07-02 2020-07-02 Cytomètre de calcul à modulation magnétique et procédés d'utilisation WO2021003369A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/621,979 US20220260481A1 (en) 2019-07-02 2020-07-02 Magnetically modulated computational cytometer and methods of use

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201962869723P 2019-07-02 2019-07-02
US62/869,723 2019-07-02

Publications (1)

Publication Number Publication Date
WO2021003369A1 true WO2021003369A1 (fr) 2021-01-07

Family

ID=74101150

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2020/040664 WO2021003369A1 (fr) 2019-07-02 2020-07-02 Cytomètre de calcul à modulation magnétique et procédés d'utilisation

Country Status (2)

Country Link
US (1) US20220260481A1 (fr)
WO (1) WO2021003369A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023028024A1 (fr) * 2021-08-23 2023-03-02 Nirrin Technologies, Inc. Imagerie par granularité laser pour la quantification de cellules vivantes
US11915360B2 (en) 2020-10-20 2024-02-27 The Regents Of The University Of California Volumetric microscopy methods and systems using recurrent neural networks
US11946854B2 (en) 2018-12-26 2024-04-02 The Regents Of The University Of California Systems and methods for two-dimensional fluorescence wave propagation onto surfaces using deep learning
US12020165B2 (en) 2019-11-14 2024-06-25 The Regents Of The University Of California System and method for transforming holographic microscopy images to microscopy images of various modalities

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6033574A (en) * 1995-02-21 2000-03-07 Siddiqi; Iqbal W. Method for mixing and separation employing magnetic particles
US7764821B2 (en) * 2002-02-14 2010-07-27 Veridex, Llc Methods and algorithms for cell enumeration in a low-cost cytometer
US20190049356A1 (en) * 2015-06-19 2019-02-14 Captl Llc Multiple-Spot Time-Sequential Cytometry
US10267736B2 (en) * 2014-09-30 2019-04-23 The Regents Of The University Of California Imaging flow cytometer using spatial-temporal transformation
US10295451B2 (en) * 2015-06-30 2019-05-21 Imec Vzw Holographic device and object sorting system

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
SE535918C2 (sv) * 2010-06-10 2013-02-19 Hemocue Ab Detektion av magnetiskt inmärkta biologiska komponenter
US9605941B2 (en) * 2011-01-06 2017-03-28 The Regents Of The University Of California Lens-free tomographic imaging devices and methods
GB2497700A (en) * 2011-01-12 2013-06-19 Idea Machine Dev Design & Production Ltd Compact microscopy system and method
IT201700006925A1 (it) * 2017-01-23 2018-07-23 Fondazione St Italiano Tecnologia Metodo e apparato per il tracciamento ottico di oggetti emettitori.
CN111183350A (zh) * 2017-08-17 2020-05-19 雅培医护站股份有限公司 用于对血细胞成像的单次使用测试设备
US20190056384A1 (en) * 2017-08-17 2019-02-21 Abbott Point Of Care Inc. Single-use test device for imaging assay beads
EP3721209B1 (fr) * 2017-10-15 2024-02-07 Berkeley Lights, Inc. Procédés pour essais « en enclos »

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6033574A (en) * 1995-02-21 2000-03-07 Siddiqi; Iqbal W. Method for mixing and separation employing magnetic particles
US7764821B2 (en) * 2002-02-14 2010-07-27 Veridex, Llc Methods and algorithms for cell enumeration in a low-cost cytometer
US10267736B2 (en) * 2014-09-30 2019-04-23 The Regents Of The University Of California Imaging flow cytometer using spatial-temporal transformation
US20190049356A1 (en) * 2015-06-19 2019-02-14 Captl Llc Multiple-Spot Time-Sequential Cytometry
US10295451B2 (en) * 2015-06-30 2019-05-21 Imec Vzw Holographic device and object sorting system

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11946854B2 (en) 2018-12-26 2024-04-02 The Regents Of The University Of California Systems and methods for two-dimensional fluorescence wave propagation onto surfaces using deep learning
US12020165B2 (en) 2019-11-14 2024-06-25 The Regents Of The University Of California System and method for transforming holographic microscopy images to microscopy images of various modalities
US11915360B2 (en) 2020-10-20 2024-02-27 The Regents Of The University Of California Volumetric microscopy methods and systems using recurrent neural networks
WO2023028024A1 (fr) * 2021-08-23 2023-03-02 Nirrin Technologies, Inc. Imagerie par granularité laser pour la quantification de cellules vivantes

Also Published As

Publication number Publication date
US20220260481A1 (en) 2022-08-18

Similar Documents

Publication Publication Date Title
JP7261414B2 (ja) 分析方法
US20220260481A1 (en) Magnetically modulated computational cytometer and methods of use
US11668641B2 (en) Image-based cell sorting systems and methods
Zhang et al. Computational cytometer based on magnetically modulated coherent imaging and deep learning
Gopakumar et al. Convolutional neural network‐based malaria diagnosis from focus stack of blood smear images acquired using custom‐built slide scanner
TWI756365B (zh) 圖像分析系統及相關方法
Kervrann et al. A guided tour of selected image processing and analysis methods for fluorescence and electron microscopy
Wang et al. Smartphone-based platforms implementing microfluidic detection with image-based artificial intelligence
Huang et al. A contact-imaging based microfluidic cytometer with machine-learning for single-frame super-resolution processing
US20140365161A1 (en) Material analysis system, method and device
CN101151623A (zh) 分类图像特征
WO2020082030A1 (fr) Dispositif et procédé de détection sans étiquette basée sur la motilité d'objets motiles dans un échantillon de fluide
Li et al. Accurate label-free 3-part leukocyte recognition with single cell lens-free imaging flow cytometry
Ota et al. Implementing machine learning methods for imaging flow cytometry
Simon et al. Shallow cnn with lstm layer for tuberculosis detection in microscopic images
Maaliw et al. A multistage transfer learning approach for acute lymphoblastic leukemia classification
US10664978B2 (en) Methods, systems, and computer readable media for using synthetically trained deep neural networks for automated tracking of particles in diverse video microscopy data sets
US20140192178A1 (en) Method and system for tracking motion of microscopic objects within a three-dimensional volume
Mudugamuwa et al. Review on Photomicrography Based Full Blood Count (FBC) Testing and Recent Advancements
Kurenkov et al. Advancing precision single-cell analysis of red blood cells through semi-supervised deep learning using database of patients with post-COVID-19 syndrome
Chenouard Advances in probabilistic particle tracking for biomedical imaging
Christian Ramarolahy et al. Classification and generation of microscopy images with plasmodium falciparum via artificial neural networks
Wang et al. Lens-free microfluidic-based method for detecting circulating tumor cells
WO2024006670A2 (fr) Méthode des plages de lyse virale sans colorant, rapide et quantitative utilisant l'apprentissage profond et l'holographie
Tran et al. Mobile Fluorescence Imaging and Protein Crystal Recognition

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20834843

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20834843

Country of ref document: EP

Kind code of ref document: A1