CN112566721A

CN112566721A - Method and apparatus for controlling and manipulating multiphase flow in microfluidics using artificial intelligence

Info

Publication number: CN112566721A
Application number: CN201980009308.3A
Authority: CN
Inventors: 阮南唐; 高勇升; 周军; 谭萨华
Original assignee: Hangzhou Chunxun Biotechnology Co ltd
Current assignee: Hangzhou Chunxun Biotechnology Co ltd
Priority date: 2018-05-28
Filing date: 2019-03-12
Publication date: 2021-03-26
Also published as: AU2019101835A4; WO2019227126A9; AU2019277193A1; WO2019227126A1

Abstract

The present invention relates to a system for controlling microfluidic droplets or bubbles. The system includes a microfluidic generator for generating microdroplets or bubbles. A feedback sensor is provided for sensing one or more feedback parameters of the generated micro-droplets or bubbles. A controller is provided for controlling the microfluidic generator using the sensed feedback parameters.

Description

Method and apparatus for controlling and manipulating multiphase flow in microfluidics using artificial intelligence

Technical Field

The present invention relates generally to sub-millimeter sized micro-droplets or bubbles.

In particular, but not exclusively, the invention can be applied to lab-on-chip platforms, biochemical and chemical analysis, biochemical analysis, particle and material analysis, pharmaceutical and cosmetic, particle and cell sorting systems, cell sealing or material synthesis, etc.

Background

The reference to prior art in this specification is not, and should not be taken as, an acknowledgment or any form of suggestion that the prior art forms part of the common general knowledge.

Micro-droplets with desired parameters are desired for many chemical, biochemical, materials, cosmetic, pharmaceutical and biological processes.

In biochemical applications, droplets with discrete fluid volumes can be handled and manipulated independently. Each droplet can act as a separate microreactor in which some reactions can be operated at high throughput (e.g., several droplets per millisecond). Another advantage is that a high surface area to volume ratio in the droplets can be used to increase the reaction rate, or to facilitate heat or material exchange.

In pharmaceutical applications, whether in designing drugs, personalized drugs, or various medical imaging techniques, droplets having desired parameters are well suited as medical drugs (usually in the form of droplets). In particular, the wearable chip device may be configured to be fitted to a human body to provide a customized dose of medication. In medical imaging, droplets sealed within various cells can be imaged, detected, separated, and counted.

In biological applications, the droplets may contain cells and act as bioreactors. Cells can be assembled and cultured into tissues or organoids. Cells such as sperm and zygotes can be sorted to initiate artificial propagation including artificial insemination, in vitro fertilization, cloning, and embryo division or cleavage.

It is well known that droplets can be generated by passing immiscible fluids into microchannels of a microfluidic generator, the microchannels being connected by channel junctions. The fluid flows into a junction and droplets of one fluid (dispersed phase fluid) are created in another fluid (continuous phase fluid). The droplet diameter and frequency at which the droplets are generated are determined primarily by the geometry of the channel, the flow rate of the fluid, and fluid properties such as viscosity, surface tension. The existing methods for generating liquid drops include methods using electric field, thermal energy, pneumatic, acoustic control, and magnetic field.

The diameter of the droplets is adjusted by changing the flow rate or the pressure applied to the channel. Typically, droplet parameters such as droplet size are measured after droplet generation, and then repeated verification is required to control the parameters of the droplet. Typically, the drop parameters cannot be fully sensed in real time at all.

This preferred embodiment provides a system for controlling microdroplets that is more efficient than iterative verification. The preferred embodiment also compensates for parameters that are not reliably sensed.

Droplet-based microfluidics are directed to the generation and manipulation of discrete droplets in chemical, biological and material processing microfluidic systems. The individual droplets can act as microreactors or compartments, providing many advantages, such as separation or spacing of different samples for analysis, reduced reagent consumption and reaction time, and the ability to perform large scale assays in a single device. These advantages enable its use in many areas ranging from music to single cell genomics. Due to the small size and high speed of droplet generation, a method of rapidly sorting droplets is needed to obtain the desired number of particles or cells for analysis.

Sorting droplets is an important process for transporting and distributing droplets into different channels downstream of the generating device. One established method of droplet sorting is Fluorescence Activated Droplet Sorting (FADS) (j. -c. baret et al. in "Lab on a Chip", volume 9,2009, p.1850-1858), which is capable of sorting approximately 2000 fluorescence activated droplets per second using an electric field. To date, electric drive is the best choice because it is the most reliable, accurate and fast reaction, which is essential for high speed sorting.

A common disadvantage of conventional droplet sorting systems based on electrical driving is that the biological sample has to be labeled with a fluorescent dye. Labeled microparticles are time consuming, dyes are not compatible with all cell types, are highly undesirable, and may affect cell behavior during live cell imaging. In some cases, cells can only be fluorescently labeled with dead cells.

For viewing the sample in its natural state, a label-free sorting system is preferred. This is achievable on the basis of most microscopes equipped with a camera, but requiring redesign of the image acquisition device and advanced processing algorithms. Early attempts were made to detect actinomycetes cultured in picoliters of liquid (E.Zang et al. in "Lab on a Chip", volume 13,2013, p.3707-3713) by means of photodiodes. The resulting signal is converted to a TTL signal to fire the camera and then to fire the frame grabber. The captured frames are processed by an a/D converter (ADC) with the contents of the droplet having a classification to trigger the generator to sort. The disadvantage of this system is that at a sorting frequency of 100Hz, the true positive rate of sorting is only 70%. This ratio is low compared to the FADS system, since it employs a simple rule-based classification method based on droplets.

Another design is to capture the drop image with a low-speed camera, using multiple template matching algorithms to detect encapsulated cells or objects in the drop (m.girault et al, "Scientific reports," volume 7,2017,40072). The accuracy of this system for successful identification and discrimination of droplets containing d.tertiolecta and p.tricornutum was 91 ± 4.5% and 90 ± 3.8%, respectively. However, the processing rate is only 10 Hz. This indicates that better imaging and processing systems must be developed to compete with FADS.

While computer vision has made great progress in the past few years in terms of efficient and effective droplets, real-time microdroplet analysis remains a challenging problem. First, the micro-droplets move very rapidly in the microfluidic system. High-throughput cameras need to track the movement process with hundreds or thousands of images captured per second. This also puts higher demands on the required image processing and object recognition methods. Second, the illumination is significantly changed given the focus setting of the microscope or the brightness of the light source. This requires image processing methods to adapt to such conditions. Third, the size of the target, e.g., particle or cell, is detected to change significantly during movement, which makes identification difficult.

Alternative embodiments seek to minimize the disadvantages of droplet sorting described above, or at least provide a useful commercial choice.

Disclosure of Invention

According to a first aspect of the present disclosure, there is provided a system for controlling micro-droplets or bubbles, comprising:

a microfluidic generator for generating said micro-droplets or bubbles;

a feedback sensor for sensing one or more feedback parameters of the generated micro-droplets or bubbles; and

a controller for controlling the microfluidic generator using the sensed feedback parameter.

An advantage of such a system is that the controller utilizes sensed feedback parameters to control the microfluidic device, which is more efficient than multiple trials.

The feedback sensor may include a predictor to predict parameters that may not be sensed or reliable, and further use the predicted parameters to control the microfluidic generator.

The controller may construct a model using the training data from the feedback sensor and test the resulting droplets or bubbles to confirm the accuracy of the model.

The controller may comprise an artificial intelligence controller, such as a machine learning classifier. The controller may comprise a pressure driven flow controller or syringe pump. Preferably, the controller is configured to control the microfluidic device such that the microfluidic device produces a monodisperse phase emulsion of microdroplets or bubbles.

The sensor may comprise an optical system. The optical system can acquire information such as the size, shape and color of the gas, liquid or solid phase in the stream of micro-droplets or bubbles. These parameters may include any one or more of the following: droplet area, diameter, shape, frequency, separation distance, and velocity.

According to another aspect of the present invention, there is provided a method of controlling micro-droplets or bubbles, comprising:

generating the micro-droplets or bubbles;

sensing one or more feedback parameters of the generated micro-droplets or bubbles; and

controlling the microfluidic generator with the sensed feedback parameter.

The controlling step may include a training phase comprising constructing a model using training data from the feedback sensors. The training phase may include determining a training area by pressure characteristics of the generator while performing the generating step. The training area is between the advection and stability domain transition lines. This training phase may include exploring the training boundaries in a serpentine fashion.

The model may be a regression model. This regression model may include determining the optimal regression parameters by minimizing the loss of training data.

The step of controlling may further comprise a testing stage to test the generated droplets or bubbles to determine the accuracy of the model. This test may include inputting a desired drop area and shape into the model. Accuracy may contain less than 2.5% error when the input expected drop area and shape is compared to the generated drop.

According to another aspect of the present invention, there is provided a system for sorting micro-droplets or bubbles, the system comprising:

a sensor for sensing one or more parameters of the microdroplets or bubbles, or their contents;

a sorter for sorting the micro-droplets or bubbles; and

a controller for controlling the sorter using the sensed parameter.

Preferably, the sorter sorts the droplets or bubbles depending on whether they contain particles and without the need for droplet marking. The defects of the traditional non-marking liquid drop sorting technology are avoided, and the application range is expanded. Sorting depends on whether the sensed parameter relates to droplets containing particles, containing a single particle, multiple particles, or particles of different nature and type. These microparticles may be biological cells such as blood cells, cancer cells, sperm or eggs.

The system may further comprise a microfluidic generator for generating microdroplets or bubbles. The system may further comprise spacers for spacing the generated droplets or bubbles. The generator, spacer and sorter may be integrated together as a microfluidic device. The microfluidic device may be formed using lithographic techniques.

The sensor may comprise an imaging device. The imaging device may include a microscope, a camera for capturing an image frame magnified by the microscope, and a frame grabber for grabbing the image frame from the camera.

The controller may include a machine learning model. The controller may comprise a pressure controller or a syringe pump.

The sorter may operate according to one or more of the following: hydrodynamics, pneumatics, electromotion, dielectrophoresis, magnetism and thermal principles.

According to another aspect of the present invention, there is provided a method of sorting micro-droplets or bubbles, the method comprising:

sensing one or more parameters of the micro-droplets, bubbles, or their contents;

sorting micro-droplets or bubbles; and

controlling the sorter using the sensed parameter.

Preferably, sensing includes using computer vision and controlling includes using machine learning or sorting is in real time.

The sensing may include a snapshot of the drop image frame web. The method may determine a target area containing one droplet per frame. This method may include droplet tracking or droplet sorting. The drop tracking may utilize Hough transform or other boundary detection methods to detect the circular boundaries of drops. The droplet classification may comprise voting.

Sorting may include accepting droplets or bubbles containing a single particle, rejecting other droplets or bubbles in the flow. The sorting may be initiated by an electrical drive. The sorting may include synchronizing with a droplet generation and identification module. The sorting may determine the contents it contains based on the droplet classification.

The control may include machine learning involving off-line training and on-line sorting using results of the off-line training.

The method may further comprise generating micro-droplets or bubbles. The controlling may include analyzing droplet generation and sorting. The method may further comprise spacing or separating the generated droplets or bubbles in the flow.

Any feature described herein may be combined with any one or more other features described herein, in any combination, within the scope of the invention.

Drawings

Preferred features, embodiments and variations of the present invention will become apparent from the following detailed description, which provides those skilled in the art with sufficient information to practice the invention. These detailed descriptions should not be construed as limiting the scope of the above summary in any way. The detailed description will refer to the following several figures:

FIG. 1a is a schematic diagram of a system for controlling microdroplets, according to a first embodiment of the present invention.

FIG. 1b shows a flow chart of the training and testing phase of the system of FIG. 1 a.

FIG. 2 shows a flow diagram of the different flow zones observed in the system described in FIG. 1 a.

Fig. 3 shows a flow chart of a procedure for capturing training data.

FIG. 4a is a 3D plot showing droplet areas and shapes at P1 and P2 obtained from training data with A, B, C, and D showing the training boundaries of the system.

FIG. 4b is a 2D plot of the same data as FIG. 4a, with oil pressure P1 corresponding to the drop area.

Fig. 4c is a 2D plot of the same data as fig. 4a, wherein the water pressure P2 corresponds to the drop area.

FIG. 5 is a user interface of the system of FIG. 1a showing real-time droplet analysis.

Fig. 6 shows the comparison of the monitored droplet area with the user-entered desired droplet area, where two independent experiments (a & b) were performed to compare the results of individual training with parameter optimization.

Fig. 7 is a schematic diagram of a system for sorting micro-droplets according to a second embodiment of the present invention.

Fig. 8a shows details of the design of the microfluidic device of the system described in fig. 7.

FIG. 8b shows the scale dimensions of the microfluidic generator of the microfluidic device of FIG. 8a

Figure 8c shows the spacer size of the microfluidic device of figure 8 a.

Figure 8d shows the size of the sorter of the microfluidic device of figure 8 a.

Fig. 9 shows the processing of a target area (ROI) in a captured video frame.

Fig. 10 is a flow chart of droplet identification within the ROI of fig. 9.

Fig. 11 is a flow chart of a machine learning method of the system of fig. 7, including off-line training and on-line sorting.

FIG. 12 is a flow diagram of water pressure versus oil pressure for producing droplets in the system of FIG. 7.

Figure 13 shows a plot of barrier oil pressure versus drop frequency observed at the drop sorting junction of the sorter of the system described in figure 7.

Figure 14 shows the accuracy of experimental classification, voting and sorting at different droplet frequencies.

Figure 15 shows the experimental classification of different voting choices, the accuracy of voting and sorting.

Detailed Description

In accordance with an embodiment of the present invention, there is provided a system 100, shown in fig. 1, for controlling microdroplets of a monodisperse phase fluid in a continuous phase fluid.

The system 100 includes a microfluidic generator 102 for generating microdroplets 104. A feedback sensor 106 senses a feedback parameter of the generated micro-droplets 104. The feedback sensor 106 includes a predictor 107, and the predictor 107 may predict parameters that are not sensed or are not reliable. The system 100 also includes a controller 108 that controls the microfluidic generator 102 using sensed and predicted feedback parameters. Preferably, the controller 108 utilizes feedback parameters to automatically control the microfluidic generator 102 in real time, which is more efficient than trial and error or trial and error methods. The system 100 is described in detail below.

The microfluidic generator 102 is a microfluidic focusing device fabricated in polydimethylsiloxane (PDMS, Dow Corning) using standard photolithography and soft lithography procedures. The controller 108 is configured to control the microfluidic device 102 such that the microfluidic device 102 produces a monodisperse phase emulsion of the microfluidic droplets 104.

The feedback sensor 106 includes an optical system that obtains information about the size, shape, and color of the gas, liquid, or solid phase of the fluid flow of the microfluidic droplet 104. The sensor 106 also includes an analyzer for analyzing the information obtained to determine droplet parameters.

To further facilitate real-time imaging and data conversion for real-time measurements, a high performance camera (CP90, Optronis) and a custom computer are provided. The computer is configured with a Core i 76850K Central Processing Unit (CPU), 128GB Random Access Memory (RAM), Direct Memory Access (DMA) controller and frame grabber (VQ8-CXP6D, Silicon Software mE5 Ironman). The camera is equipped with a 4 megapixel monochrome image sensor and captures images at full resolution (2304x 1720) at 500 frames per second. The captured images are then transmitted from the camera in near real-time via the coaxpress interface (CX-304-1-304-05, Optronis). The coaxpress interface is a high speed multiple (4-core) coax cable and is directly connected to the frame grabber. These images take on average about a few picoseconds to transmit to the frame grabber. After the frame grabber receives the image from the camera, the image is transferred line by line to the computer RAM using the DMA controller. The DMA controller sends the data directly to main memory and bypasses the computer CPU to speed up the storage of the memory. The pressure controller 108 is integrated with the predictor 107 via the (OB1 MK3, Elveflow) Software Development Kit (SDK).

This predictor 107 is executed using a cross-platform development tool QT (QT, QT project). Several libraries are used here, including armadillo and OpenCV libraries, as well as Application Programming Interfaces (APIs) for the pressure controller 108, image collector, and camera. For program efficiency, five threads are used. These threads are responsible for the following tasks, respectively: frame capture and memory buffering, calculation of droplet parameters, display of droplet parameters and video and system parameters on a graphical interface, saving of the results of the calculations, and interaction with the pressure controller 108. When the desired droplet size is entered into the system 100 via the graphical interface, the corresponding pressure value is immediately calculated by regressing the model and passing it to the pressure controller 108. The pressure controller 108 sets specific oil and water pressure values that will immediately change the droplet area. This effect is captured by the thread that measures the droplet parameters, which is used to determine whether the droplet has reached the desired size. In video, aggregation requires only a few frames and can be achieved in about 1-2 seconds, allowing the predictor 107 to perform near real-time tasks.

FIG. 1b shows the predicted framework structure. The prediction includes a training phase in which the model is built using training data from the feedback sensor 106, and a testing phase 112 in which the generated droplets are tested to determine the accuracy of the model.

First, the system 100 goes through a training phase 110 to collect the data needed to build a predetermined model. At this 110 stage, real-time measurements of various droplet parameters, such as droplet spacing distance and area, are also collected and validated with manual image analysis (ImageJ, NIH).

After the model is built, the system 100 is then tested to determine the accuracy and reliability of the model. The model is verified by setting criteria that the produced drops do not deviate more than 5% from the corresponding set values. In this embodiment, the focus is only on the droplet area (assuming the droplet is spherical) for the generation of the model, since the corresponding droplet volume is more important in most applications. However, in other embodiments, other drop parameters (drop diameter, shape, frequency, separation distance and velocity) may be programmed into the constructed model.

Turning to fig. 2, before training is initiated, the operator manually confirms the appropriate droplet generation protocol to reduce down time due to flow instability that can occur when the protocol is changed to advection or reflux. The fact is that this step can also be automated, further reducing time and costs. An error bar can be obtained for 5 replicates. P1 is oil pressure and P2 is water pressure. The pressure values for the stable interface (red line) and parallel flow regime (black line) are given, between which a solution for droplet generation can be observed. Reflux was observed above the red line. For model generation, training requires uninterrupted droplet generation. The scale bar of the inset in fig. 2 represents 100 um.

The limits of the training area 200 are then labeled with four boundary points A, B, C and D. These points are selected to be close to the stability interface and the transition line of the advection to ensure that the training area 200 used is maximized and does not change during the training phase. These points are also outside the error line of the two transition lines to ensure that the system also maintains a stable droplet production regime at all times.

Fig. 3 shows a routine 300 for obtaining the desired training data. Briefly, the initial pressure values (P1 and P2) were initially set at point a. The system 100 then measures the area of the droplets produced to determine the stability of the system 100. When the 10 consecutive drop areas are within 2% of the average drop area, the system is stable. After stabilization, the system was moved in a serpentine motion at 5 mbar intervals to the next pressure value setting. When the system reaches point D, the training process is complete. The settling time at each set pressure value is typically 1-36 seconds, depending on the flow conditions.

In the current example, the longest stabilization time was observed to be reached when P1 was 285 mBar (mBar) and P2 was 155 mBar (mBar). The separation distance between two droplets at this point is about 544 um. This means that using the current magnification and image resolution requires more frames to be acquired to obtain 10 consecutive drop images.

In summary, using 2600 different pressure values, approximately 650000 sets of data were collected and processed during the training phase. Approximately 250 sets of data were collected for each pressure value. The pressure driven flow controller 108 used to generate the droplets 104 comprises a set of syringe pumps. However, a syringe pump may not be used, considering that the time to achieve stabilization may be significantly longer (typically on the order of several minutes). The separation distance of the droplets 104 was also measured and used for model building.

The method 300 of FIG. 3 enables the system 100 to look up different pressure values in a serpentine movement. At each pressure value, the captured droplet area was used for the subsequent M-SVR model construction. A close-up 302 of initial training pressure values shows serpentine movement region 304 intersecting training region 200. When the force value is at the training boundary, the method 300 adjusts the force value accordingly and changes the query direction.

In order to enable the system 100 to predetermine the pressure values required for a fixed drop area, a regression model is constructed using machine learning methods to describe the relationship between the relevant variables. This method is called multivariate support vector regression (M-SVR), which is used for Multiple Input Multiple Output (MIMO) systems. This machine learning technique is able to predict target variables through a model constructed based on initial training data. Other machine learning techniques may also be used to construct the regression model.

Given a training data set of n samples (x)₁，y₁)，…，(x_n,y_n) Wherein x is_i＝(a_i，s_i) I-1, …, n is an area containing a droplet_iAnd the separation distance s of two adjacent droplets_iThe vector of (i), yi ═ p₁，p₂) Involving oil pressure p₁And water pressure p₂. The purpose of model construction is to define y ═<w，x>+ b to estimate a set of multi-dimensional regression parameters from the training data.

W＝[w¹，...，w^Q]

b＝[b¹，...，b^Q]^T

Since the model takes two input variables and generates two output variables, i.e., Q ═ 2, W is a 2x2 matrix and b is a 2x1 vector. Note that the regression model can be extended to more input and output variables as the Q value increases.

To obtain the optical regression parameters, the following loss function is minimized:

L(u_i) Is a normalized loss function with a gaussian kernel. The first term in the above equation is the regularization term that controls the complexity of the learning model. C is a contribution to balance losses and complexity. Based on these equations, a multi-dimensional hyperplane model is generated. Optimization can be achieved by solving a quadratic approximation of the equation for the loss function in the equation, thereby minimizing the loss on the training data.

Results

During the training phase, several variables need to be adjusted. Including the ratio between the two inputs and the M-SVR parameters (e.g., the parameter C and other relevant parameters in the loss function). The adjustment of these parameters is based on feedback from the droplet area prediction process. Setting a desired drop area value, the pressure value predicted by the learning regression model will be used to generate the drop. The area of the drop is then automatically calculated and compared to the desired value. When a set of model parameters yields error values below the error criteria, they are selected for the final system. During the parameter adjustment step, a greedy strategy is followed, i.e. the parameters are individually adjusted by fixing all other parameters. Once the optimum value is reached, the parameter value is fixed when the next parameter is adjusted.

Other factors that affect the performance of the predictor 107 are the choice of kernel. In system 100, a non-linear gaussian function is used. This is a choice made after comparing both the linear and non-linear kernels, including polynomials and gaussian functions. The experimental results show that the gaussian function is the best of all options. After the M-SVR model is trained, predictions are made by applying the model to an invisible test sample and calculating the required oil and water pressures given the desired droplet size and fixed separation distance. Fig. 4 shows a 3D plot 400 illustrating the droplet areas at different points P1 and P2 obtained from training data.

The training phase takes on average about 20 minutes, including the time for data collection and parameter adjustment. Given a set of collected data, the model construction takes about 10 seconds. In the prediction process, once the desired drop size is set, the system 100 automatically predicts the optimal pressure, which may also be input to the pressure controller 108. This prediction process is very fast, and any bottleneck at time comes mainly from the training and parameter tuning stages. However, for a fixed geometry and fixed fluid set up device, this stage only needs to be operated once. In addition, training time may be reduced by reducing the number of data points that need to be collected in the training area 200 of FIG. 2, or by relaxing the standard of deviation in the parameter adjustment process.

In view of the above-mentioned system setup, several experiments were performed to calculate the predicted effect and efficiency. The purpose of the first test was to assess the accuracy of the real-time automatic droplet parameter measurement step. This step provides for the input of a regression model, including droplet separation distance and droplet area. The system measurements for 10 different drops were also compared to ImageJ's hand data. For each droplet, the manual measurement calculation was repeated 5 times to obtain the mean and standard deviation. This comparison value also serves as an illustration that human measurements may be subject to uncertainty or error. The results are summarized in the table below. It can be seen from the table that the results of the automatic drop parameter measurements have approached the results of a human operator. The maximum difference in droplet spacing and area was 2.65% and 1.67%, respectively. On the one hand, this ensures a high accuracy of the droplet parameter evaluation of the system 100. On the other hand, it is also clear that manual measurement produces different values each time unless great effort is made to ensure accuracy. It is emphasized here that automatic measurement can produce objective consistent results for the same drop or image. This property is highly desirable and will be useful in many droplet-based microfluidic applications.

The table is a comparison between manual and system measurements. The manual measurements were analyzed using ImageJ and repeated 5 times.

In a second experiment, the accuracy of the system 100 was calculated based on the difference between the expected droplet size and the actual measured droplet size at the predicted pressure setting. To do so, the input range on the user input panel 500 as shown in FIG. 5 is 7.0x10³ um²To 11.0x10³ um²The system 100 was tested for the drop area value. Two runs were run at different times, but using the same apparatus 102 and operating procedure. In the first round, the parameter adjustment complies with a broad version with an upper error limit of 5%. For the second round, this upper limit is reduced to 2.5%. To minimize the differences caused by the configuration, the experiment was performed using one device 102.

For initiating the training session, the user manually sets the pressure values for

channels

1 and 2 at user input panel 502 to determine four points A, B, C and D. The "initialize" button 503 is used to begin training. After the model is generated, the user can then set the desired drop area. The system 100 determines the most appropriate point and presents the P1 and P2 values next to the text box. The bottom panel 504 shows the reading of the system analysis results.

The results of the first and second rounds are shown in fig. 6a and 6b, respectively. This result indicates that the error between the desired drop area and the actual drop area can be adjusted to a very low level to achieve a specified range of drop areas. Notably, this area range was chosen because it is within the measurements obtained for the training data (fig. 3). The error obtained in fig. 6 is caused by various factors. First, there are small pressure pulsations induced by the piezoelectric regulator within the pressure controller 108. These pulsations translate into small pressure changes that can lead to observable differences. Second, the regression model is not 100% accurate since only the connection between droplet size, separation distance, and pressure is modeled. Due to the long operating time, other factors such as drop velocity and frequency, device stability and dimensional variations can also affect the relationship between the parameters. These factors are also incorporated into the model.

Conclusion

The above illustrates the use of the system 100 with a machine learning approach for real-time automated droplet analysis and size prediction. During training data acquisition, drop parameters such as drop area and separation distance are evaluated using image-based techniques. The system 100 develops a fluid pressure that analyzes these parameters back and forth using a model of the M-SVR machine learning technique.

The model is then used to predict pressure values over the area of the drop desired by the user input. The test results show that the error of the generated drop from the expected drop area input by the user is less than 2.5%. This system 100 allows accurate droplet generation without repeated manual adjustments. This system has also proven to be a cost effective solution for various biomedical applications requiring precise droplet sizes.

An artificial intelligence (Al) controller 108, including machine learning, provides learning schemes such as supervised, unsupervised, semi-supervised, and deep learning, using visible data from the model to predict invisible data. For example, the Al controller 108 can learn the relationships between the active controls and then build a model to predict the desired drop parameters. The use of active droplet generation provides an additional level of management and freedom to manipulate droplets. For example, these active controls allow droplets to be formed on demand.

Artificial intelligence system for real-time unmarked micro-droplet sorting

Another embodiment of the present invention is described in detail below with respect to an intelligent droplet system 700 for real-time sorting of label-free microdroplets 104 containing microparticles. This example, not exclusively, has particular applicability to lab-on-chip platforms, biochemical and chemical analysis, biochemical assays, particle and material analysis, pharmaceutical and cosmetic, particle and cell sorting systems, cell encapsulation or material synthesis.

According to this embodiment, the system 700 for sorting microfluidic droplets 104 comprises an optical sensor 702 to sense a parameter of the microfluidic droplet 104. The sensor 702 includes an imaging device 703 that integrates a microscope, camera and frame grabber. The high speed imaging and data conversion device 703 enables video sequences of the microdroplets 104 to be captured at a frame rate of 1000+ and then converted into the memory of a computer system for further processing.

The microfluidic device 704 includes a sorter 706 to sort the microfluidic droplets 104. For example, useful droplets 104 containing a single particle are sorted from other, non-useful droplets. The microfluidic device 704 further comprises a microfluidic generator 705 that is controlled by the controller 708 to generate microfluidic droplets 104. The microfluidic device 704 further includes a spacer (or spacer) 709 to space the generated droplets 104.

The system 700 further includes a controller 708, the controller 708 controlling the sorter 706 through an electric actuator 711 and using the sensed parameter. The sorter 706 sorts the droplets 104 according to whether they contain particles and without droplet marking. The defects of the traditional non-marking liquid drop sorting technology are avoided, and the application range is expanded. Controller 708 includes a classifier 710 and a pressure controller 712.

The microfluidic device 704 is a polydimethylsiloxane (PDMS, Dow Corning) droplet sorting microfluidic device. The device 704 is fabricated using conventional soft lithography and photolithography techniques. This PDMS layer consists of fluidic and electrode channels plasma bonded to a glass slide (7101, Sail brand). The flow channel was then further hydrophobized with Aquapel (PPG industries) prior to the experiment. The apparatus 704 includes three

key modules

705, 709, and 706 that define droplet generation, isolate oil connection points and sorting areas.

Using the fluid focusing module, droplets 104 are generated by generator 705. HFE7500 oil (Novec, Sigma Aldrich) with 5% (w/w) Krytox ammonium salt was used as surfactant both as continuous phase and as spacer oil, while deionized water containing 5% wt of 7 micron polymer beads (78462, Sigma Aldrich)) was used as dispersed phase. All fluids were driven using a pressure controller (OB1-MK3, Elveflow) with flow pressures ranging from 0 to 1000 mbar. In the middle portion of the device, the droplets 104 are separated by spacer oil junctions provided by spacers (or separators) 709. The pressure flow is adjusted accordingly to prevent the generated droplets 104 from flowing into the spacer oil channel.

Droplet generation and droplet spacing is controlled by pressure controller 712 adjusting the flow rate pressure through the inlet. To obtain stably generated droplets 104, images of the droplets 104 are captured by the high-speed camera 703. The system is then activated to detect, track, count the droplets 104 and classify them as containing no, single or multiple particles using the classifier 710. The droplets of the individual particles are then sorted by the electrical drive of the actuator 711. The red inset of fig. 7 shows an enlarged view of the sorting area, wherein droplets of a single particle are directed into the sorting channel and other droplets enter the waste channel.

The sorting of the droplets 104 using the actuator 711 is performed by a dc electric field. The electrode channels were filled with Indium alloy (Indium Corporation, usa) to generate a dc field. The electric field was generated using a DAQ card (national instruments) that sent square waves at 25000Hz at 5V to the sorting electrodes and then amplified 1000 times by a high voltage amplifier (623B Trek). The entire sorting process takes approximately less than 1 millisecond.

Image processing

Fig. 8 shows the physical dimensions of the fluid channel reader of the microfluidic device 704. The total height of the device is about 52 μm.

The device 704 was observed with the sensor 702 under an inverted microscope (Nikon Ti-E, japan) equipped with a high speed camera (CP90-4-M-500, Optronis) by a custom made computer (Intel i 76850K CPU,128GB RAM). The droplets were observed with a 10-fold objective magnification (Plan flow Nikon). At 2304^*The resolution of 452 pixels captures images at 1000 frames per second. The captured image is transmitted from the camera to a frame connected to a computer via a coaxpress connection (CX-304-1-SOS OS, Optronis)A catcher (VQ8-CXP6D, Silicon Software ME5 Ironman). After the frame grabber captures an image from the camera, the image is transferred line by line to computer RAM via a Direct Memory Access (DMA) controller. The process of transferring each image takes about 1ms, ensuring that there is sufficient time for image processing. Once the controller 708 completes image processing and droplet sorting, the controller computer generates a signal to the DAQ to control droplet sorting activity.

The system 700 includes key artificial intelligence functions that enable the classifier 710 to automatically process droplet images in real time. Software was developed in C + + using the computer vision library OpenCV. The software comprises a framework for processing captured video sequences using computer vision, machine learning, and multi-threaded paths. This framework includes three key components, namely: droplet identification, droplet sorting and synchronization.

Turning to fig. 9, for each captured video frame, the system 700 first processes the binarized image 900 by extracting a region of interest (ROI)902, which defines a rectangular window, containing only droplets within the channel boundaries and removing the noise background. To determine the width of the ROI window 902, an inflection point is located within the channel. To evaluate the top and bottom boundary positions of the ROI902, the image is projected in the horizontal direction. The local maxima of the projected histogram are then extracted as the locations of the channel boundaries. The parameters 904 of the droplets 104 are also shown.

Turning to fig. 10, droplet recognition 1000 is performed within the ROI902, which includes two tasks: namely drop tracking 1002 and drop classification 1004.

Because the droplet 104 is circular, the system 700 utilizes a fuff (Hough) transform to detect the circular boundary of the droplet during droplet tracking 1002. Hough (Hough) transformation is a feature extraction in computer vision. The Hough transform finds some kind of shape class by a voting procedure, which is done already at the parametric distance of the shape. The principle is to fit the following equation to the map boundary map detected from the image:

(x-xo)²+(y-yo)²＝r²

where x and y are coordinate points on the target circle, (xo, yo) is the origin, and r is the circle radius.

The Hough transform finds all possible (xo, yo) by searching for local maxima in the accumulation space. Because the radius of the microdroplets 104 can be easily determined before, such as by controlling the size of the microdroplets 104 in the microfluidic system 700, the system 700 in fact directly specifies the radius to reduce the computational cost.

As each droplet 104 with a complete boundary enters the ROI902, a unique tracking ID (identifier) is assigned. Since the image is captured at a high frame rate, the velocity of the droplet 104 is controlled by a threshold such that the droplet 104 travels a smaller distance between adjacent frames than between two adjacent droplets 104. This allows the distance between the drops 104 in two adjacent frames to be calculated to track the movement and the streaming speed of the drops. This in turn is used to assess when the droplet 104 will move to the sorting location.

Following detection of the droplet 104, the next step is to determine whether the droplet contains a particle, contains a particle or particles, so that they can be properly sorted. To do so, the drop region 902 is cut out of the image 900. Cropped droplet area 902 is normalized to a square image block of the same size (50x50 pixels). Each tile is then converted into a feature vector. These vectors are input to form a classifier. For feature extraction, the circular boundary of each drop 104 is removed first, since the boundary is common to all drops and contains no distinctive information. And then all remaining connected black pixels in the tile are restored. These connected pixels correspond to the presence of objects such as particles, noise and unfinished moving boundaries. The system 700 measures the area, height and width of each object. The object is sorted according to its area. The measurements of the first 10 objects are concatenated to form a 30-dimensional feature vector. When there are less than 10 objects in the drop 104, the remaining entries of the feature vector are set to zero. That can compute feature vectors from the number of droplets 104, forming learning training data for a linear Support Vector Machine (SVM) classifier. The training model is then used to predict the kind of new droplet. It is noted that other types of classifiers may be trained to accomplish the prediction task.

Droplet classification 1004 is a challenging task, mainly due to the fact that: when a droplet 104 has multiple particles, the particles may touch the boundaries of the droplet 104 or touch each other. It is sometimes not easy for human vision to even distinguish them. The position of the particle may change during the movement of the droplet 104, and one droplet 104 may appear in multiple frames. Thus, based on the per-frame recognition results, the system 700 employs a classifier voting scheme. Each frame is predicted to have a particular droplet 104 classification, and then the final result is achieved by most voting across frames. Such voting increases the accuracy of droplet classification.

Classification

Once the droplet 104 with a single particle is determined, the system 700 uses electrophoretic forces to pull the droplet 104 into the upstream channel when the droplet reaches the sorting position of the sorter 706. The key to successful sorting is synchronization with the system. Image acquisition module 702 captures images at a rate of 1000 frames per second. Therefore, it takes 1 millisecond (ms) to generate one frame. Several frames of the same droplet 104 may be captured before the droplet 104 reaches the sorting location of the sorter 706. The number of actual frames is related to the velocity of the drop 104. Such a condition requires that each frame be processed within 1 ms. However, the processing time stamp will be delayed, lagging the time stamp of the capture frame. As time passes, delays accumulate, resulting in loss of frames and even failure of the sorting system 706.

To address this problem, this embodiment employs a framework to speed up frame processing using multi-threading techniques. The framework process includes four modules: capture, process, trigger, and control. The capture module is an image acquisition module. The processing module is a machine learning module. The triggering and control module is responsible for droplet sorting. These threads are run in parallel and independently using multi-core CPUs to increase processing speed.

The system 700 also includes a buffer time between droplet identification and sorting such that the droplet 104 is ready to continue moving with its type tag before sorting. Because the velocity of the droplet 104 has been calculated, the system can predict when it will reach the sort location 706. This allows the system 700 to make a queue of droplets 104. Each droplet 104 in this queue has three attributes: trace ID, type flag and delay time. The tracking ID identifies the different droplets. The class mark indicates how the droplets are sorted. The delay time controls when the voltage is triggered. Once sorting is complete, the droplets are removed from the queue.

Once the system 700 detects the circular shape of the micro-droplets 104, the detected circles are sorted in order according to the x-axis coordinates of their images. The earliest and latest generated circles are lined up from the rightmost side of the frame to the leftmost side of the frame, and the age of the micro-drop 104 is replaced with the x-axis coordinate of the micro-drop 104. After sorting, the microdroplets 104 are arranged by their age. Based on the sorted x-axis coordinates on hand, the system 700 can find the relationship of the micro-droplets 104 between the current frame and the previous frame. Here we assume that the velocity of the droplet 104 is nearly constant, as in motion, a new microdroplet 104 cannot overtake an old microdroplet 104. The system 700 can therefore use this assumption to determine the tracking ID of the microdroplet.

In the control thread, the system 700 uses the DAQ to start the control program. DAQ entry duration. Since it takes time to activate the electrodes, the actual execution time is about 1ms, longer than the duration. The maximum time allowed for triggering depends on the distance and velocity between two adjacent drops 104. Ending when the next drop 104 reaches the sort location 706. Otherwise both drops 104 are affected by the same triggering action. This means that there is an upper limit for the trigger length. At the same time, the lower limit T of the trigger length_maxShould be long enough to pull the current drop 104 into the downstream channel at a given velocity. The following relationship may be determined by the following formula:

D_min(p_i.p_i+1)＝T_min*V_i+1wherein Pi and P_i+1Is two adjacent droplets, V_i+1Is P_i+1Velocity, T_minIs the minimum pull time, determined empirically. The sorting speed of the system is limited by T_minAnd the sum of the time required for image acquisition.

Fig. 11 shows an implementation of the machine learning method in system 700, which includes two phases: offline training 1100 and online sorting 1102. The off-line phase 1100 is the training to develop a classification model using labeled training samples. The training sample is a droplet 104 detected from a video sequence of the microfluidic device 704, which contains a different number of microparticles. In general, it is recommended to use more labeled training samples to achieve better classification.

Of these labeled samples, 60% were used for initial training and the remaining 40% were used to validate the trained classification model. The parameters of classifier 710 are adjusted until the verification accuracy is greater than 99% and system 700 can be reliably used in online phase 1102. Many machine learning methods are used to develop classifiers such as neural networks, decision trees, Bayesian classifiers, add or combine support vector machines.

During the online sorting phase 1102, the trained models are used to classify the droplets 104 in real-time. To accept a droplet 104 containing a single particle, an electric field is turned on, pulling the droplet into the desired sorting channel. For droplets that do not contain or contain multiple particles, the electric field remains off, allowing rejected droplets 104 to enter the waste channel.

Results

Because the system 700 requires time to process a single droplet 104 and activate the sorting electric field, the resulting droplets 104 must be separated. In order to determine the optimal separation distance, parameters such as droplet size, sorting frequency and distance between individual droplets are taken into account. This process was analyzed in two stages: namely droplet generation and droplet sorting.

Fig. 12 shows a flow diagram 1200 for determining the extent of droplets produced. The lowest blue line 1202 gives the pressure value (blue inset 1204) for the stable interface before switching to droplet generation (green inset 1206). The red line 1208 depicts the pressure value at the onset of advection (red-edged inset 1210). The scale bar of fig. 12 represents 100 microns. The black box dots are pressure combinations used for subsequent studies to isolate oil effects.

The pressure values versus pressure ratio pwater/Poil are redrawn in inset 1212 and also give a representative image of the droplet size generated. The droplet area remains constant at the same pressure ratio. Scale bar represents 50 um.

When the spacer oil is introduced, the pressure controller limit is at 900mBar and pressures below 400mBar are not used because no droplets 104 are generated. The values of the stabilizing interface and advection are concentrated against the corresponding water pressure. Plotting these values results in a drop generation region 1214 as shown by the green region in fig. 12. Maximum and minimum drop sizes are also measured and are shown in inset 1212 of fig. 12. This ensures that no droplets break apart at the barrier oil interface, as the joint will not form lumps. Once the drop generation area 1214 is determined, 10 different points across this area 1214 (black points in fig. 12) are read to determine the drop frequency and distance.

The droplet sorting frequency is determined by the system reaction time and the rate at which droplets 104 enter the sorting region 706. The time required for system processing and electric field activation is approximately 1ms (measured with a computer timer) and 1 ± 0.5ms, respectively. An additional 3ms is added to keep the electric field on, pulling the single particle droplet 104 into the right channel. System processing can run in parallel with sorting using multi-threading in system 700. The maximum value of the sorting frequency is therefore theoretically completely dependent on the sum of the time of activation of the electric field and pulling, which is approximately 286 Hz. The rate of droplets entering the sorting field area during sorting is affected by the barrier oil spray prior to the sorting stage. Oil pressure was isolated using a variation of 100 to 1000mBar, applied to 10 selected points. At some point, the barrier oil pressure is stopped before it reaches 1000mBar because the water pressure cannot overcome the Laplace pressure threshold at the liquid-oil interface to produce droplets.

Fig. 13 shows a graph 1300 of barrier oil pressure versus drop frequency observed at a drop sorting juncture of the sorter 706. The unstable droplet motion region is shown as pink, where the droplets 104 are shown entering the spacer oil or sort channels. The dots within the blue region 1304 are stable droplet motion. The points in this region 1304 correspond to drop spacing forming the curve shown in inset 1306, which shows that each pressure combination can be used for subsequent drop sorting trials when greater than 1mm minimum spacing.

From the observation, the detected unstable region 1302 is shown as pink in fig. 13. In this region 1302, the droplets either enter the spacer oil channel or the sorting channel due to low spacer oil pressure or the droplets are too close together. This result produces false positives and inconsistent droplet spacing distances, making droplet sorting features unusable. In the blue region 1304, the droplets produced are a single dispersed phase with regular droplet spacing. These droplets are therefore suitable for droplet sorting by the droplet sorter 706. Because the system limit is set at 286Hz, the bright blue area 1308 shows the setting of the pressure controller 712. Inset 1306 of fig. 13 highlights the drop spacing for different data points within bright blue region 1308. The droplet spacing is also critical because the droplets 104 need to be a minimum spacing distance apart. This prevents the sorting machine 706 from affecting subsequent drops 104 while driving, resulting in false positive results. This is included in inset 1306 of figure 13. Therefore, these limitations, the bright blue region 1308 in inset 1306 highlights the list of useful points that are suitable for the following sort test.

In the first experiment, the accuracy of classification, voting, and sorting was calculated as the drop frequency varied with classification. The results of 3 frames are used for voting. The drop frequency refers to the number of drops produced per second. In this experiment, 1000FPS runs 10000 frames at a time, the middle 1000 frames are used for testing because they are the most stable frames. The mean and standard deviation of each accuracy was then calculated. The results are shown in FIG. 14.

The system 700 can process the droplets 104 at a maximum frequency of 270 Hz. In some cases, the classification accuracy is not high. This is mainly due to the fact that the particles in the droplet 104 touch the boundaries of the droplet 104, making it difficult to distinguish between zero/single or single/multiple particles. This problem is solved well by the voting strategy, which improves the classification result. The final sorting accuracy is related to the voting result, but performance is reduced. Overall, the voting accuracy is higher than the classification accuracy, and the sorting accuracy lies between the classification and voting results. At a frequency of 260Hz, the system achieves the best results with 98.85% accuracy.

A second experiment was performed to analyze the role of the voting strategy. Given a fixed drop frequency of 220Hz, the system 700 votes from 3, 5 and 7 frames, respectively, for detection. The results are shown in FIG. 15. Regardless of which voting option is employed, the voting accuracy is higher than the classification accuracy. This again proves the validity of the classification vote. In this experiment, voting using 7 frames produced the best sorting results with 94.53% accuracy. In view of this result, the inventive system has proven to be very powerful, making single particle sorting applications efficient.

No droplet breakup was observed throughout the experiment during electric field driving, indicating that the ability of the applied electric field does not affect the droplet 104. Some improvements that improve the optimal response speed of high frequency or high throughput droplet sorting can be easily achieved by increasing the computer processing speed or integrating Field Programmable Gate Arrays (FPGAs) in the system 700, or using neural networks that classify multiple droplet libraries. The dc voltage may also be replaced by an ac voltage. FPGA integrated circuits may also be used to provide power or drive.

Conclusion

The system 700 provides real-time droplet sorting based on machine learning. The system 700 further provides a label-free, image-based approach that utilizes computer vision and machine learning for real-time analysis and sorting. The droplets 104 are automatically sorted without the need to label or use additives that interfere with the system or downstream testing. The drop generation guidelines were determined by experiments showing the effect of various flow pressures on drop sorting frequency. Second, prior to online testing, the classifier model has been trained and validated offline. Droplets were sorted from a continuous stream of 1000 frames at 260Hz to a high accuracy of 98.85%. The ability to efficiently sort droplets in this way is of great value for future biomedical applications, particularly single cell analysis.

Those skilled in the art will appreciate that many embodiments and variations may be made without departing from the scope of the invention.

The preferred embodiments involving microfluidic droplets 104 are also applicable to microbubbles.

The sorter 706 in the preferred embodiment is electrically driven. In other embodiments, the sorter may operate according to one or more of the following: hydrodynamics, pneumatics, electromotion, dielectrophoresis, magnetism and thermal principle.

In compliance with the statute, the invention has been described in language more or less specific to structural or methodical features. It is to be understood that the invention is not limited to the specific features shown or described, since the means herein described comprise preferred forms of putting the invention into effect.

Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more combinations.

Claims

1. A system for controlling microdroplets or bubbles comprising:

a microfluidic generator for generating said micro-droplets or bubbles;

a feedback sensor for sensing one or more feedback performance parameters of the generated micro-droplets or bubbles; and

a controller for controlling the microfluidic generator using the sensed feedback performance parameter.

2. The system of claim 1, wherein the feedback sensor comprises a predictor of predicted performance parameters that can predict parameters that are not sensed or are not reliable, and further utilizes the predicted performance parameters to control the microfluidic generator.

3. The system of claim 1, wherein the controller builds a model using training data from the feedback sensor and tests the generated droplets or bubbles to determine the accuracy of the model.

4. The system of claim 1, wherein the controller comprises an artificial intelligence controller integrated with, for example, a machine-learned classifier.

5. The system according to claim 1, wherein the controller comprises a pressure driven flow controller or a syringe pump.

6. The system of claim 1, wherein the controller is configured to control a microfluidic generator such that the microfluidic generator generates a monodisperse phase emulsion of the microdroplets or bubbles.

7. The system of claim 1, wherein the sensor comprises an optical system.

8. The system according to claim 7, wherein the optical system requires information of gas, liquid or solid phase in the fluid flow of said micro-droplets or bubbles, such as size, shape, color.

9. The system of claim 7, wherein the parameters determined by the optical system include any one or more of: droplet area, diameter, shape, frequency, separation distance, and velocity.

10. A method of controlling microdroplets or bubbles comprising:

generating the micro-droplets or bubbles;

sensing one or more feedback performance parameters of the generated micro-droplets or bubbles; and are

The microfluidic generator is controlled using the sensed feedback performance parameter.

11. The method of claim 10, wherein the controlling step includes a training phase including a model constructed using training data from the feedback sensor.

12. The method of claim 11, wherein the training phase includes determining a training range, the training range being related to the pressure characteristic of the generator performing the generating step.

13. The method of claim 12, wherein the training region is between the advection and the transition line of the stability region.

14. The method of claim 11, wherein the training phase comprises retrieving trained boundaries in a serpentine manner.

15. The method according to claim 11, wherein the model is a regression model.

16. The method of claim 15, comprising determining the optimal regression parameters for the regression model by minimizing loss of training data.

17. The method of claim 11, wherein the controlling step further comprises a testing phase comprising testing the generated droplets or bubbles to determine the accuracy of the model.

18. The method of claim 17, wherein said testing comprises inputting an expected droplet area and shape into said model.

19. The method of claim 17, wherein the accuracy includes an error of less than 2.5% when the input expected droplet area and shape is compared to the generated droplets.

20. A system for sorting micro-droplets or bubbles, the system comprising:

a sensor for sensing one or more parameters of said microdroplets or bubbles or their content;

a sorter for sorting said micro-droplets or bubbles, and

a controller for controlling the sorter using the sensed parameter.

21. The system of claim 20, wherein a sorter sorts the micro-droplets or bubbles depending on whether they contain micro-particles and the need for no droplet marking.

22. The system of claim 21, wherein the microparticles comprise biological cells such as blood cells, cancer cells, sperm, or eggs.

23. The system of claim 20, wherein the sorting is dependent on whether the sensed droplet parameters do not contain particles, contain a single particle, multiple particles, or particles of different properties and types.

24. The system of claim 20, further comprising a microfluidic generator for generating microdroplets or bubbles.

25. The system of claim 20, further comprising a spacer or separator for separating the generated droplets or bubbles.

26. The system of claim 24, wherein the generator, spacer or sorter are integrated together as a microfluidic device.

27. The system of claim 25, wherein the microfluidic device is shaped using lithographic techniques.

28. The system of claim 20, wherein the sensor comprises an imaging device.

29. The system of claim 27, wherein the imaging system comprises a microscope, a camera to capture image frames magnified by the microscope, and/or a frame grabber to grab image frames from the camera.

30. The system of claim 20, wherein the controller comprises a machine learning model.

31. The system of claim 20, wherein the controller comprises a pressure controller or a syringe pump.

32. The system of claim 20, wherein operation of the sorter is dependent on one or more of: hydrodynamic, pneumatic, electrodynamic, dielectrophoretic, magnetic and thermal principles.

33. A method for sorting micro-droplets or bubbles, the method comprising:

sensing one or more parameters of the microdroplets, bubbles, or contents thereof; sorting the micro-droplets or bubbles; and

controlling the sorter using the sensed parameter.

34. The method of claim 32, wherein said sensing comprises utilizing computer vision and said controlling comprises utilizing machine learning and/or real-time sorting.

35. The method of claim 32, wherein the sorting comprises snapping image frames of the drop.

36. A method according to claim 34, comprising identifying a target region in each frame containing a droplet.

37. The method of claim 32, comprising droplet tracking or droplet classification.

38. The method of claim 36, wherein the drop tracking comprises detecting a circular boundary of a drop using Hough transform or other boundary detection methods.

39. The method of claim 36, wherein the droplet classification comprises voting.

40. The method of claim 32, wherein sorting comprises accepting droplets or bubbles containing a single particle and rejecting other droplets or bubbles in the fluid.

41. The method of claim 32, wherein said sorting can be achieved by electrical actuation, preferably synchronized droplet generation, identification modules and determination of droplet classification based on the content contained.

42. The method of claim 32, wherein the controlling comprises machine learning, the machine learning comprising off-line training and on-line sorting using off-line training results.

43. The method of claim 32, further comprising generating the micro-droplets or bubbles, further comprising spacing or separating the generated droplets or bubbles in flow.