CN113639823B - High-precision liquid level detection system and method based on ripple image recognition - Google Patents

Abstract

The invention discloses a high-precision liquid level detection system and method based on ripple image recognition, wherein a piezoelectric transducer (PZT) is used for enabling an end effector in a descending process to generate high-frequency vibration, ripples are generated on a liquid level in a liquid level descending and liquid level contacting process, the generation of contact instant ripples is recognized through an image recognition technology, so that the judgment of a contact moment point of a needle point of the end effector and the liquid level is realized, and the contact positioning of the needle point and the liquid level is realized by generating recognizable ripples and image recognition of the ripples.

Description

High-precision liquid level detection system and method based on ripple image recognition

Technical Field

The invention belongs to the technical field of biochemistry, relates to a liquid level detection instrument and method for a liquid sample, and particularly relates to a high-precision liquid level detection system and method based on ripple image recognition.

Background

The traditional biochemical analysis instrument can rapidly provide detection basis for doctors and chemical detection personnel by detecting liquid environments in human bodies such as blood, urine, cell metabolic culture solution and the like, and is important in clinical diagnosis analysis and biochemical detection. The method has the advantages that specific liquid to be detected needs to be extracted and liquid is transferred in a biochemical monitoring and analyzing system, a complete liquid transferring process comprises operations of extracting a liquid transferring device, detecting liquid level, sucking samples, transferring the liquid transferring device, distributing the samples and the like, the full-automatic operation of the whole liquid transferring process can be realized, the tedious work of medical detection personnel and biochemical scientific research personnel can be reduced, and the biochemical quantitative detection and analysis, the data precision and the reliability can be guaranteed. Furthermore, recent advances in micro-nano-scale single cell research require unprecedented capabilities of automated liquid handling systems to handle small volumes of liquid media and even small droplets (nL to pL volumes) for single cell-based sequencing, metabolomics, and proteomics analysis.

The reliable and stable liquid level detection technology is the key for realizing full automation.

To operate automatically, the liquid handling system needs to detect the liquid level and then transfer the end effector (e.g., pipette, microneedle, etc.) to the desired liquid destination. The liquid level detection technology can ensure that when the needle point at the bottom of the end effector descends rapidly, whether enough samples exist in the sample container can be detected or not is judged in time by detecting the position of the liquid level in real time. The liquid level detection is used for acquiring the position information of the liquid level and the needle point so as to ensure that the accurate liquid amount is acquired under the condition that the needle point is not inserted too deeply into the liquid level, thereby reducing the carrying pollution caused by the liquid hanging phenomenon of the liquid transfer needle point to the maximum extent, reducing the carrying pollution rate of an instrument and being beneficial to improving the analysis precision. In addition, the diameter of the tip of the glass micro needle used as an end effector in micro-nano operation is very small in the micron level and easy to break, and accurate detection of the liquid level can realize successful precise micro operation in the micro-nano liquid environment. However, the detection precision and the use condition of the current liquid level detection technology are limited, and the current liquid level detection technology is particularly difficult to be used in the field of microorganism micromanipulation. The liquid level detection techniques commonly used at present are mainly classified into contact type and non-contact type. The traditional contact measurement method comprises a thermal method, a mechanical vibration method, a resistance method, a capacitance method and an air pressure method; non-contact methods include ultrasonic, laser, electromagnetic. However, these techniques have certain application limitations. The non-contact detection method has a longer detection range and higher accuracy, and is widely used in many cases. However, since the sensor probe is not a probe that directly manipulates the liquid, another problem is introduced in practical use that requires additional determination of the position of the end effector. The thermal method has a maximum accuracy of about 100 μm because of the limited response time of the cooling of the sensor. Mechanical vibration methods are capable of detecting the level of liquid under the foam or bottle cap, but their accuracy is limited to 100 μm since the end effector must be immersed 100-. The resistance method can detect the change in resistance with a precision of 2 μm when in contact with a liquid, but this method is not suitable for the detection of a non-conductive liquid. The capacitance method does not require liquid conductivity, but requires an additional metal plate to be added below the liquid container to be measured. The air pressure method is suitable for most liquids under general conditions, but a high-sensitivity pressure sensor is needed to detect pressure change in a capillary, and the processing of a sensor probe is complex and high in cost, so that interchangeability measurement in micro-nano operation is difficult to realize.

In summary, the conventional liquid level detecting apparatus and method mainly have the following problems:

1) the type of liquid that can be detected is limited, or requires additional experimental setup: at present, most detection technologies have requirements on the properties of liquid to be detected, for example, a resistance method requires liquid conduction; or the detection method is greatly influenced by the liquid property, for example, the electromagnetic method causes that the measurement accuracy changes along with the liquid property due to the change of electromagnetic attenuation along with the dielectric constant of the liquid, and the universal measurement of common liquid in a laboratory or industry is difficult to realize. In addition, some technologies such as capacitance method require additional metal plate at the bottom of the container to realize the measurement, which increases the requirement of equipment environment configuration.

2) The liquid level detection precision is low: most of the existing technologies have the liquid level measurement accuracy in the mum level, and the nm level positioning accuracy is difficult to provide so as to meet the requirements of micro-nano operation.

Object of the Invention

The invention aims to solve the problems in the prior art, realize the positioning and detection of the liquid level in the microscopic operation based on the image recognition technology, and provide a high-precision full-automatic microscopic operation system and a method for realizing the contact type liquid level positioning detection of the needle point of the end effector and the liquid level by combining the microscopic operation technology and the image recognition technology. The invention is compatible with a microscopic-operation needle point liquid level contact detection system, does not need to be provided with an additional sensor, can identify liquid level contact only by image identification ripples, realizes the liquid level detection precision of less than 1 mu m, and is suitable for the liquid level positioning measurement of liquids with different types of attributes.

Disclosure of Invention

According to one aspect of the invention, a liquid level detection system based on ripple image recognition is provided, and the system is placed on a shockproof table and consists of a microscope (301), a micro-operation moving table (2), a holder (203), a micro-needle (204), a piezoelectric transduction device and driver (201), an image acquisition device (302), a computer upper computer (401) and liquid to be detected; the piezoelectric transduction device comprises a signal generator (101), a power amplifier (102) and a piezoelectric transducer PZT (103), wherein the bandwidth of the signal generator (101) and the amplification factor of the power amplifier (102) are determined by the working voltage frequency and amplitude range of the piezoelectric transducer PZT (103); the holder (203) is connected with the micro-operation moving table (2) and the micro-needle (204), and the outer diameter width and the inner diameter width of the holder (203) are respectively matched with the holding structure of the micro-operation moving table (2) and the outer diameter specification of the micro-needle (204); generating a sinusoidal voltage signal capable of driving a piezoelectric transducer PZT (103) to periodically vibrate by cascading a power amplifier (102) through the signal generator (101), and generating stable periodic vibration of the micro-needle (204) under the fixation of the clamp (203); the micro-operation moving table (2) controls the holder (203) and the micro-needle (204) to move in a three-dimensional space; the microscope (301) observes the three-dimensional space movement, and the image acquisition device (302) acquires image information observed by the microscope (301); the computer upper computer (401) displays the image information acquired by the image acquisition device (302) in real time, judges the contact time of the micro needle and the liquid level to be detected through image processing and controls the three-dimensional space movement and vibration of the needle point of the micro needle.

Preferably, the holder (203) and the micro-needle (204) move in a three-dimensional space, namely, the holder (203) and the micro-needle (204) are controlled by the micro-operation moving platform (2) to descend along a direction vertical to the liquid level to contact with the liquid level, the whole process is collected by the image collecting device (302) through the microscope (301) and is observed on the computer upper computer (401) in real time, whether the liquid level generates ripples or not is detected through an image recognition mode, whether the needle point contacts with the liquid level or not is judged, and when the liquid level contacts with the liquid level, descending of the micro-operation moving platform (2) is stopped through feedback.

Preferably, the piezoelectric transducer PZT (103) is P-007 of PI company, the signal generator (101) is AFG3052C of Tektronix company, and the power amplifier (102) is TEGAM2350 of Tegam company.

Preferably, the material of the micro-needle (204) is boron glass, common glass, metal or plastic.

Preferably, the microneedles (204) are replaced with capillary or pipette tips.

Preferably, the image acquisition device (302) is a CCD or CMOS camera, and is matched and connected with the microscope (301) through a data interface, and the camera frame rate is set to be more than 50 frames/second.

Preferably, the detection precision of the liquid level detection system reaches 40 nm.

According to another aspect of the present invention, there is provided a method for detecting liquid level by using the above liquid level detection system, after placing the liquid to be detected on the detection platform of the detection system, the method comprises the following steps:

step 1, the initialization configuration of the liquid level detection system comprises the following steps: processing, selecting, clamping and mounting micro-needles (204) with specific shapes and sizes, configuring the amplitude and frequency of a piezoelectric transducer PZT (103) driving sinusoidal signal and the descending speed of a micro-operation moving table, and setting the exposure time and the frame rate of an image acquisition device (302);

and 2, identifying the needle point of the microneedle (204), specifically, controlling the needle point of the microneedle (204) to slightly move along the normal direction of the microneedle (204) in the image, comparing the gray level changes of the adjacent frame images to obtain the position of the needle point of the microneedle (204) in the image, and selecting a part near the needle point as an ROI (region of interest).

Step 3, descending the needle point of the micro needle (204), after the liquid level detection system is initialized and configured and obtains the needle point position of the micro needle (204), opening the signal generator (101), enabling the piezoelectric transducer PZT (103) to vibrate periodically and driving the needle point of the micro needle (204) to vibrate in three dimensions, and descending along the vertical direction under the action of the micro-operation moving table (2);

step 4, liquid level ripple identification: in the process that the needle point of the microneedle (204) continuously descends, the image acquisition device (302) is used for acquiring and analyzing a liquid level image observed by the microscope (301) in real time, according to the actual scene requirement of a user, a wavelength extraction method is used for identifying specific concentric circular ripples or calculating image contrast parameters, and the generation of the liquid level ripples is judged and identified through a threshold value, wherein the moment when the liquid level ripples appear is the moment when the needle point of the microneedle (204) contacts the liquid level to be detected;

step 5, stopping the needle tip of the microneedle (204): and after the liquid level ripple identification is completed and the contact point is judged, the liquid level detection system records and returns the coordinates of the contact liquid level of the needle point of the micro-needle (204), and stops vibrating to finish the operation.

Drawings

FIG. 1 is a schematic diagram of a fluid level detection system based on ripple identification according to the present invention.

Fig. 2 shows 4 different shapes of ripples observed by microscope when the microneedle contacts the liquid surface in the embodiment of the present invention.

Figure 3 is a graph of the ripple pattern of four different microneedles in distilled water excited by a signal with PZT frequencies of 5-40kHz and amplitudes of 10 Vpp.

FIG. 4 shows the shape and size of a 40-fold microscope objective lens for a selected tip in an embodiment of the invention.

FIG. 5 is a schematic illustration of the generation and attenuation of ripples in liquids of different viscosity coefficients.

FIG. 6 is a graph of actual concentric wavelength versus theoretical concentric wavelength.

Fig. 7 is a graph comparing the results of concentric circle recognition processing based on wavelength extraction before and after the microneedle tip contacts the liquid surface.

Fig. 8 is a schematic diagram of a generation manner of a gray level co-occurrence matrix in the moire identification method based on image texture parameter extraction.

Fig. 9 is a graph of the variation of different characteristic parameters before and after the microneedle tip contacts the liquid level.

Fig. 10 is a schematic flow chart of liquid level detection using the liquid level detection system of the present invention.

Fig. 11 is a schematic diagram of a cross-validation experiment of the accuracy and stability of a detection system with MP285 as a driver.

Fig. 12 is a timing chart of the single step movement in the position control mode.

Fig. 13 is a schematic diagram of the correspondence of the electrical signals and the corresponding key image frames during the position and speed control mode contacting the liquid level.

Figure 14 is the results before and after droplet contact at three different wetting angles on PDMS.

Fig. 15 is a schematic view of a flow pattern that may be caused by vibration of a microneedle tip in the z-axis.

Fig. 16 is a COMSOL simulation result of different forms of waviness caused by vibration of microneedle tips along different axes.

Detailed Description

In order to more clearly illustrate the present invention, the present invention is described in detail below with reference to the accompanying drawings and examples.

The invention provides a high-precision liquid level detection system based on ripple image recognition, which consists of a microscope, a micro-operation moving platform, a holder, a microneedle, a piezoelectric transduction device, a driver, an image acquisition device and liquid to be detected. The system composition schematic diagram is shown in fig. 1, a sinusoidal voltage signal capable of driving PZT to vibrate periodically is generated by a signal generator and a cascade power amplifier, the microneedle vibrates stably and periodically under the fixation of a holder, meanwhile, a micro-operation moving platform controls the whole holder to descend along the direction vertical to the liquid level to contact the liquid level, the whole process is collected by an image collecting device through a microscope system and observed on an upper computer in real time, whether the liquid level generates ripples or not is detected through an image recognition mode, whether the needle point contacts the liquid level is judged, and finally the micro-operation moving platform is fed back to stop descending.

In fig. 1, 101-. 101 is a signal generator, 102 is a power amplifier, 103 is a piezoelectric transducer PZT, wherein the bandwidth of the signal generator and the amplification factor of the power amplifier are designed and selected according to the working voltage frequency and amplitude range of the piezoelectric transducer. The sinusoidal signal is generated by a signal generator and amplified in voltage amplitude by a power amplifier, and finally the piezoelectric transducer is driven to generate mechanical vibration by a piezoelectric effect. In one embodiment, the PZT is P-007 from PI, the signal generator is AFG3052C from Tektronix, and the power amplifier is TEGAM2350 from Tegam.

201 and 202 constitute a micro-displacement actuator module, which satisfies the three-dimensional space movement of the end effector, the movement precision requirement is designed according to the actual requirement, the higher the movement precision of the micro-operation moving table is, the higher the positioning precision of the corresponding needle point contacting the liquid level is, in this embodiment, an instrument type selection is disclosed as a reference, for example, MP285 of SUTTER corporation in usa. And 203 is a clamper for connecting the micro-operation mobile station and the micro-operation needle. The outer diameter width and the inner diameter width of the micro-operation moving platform are respectively matched with the clamping structure of the micro-operation moving platform and the outer diameter specification of the micro-operation needle. 204 is a micro needle, the material can be boron glass or common glass, the shape and size of the needle point are selected and related to the ripple shape, and can be selected according to the actual requirement, and the drawing of the needle point is automatically drawn by a full-automatic needle drawing instrument device, such as a needle drawing instrument of model P-1000 of SUTTER company in America; the microneedles may also be made of other materials such as metal or plastic as desired to suit the needs of the particular application. 301 is a microscope, and the model can be selected according to the actually required resolution. 302 is image acquisition device, can be common CCD or CMOS camera, also can be other event cameras, and its model is selected the type according to the image resolution ratio of actual demand equally, matches with the microscope interface simultaneously, guarantees to integrate and installs in microscope system. Further, it is recommended that the camera frame rate setting should be more than 50 frames/sec, as the image recognition time error is smaller as the camera frame rate is higher. It should be noted that a microscope is not always necessary, since it is assumed that the actuator end is a micro-nano target, and the end needs to be magnified by the microscope for imaging observation.

And 401 is a computer upper computer and is responsible for displaying the acquired images in real time, processing the images, judging the contact time, controlling the three-dimensional movement and vibration of the needle point and the like.

Note that the whole liquid level detection system should be generally placed on a shockproof table, so as to avoid the influence on the measurement accuracy caused by the interference of external vibration on the mechanical structure.

The working realization principle of the liquid level detection system comprises the following steps:

1. generation of different form ripples

The stable ripple observed in the microscope is the basis for judging the contact time of the micro-needle point. As shown in fig. 2, the corrugation patterns observed upon contact are mainly classified into concentric circles, eccentric circles, spiral patterns, and irregular patterns. In addition, since different moire patterns directly affect the selection of the image recognition algorithm, it is necessary to discuss the forming conditions of different moire patterns.

The form of the ripples is related to the shape and size of the microneedles and the frequency of the PZT drive signal. The shape and size of the micro-needle determine the inherent vibration property of the micro-needle, and the frequency of the PZT driving signal influences the three-dimensional vibration mode of the micro-needle, so that ripples with different forms are generated. The fluid-solid coupling dynamic model is difficult to fully model and analyze, and real scenes such as device installation and the like can also influence the vibration form at the needle point, so that a specific ripple mode must be determined through experiments. Table 1 discloses four specific microneedle shape drawing procedures, where cone angles are used to describe the shape of the microneedles. Figure 3 discloses the ripple mode of the four microneedles in table 1 in distilled water under excitation of a signal with PZT frequencies of 5-40kHz and amplitudes of 10 Vpp. Referring to the waveform frequency diagram as a ripple spectrum, the ripple spectrum of the four tips may help the user select the appropriate microneedle to be associated with the desired ripple pattern. For example, for a needle tip 3 commonly used for nematode injection, a wider frequency range can be selected in the 26-39kHz range if concentric circles are the desired pattern in image recognition.

TABLE 1 procedure for making four microneedles with different taper angles

Fig. 4 discloses the shape and size of the needle tip 3 selected in the experiment under a 40-fold microscope objective, the outer diameter of the needle tip is about 1 μm, and the needle tips drawn in batches with the same process parameters have higher consistency.

The properties of the liquid to be measured, such as the viscosity coefficient, also influence whether a stable observed ripple can be generated. Taking the tip 3 as an example, setting the PZT frequency to 37kHz produces observable concentric ripples in eight liquids with increasing viscosity as shown in FIG. 5. It was found that the ripple decays more quickly as the increase in viscosity of the liquid exacerbates the dissipation of the vibrational energy propagation. When the viscosity is greater than 10cP, the concentric circles cannot be clearly observed even if the needle tip vibration frequency is set to other values. Namely, for the liquid with an excessive viscosity coefficient, the PZT model with larger amplitude needs to be selected to increase the drive energy to generate ripples.

Finally, the ripple wavelength generated by vibration under five liquid solutions (distilled water, acetonitrile, acetone, methanol and isopropanol) was measured and compared with the ripple theoretical wavelength in the appendix. As shown in FIG. 6, using the tip 3 as an example, PZT (20-60kHz, 10Vpp) excitation was used and concentric circle wavelengths were extracted, and the curve represents theoretical wavelength values calculated by the following equation:

regression analysis was performed on the theoretical and experimental wavelengths using the least squares method and the R2 values for the five solutions are summarized in table 2. All values are greater than 95%, which proves that the ripples generated by the vibration of the needle tip in different liquids belong to capillary waves, and the unit of the wavelength model is a micrometer scale, so that the concentric circular ripples at different vibration frequencies can be specifically identified by extracting the wavelength parameters of the ripples.

TABLE 2R of experimental and theoretical wavelength values²Regression analysis value

2. Image-based ripple identification method

(1) Concentric circle ripple identification based on wavelength extraction

Because the concentric circle ripples have high regularity, the interval of the image gray value peak value is the ripple wavelength, and other image noise interference can be accurately eliminated by extracting the ripple wavelength information to identify the concentric circle under the specific frequency, so that the requirement of high-precision contact detection of a user can be fully met. Therefore, a specific algorithm is proposed here to extract the wavelengths of the concentric circles from the microscopic image to identify the ripple generation time.

The algorithm is based on projecting the image from cartesian coordinates to polar coordinates with the origin set as the tip of the microneedle. As shown in fig. 7, the needle tip position in cartesian coordinates is determined by subtracting the background image without the needle tip from the image after the needle tip is moved. The projected image is then processed by a histogram equalization algorithm (fig. 7b) to enhance the contrast of the moire (fig. 7 c). The gray value profile curves at different angles are then reflected as the waveform of the liquid level, wherein the ripple wavelength can be represented by the distance between two adjacent peaks (fig. 7 d). In order to increase the image processing speed and to eliminate the interference of foreign substances and the like, the wavelength is determined as an average value of several typical angles (e.g., 45 °, 90 °, 270 °, 315 °, 360 °). The algorithm can obtain the wavelength in 16 milliseconds, which is 3 times faster than Hough transform.

(2) Moire identification based on image texture parameter extraction

When the ripple form is not concentric circles, the ripple cannot be accurately identified based on the algorithm of wavelength extraction, and in order to realize that different ripples generated under different vibration modes can be identified, another identification technology based on image texture parameters is provided. The detection mode is suitable for detecting scenes with reduced specificity requirements and can be identified under different ripple forms.

The different ripple patterns on the liquid surface are essentially texture information and can be characterized by partial eigenvalues of the gray level co-occurrence matrix. The visual texture refers to a repeated arrangement of a certain basic mode, the essence of the gray level co-occurrence matrix is to study the spatial correlation characteristics of the image gray level, namely the gray level relation existing between two pixels separated by a distance in the image space, and parameters such as energy, contrast, correlation, entropy, homogeneity and the like derived from the co-occurrence matrix can well reflect the texture characteristics of the image.

As shown in FIG. 8, the co-occurrence matrix is defined by the joint probability density of the pixels at two locations and is a second-order statistical feature related to the change in image brightness. Assuming that I (x, y) is a two-dimensional digital image with a size of M × N and a gray scale level of Ng, the gray co-occurrence matrix is calculated as follows:

wherein # { } represents the number of elements in the set, and the gray level co-occurrence matrix G is a matrix of Ng × Ng.

As an example, four features are selected, including two grayscale-based features (mean, variance) and two texture features extracted by a grayscale co-occurrence matrix (entropy, an indicator of information richness, and contrast, an indicator of texture). The calculation is as follows:

selecting a needle point part as a region of interest (ROI) to perform texture analysis on the collected water ripple pattern, lifting the needle point after the needle point is contacted with the liquid level downwards, and contacting downwards after the needle point leaves the liquid level, repeating the steps for a plurality of times, and extracting and calculating the characteristic parameters in the process of generating ripples through vibration contact, wherein the result is shown in fig. 9. The blue dotted line marks the contact time of the needle tip and the liquid level. By comparing the correlation coefficients (ρ xy) between the four feature vectors and the contact event vector, which is composed of a series of 0 (non-contact event) and 1 (contact event), it is demonstrated that the contrast feature (correlation coefficient 0.97) outperforms the other three features in distinguishing the contact events. Note that the above-described preliminary experimental determination needs to be performed in the context of the respective microscope parameter settings when the contrast threshold is actually selected as a basis for the determination. According to experiments, the recommended threshold for contrast characteristics when determining a contact event is 0.001-0.004.

The following describes a method for detecting liquid level by using the liquid level detection system according to the present invention with an embodiment, and a specific flowchart is shown in fig. 10, which includes the following steps:

1. initialization configuration: the initialization configuration of the system mainly comprises the processing and the clamping installation of the micro-needle with a specific shape and a specific size, the amplitude and the frequency configuration of a PZT driving sine signal, the descending speed configuration of a micro-operation moving table, and the setting of the exposure time and the frame rate of a camera.

2. Identifying a needle point: in order to obtain the position information of the needle tip and extract an effective ROI, the needle tip is controlled to slightly move along the normal direction of the needle in the image, the position of the needle tip in the image is obtained by comparing the gray level changes of adjacent frame images, and the position near the needle tip is selected as the ROI.

3. Descending the needle point: after the system is initialized and the position of the needle point is obtained, the signal generator can be turned on, so that the PZT periodically vibrates and drives the needle point to vibrate in a three-dimensional manner, and the PZT descends along the vertical direction under the action of the micro-operation mobile platform.

4. Ripple identification: in the process of continuously descending the needle tip, the liquid level patterns in the microscope are collected and analyzed in real time, specific concentric circular ripples are identified by using a wavelength extraction method or image contrast parameters are calculated according to the actual scene requirements of a user, and the generation of the ripples is judged and identified through a threshold, wherein the occurrence time of the ripples is the time when the needle tip contacts the liquid level.

5. Stopping the needle tip: after the ripple recognition is finished and the contact point is judged, the system records and returns the coordinates of the contact liquid level of the needle point, and stops the vibration ending operation.

System accuracy and stability verification

The liquid level positioning detection accuracy of the system was verified by replacing the micro pipetting needle with a 10 μm copper needle and using saturated saline (45.2mS/cm) as the conducting liquid, designing a cross-validation experiment while implementing an impedance-based and wavelength-extraction-based concentric circle detection method. First the precision (or error) is defined and consists of two parts. As shown in FIG. 11, assuming that the liquid level is zero, the tip position at the time of contact detection by concentric circles is represented by hc. Furthermore, due to control latency, the tip may move hl further down before commanding it to stop completely. The velocity of the robot is denoted by Vm, and the overall measurement accuracy (E) can be written as:

E＝hc+hl＝Vm(tc+tl) (6)，

where tc is the time from the last image without ripples to the successful detection of the touch event, and tl is the control delay time for the micro-op mobile station from the end of tc to a complete stop. the values of tc and tl depend on the hardware system employed. The evaluation results of an embodiment of the optional hardware device (MP285) are used as reference.

For the MP285 selected, there are two control modes. The first is position control, where the speed can be set to 1 μm/s to 3mm/s, with a maximum resolution of 0.04 μm, which is achieved via a serial port, with significant control delay time. The second is speed control in which the speed depends on the value of the external voltage, and the control delay time is short. In the position control, the tip was gradually lowered at a speed of 1 μm/s (0.04 μm per step). The timing sequence is shown in fig. 12. In the speed control, the tip of the needle was continuously lowered at a speed of 1 μm/s.

Fig. 13 shows the correspondence between the electrical signals and the corresponding key image frames during the position and speed control mode contacting the liquid level. In the transition from the non-contact state to the contact state, the last image frame without ripples occurs almost at the initial point of the rising edge of the electric signal, the first image frame with ripples is definitely present at the rising edge, and ripples are present in all subsequent image frames.

In position control mode, the electrical signal rises less than a step, indicating that the transition from non-contact to contact is completed within a single step. It can be confirmed that a ripple is generated somewhere in the middle of performing the step down, tl ≦ 25 ms. I.e. a one-step precision better than that of a micro-manipulation mobile station, i.e. 0.04 μm. The accuracy can be further improved if a higher resolution micro-manipulation mobile station is used. In the speed control mode, tl is 3.75 ms. Although this control delay time is shorter, its accuracy is not better than the resolution of the micro-manipulation mobile station (0.04 μm). The accuracy verification experiments were performed 50 times for both position control and velocity control (velocity 100 μm/s), with accuracies of 0.04 ± 0.00 μm and 2.66 ± 0.22 μm, respectively.

And secondly, the stability and repeatability of the detection method under the speed control are evaluated by changing the working conditions for experiments. These conditions include the illumination intensity, the microscope magnification, the exposure time of the camera and the focal plane of the microscopic imaging. The liquid surface contact detection test was repeated 50 times at a speed of 10 μm/s for each of the different working conditions, and 700 tests were performed in total. Tables 3-6 summarize the touch detection errors, expressed as mean (m) and standard deviation (σ).

Intensity of illumination	Height of	In	Is low in
				Mean value (mum)	0.28	0.28	0.28
Variance (mum)	0.22	0.24	0.13

TABLE 3 detection accuracy under different illumination intensities

Magnification factor	4 times of	10 times of
			Mean value (mum)	0.28	0.26
Variance (mum)	0.18	0.20

TABLE 4 detection accuracy under different microscope magnifications

Exposure time	4μs	10μs	15μs	20μs
					Mean value (mum)	0.27	0.28	0.28	0.38
Variance (mum)	0.19	0.16	0.17	0.29

TABLE 5 detection accuracy at different exposure times

TABLE 6 detection accuracy under different focusing planes

The result shows that the liquid level detection method based on image recognition has high repeatability and stability. In all 700 experimental trials, contact detection was achieved regardless of the change in the above micromanipulation conditions. For most experiments, the detection error at a velocity of 10 μm/s was less than 0.30 μm.

Liquid drop level positioning

In addition, surface contact positioning experiments were performed on nano-to pico-liter sized droplets on polydimethylsiloxane PDMS to demonstrate the applicability of the method. Distilled water droplets from 50nL to 450pL were generated and droplet detection was repeated 20 times at different PDMS positions for each droplet volume. And flexibly switching and applying two detection methods, namely using a concentric circle identification method based on wavelength extraction when the volume of the liquid drop is more than 18nL, and using a ripple identification method based on image texture parameter extraction when the volume of the liquid drop is less than 18 nL. The recognition results are shown in table 7, and both methods maintained recognition rates higher than 90%.

TABLE 7 success rate of liquid level detection of distilled water drops of different volumes

In addition, the drop levels at three different wetting angles on PDMS were also successfully detected. As shown in fig. 14, the distilled water wetting angle was 86 °, the acetonitrile wetting angle was 19 °, and the methanol wetting angle was 17 °. It is proposed to flexibly select an appropriate image recognition detection method for different droplet surface tension conditions to minimize possible adverse effects.

Model for waviness due to tip vibration:

after the microneedles contact the liquid surface, the microneedle tips can be modeled as thin, partially immersed cylindrical shells. Depending on the amplitude of the vibration, different flow patterns will be generated around the tip. Generally, as the amplitude increases, the liquid flow exhibits a pattern from linear to non-linear. Taking the vertical vibration along the z-axis in fig. 15 as an example, three flow structures (surface wave, submerged flow and vertical jet) may occur sequentially or simultaneously. For the case of horizontal axis vibration, the flow mode may be different from the z-axis vibration case, but the surface wave certainly exists in the linear phase. In practical application of micromanipulation, surface waves were chosen for study because it facilitates focusing of the microscope on the surface of the liquid for identification. In addition, PZT is required to ensure that the vibration amplitude is small to ensure that the generated surface wave is a linear wave.

Surface waves, i.e., water ripples, fig. 16 simulates three main ripple types corresponding to the tip vibration direction using COMSOL simulation. Three types of ripples, concentric circle, eccentric circle and spiral circle, are generated along only the z-axis, only the y (or x) axis, the x and y axes (vibration amplitude 1: 1) and the z and y (or x) axes (vibration amplitude 1: 1), respectively.

In addition, regardless of how the microneedles are placed obliquely. On the liquid surface, the vibrational mode will remain the same as for the z-axis only vibration in the figure, as long as the tip vibration is always along the axis of the microneedle. Because under axial vibration, the actual point of contact is a constant point on the surface of the liquid, it does not vibrate in a plane (i.e., x// y).

Description of a ripple theoretical model:

the ripple caused by the oscillation of the microneedles belongs to the capillary wave and can be described by typical parameters, such as frequency (f), wavelength (λ), and velocity (c). The surface tension of the liquid is expressed by sigma and the height of the liquid level in the vessel is expressed by h, and the ripple can be expressed by the formula:

where th () is the hyperbolic tangent function in the trigonometric operation.

Substituting c ═ λ f into equation (7) yields:

the frequency of the concentric circles is the same as the frequency of the microneedles or PZT. When the wavelength is shorter than 1.7cm, the surface tension of the liquid plays a dominant role, and the influence of gravity is negligible. Considering that the wavelength at the normally used kHz excitation frequency is on the μm scale, the capillary waves are therefore negligibly affected by gravity, and the first term of the summation disappears. Also, typically the height h >1mm of the liquid level in the vessel, the term in th () is approximately 0, so equation 8 can be simplified as:

this indicates that the wavelength of the concentric circles is inversely related to the vibration frequency of the micro-needle or PZT.

In conclusion, the invention realizes the positioning and detection of the liquid level in the microscopic operation based on the image recognition technology. The detection principle is based on the change in the topography of the liquid surface upon contact. The liquid surface remained calm before contact. After the tip has established contact with the liquid, the vibrational energy will exceed the surface tension limit, thereby forming a patterned wave around the point of contact. Therefore, the contact point between the needle tip and the liquid surface can be determined by extracting the characteristics of the stable ripple image generated on the liquid surface at the time of contacting the liquid surface by the image recognition algorithm and recognizing the presence or absence of the detected ripple.

The invention provides a high-precision full-automatic microscopic operation system and a method for realizing contact type liquid level positioning detection of a needle point and a liquid level of an end effector by combining a microscopic operation technology and an image recognition technology. The system designed by the invention utilizes the piezoelectric transducer (PZT) to enable the end effector in the descending process to generate high-frequency vibration, ripples are generated on the liquid surface in the descending contact liquid surface process, and the judgment of the contact time point of the needle point of the end effector and the liquid surface is realized by identifying the generation of the contact instant ripples through the image identification technology. The invention can play an important role in the field of full-automatic micro-nano operating instruments which need to accurately judge the liquid level, and is applied to occasions related to liquid sample processing, such as medicine distribution, food and beverage processing, biochemical analysis and the like.

Compared with the prior art, the invention has the following beneficial effects:

1) only common micromanipulation equipment is used, and complex photoelectric sensor detection is not required to be installed, so that convenience in subsequent micromanipulation is provided for a user;

2) the liquid level positioning device can be widely suitable for the liquid level positioning requirements of liquids with different attributes, and the problem that the type of measurable liquid is limited is solved;

3) different pertinence image recognition strategies in different ripple modes are provided, and the requirements of a user on high precision and universality liquid level positioning are met flexibly;

4) the liquid level positioning precision with typical precision reaching 40nm can be realized, and the liquid level positioning precision can be further improved under the condition of using a high-speed camera and a higher-precision micro-motion platform; the method can be stably identified under different working conditions, and the requirements of high-precision robustness contact type liquid level detection in the field of micro-nano control are met;

5) the invention can be used for measuring the plane liquid level and can also be used for positioning and detecting the liquid drop with the curved liquid level and the volume of nanoliter or even pico-upgrade.

Claims (7)

1. A liquid level detection system based on ripple image recognition is characterized in that the liquid level detection system is placed on a shockproof table and consists of a microscope (301), a micro-operation moving table (2), a holder (203), a micro-needle (204), a piezoelectric transduction device and driver (201), an image acquisition device (302), a computer upper computer (401) and liquid to be detected;

the piezoelectric transduction device comprises a signal generator (101), a power amplifier (102) and a piezoelectric transducer PZT (103), wherein the bandwidth of the signal generator (101) and the amplification factor of the power amplifier (102) are determined by the working voltage frequency and amplitude range of the piezoelectric transducer PZT (103);

the holder (203) is connected with the micro-operation moving table (2) and the micro-needle (204), and the outer diameter width and the inner diameter width of the holder (203) are respectively matched with the holding structure of the micro-operation moving table (2) and the outer diameter specification of the micro-needle (204); generating a sinusoidal voltage signal capable of driving a piezoelectric transducer PZT (103) to periodically vibrate by cascading a power amplifier (102) through the signal generator (101), and generating stable periodic vibration of the micro-needle (204) under the fixation of the clamp (203); the micro-operation moving table (2) controls the holder (203) and the micro-needle (204) to move in a three-dimensional space;

the microscope (301) observes the three-dimensional space movement, and the image acquisition device (302) acquires image information observed by the microscope (301);

the computer upper computer (401) displays the image information acquired by the image acquisition device (302) in real time, judges the contact time of the micro needle and the liquid level to be detected through image processing and controls the three-dimensional space movement and vibration of the needle point of the micro needle;

the gripper (203) and the micro-needle (204) move in a three-dimensional space, namely, the gripper (203) and the micro-needle (204) are controlled by the micro-operation moving platform (2) to descend along the direction vertical to the liquid level to contact the liquid level, the whole process is collected by the image collecting device (302) through the microscope (301) and is observed on the computer upper computer (401) in real time, whether the liquid level generates ripples or not is detected through an image recognition mode, whether the needle point contacts the liquid level is judged, and when the liquid level contacts the liquid level, the descending of the micro-operation moving platform (2) is stopped by feedback.

2. The liquid level detection system of claim 1, wherein the piezoelectric transducer PZT (103) is selected from P-007 of PI corporation, the signal generator (101) is selected from AFG3052C of Tektronix corporation, and the power amplifier (102) is selected from Tegam2350 of Tegam corporation.

3. The fluid level detection system of claim 1, wherein the microneedles (204) are made of borosilicate glass, plain glass, metal, or plastic.

4. The fluid level detection system of claim 1, wherein the microneedles (204) are replaced with capillaries or pipette tips.

5. The liquid level detection system according to claim 1, wherein the image acquisition device (302) is a CCD or CMOS camera, which is coupled to the microscope (301) via a data interface, and wherein the camera frame rate is set to be greater than 50 frames/second.

6. The fluid level detection system of claim 1, wherein the fluid level detection system has a detection accuracy of up to 40 nm.

7. A method of liquid level detection using the liquid level detection system of any one of claims 1-6, wherein the liquid to be detected is placed on a detection platform of the detection system and then the method comprises the following steps:

step 1, the initialization configuration of the liquid level detection system comprises the following steps: processing, selecting, clamping and mounting micro-needles (204) with specific shapes and sizes, configuring the amplitude and frequency of a piezoelectric transducer PZT (103) driving sinusoidal signal and the descending speed of a micro-operation moving table, and setting the exposure time and the frame rate of an image acquisition device (302);

step 2, identifying the needle point of the microneedle (204), specifically, controlling the needle point of the microneedle (204) to slightly move along the normal direction of the microneedle (204) in the image, comparing the gray level changes of adjacent frame images to obtain the position of the needle point of the microneedle (204) in the image, and selecting a part near the needle point as an ROI (region of interest);

step 3, descending the needle point of the micro needle (204), after the liquid level detection system is initialized and configured and obtains the needle point position of the micro needle (204), opening the signal generator (101), enabling the piezoelectric transducer PZT (103) to vibrate periodically and driving the needle point of the micro needle (204) to vibrate in three dimensions, and descending along the vertical direction under the action of the micro-operation moving table (2);

step 4, liquid level ripple identification: in the process that the needle point of the microneedle (204) continuously descends, the image acquisition device (302) is used for acquiring and analyzing a liquid level image observed by the microscope (301) in real time, according to the actual scene requirement of a user, a wavelength extraction method is used for identifying specific concentric circular ripples or calculating image contrast parameters, and the generation of the liquid level ripples is judged and identified through a threshold value, wherein the moment when the liquid level ripples appear is the moment when the needle point of the microneedle (204) contacts the liquid level to be detected;

step 5, stopping the needle tip of the microneedle (204): and after the liquid level ripple identification is completed and the contact point is judged, the liquid level detection system records and returns the coordinates of the contact liquid level of the needle point of the micro-needle (204), and stops vibrating to finish the operation.

Images