TWI733355B - Pipetting system and calibration method thereof - Google Patents

Abstract

A pipetting system and a calibration method thereof are provided, which belong to the technical field of pipetting equipment, and are used to solve the calibration problem of the pipetting equipment. The method for calibrating a pipetting system includes: acquiring a first image of a calibration point by an image capturing device; obtaining a first offset of a center point of the image capturing device from a calibration point in the first image, and the image capturing device is calibrated according to the first offset; using the calibrated image capturing device to capture a second image of a tip of the pipette of the pipetting system and a third image after the tip is displaced; obtaining a second offset of a third image relative to the second image of the tip; obtaining a focus distance offset of a sharpness value of the third image relative to a preset sharpness value according to the displacement, a variation of the sharpness value of the third image relative to the second image, and a preset mapping table between the focus distance and the sharpness value; and the pipette is calibrated according to the second offset and the focus distance offset. Thus, the calibration of the pipetting system is realized.

Description

Liquid pipetting system and its calibration method

The present invention relates to the technical field of pipetting equipment, and particularly relates to a pipetting system and a calibration method thereof.

Liquid extraction and injection are operations that are often required in industrial applications. In the automated process, the pipetting equipment can be used for liquid extraction and injection, and the pipette and turntable of the pipetting equipment can meet the operation requirements of micropipetting.

In practical applications, different liquids and different operating scenarios need to be replaced with different pipettes. In the process of disassembling and assembling the pipette, due to the assembly deviation, the movement deviation of the turntable, etc., the tip position of the pipette is shifted in three dimensions to varying degrees. This shift has the following effects on micropipetting operations:

When extracting liquid, the offset of the tip of the pipette will cause the pipette to inhale air bubbles. Excessive air bubbles will cause errors in subsequent operations;

When injecting liquid, the displacement of the tip of the pipette causes the liquid to be unable to be injected to the predetermined position smoothly, resulting in liquid loss;

When the turntable rotates, the position deviation of the pipette tip causes the pipette to collide with the test tube on the turntable and other parts, which affects the operation of the system.

It can be seen that due to the offset of the pipette tip, the pipetting operation and system operation are affected.

It should be noted that the information disclosed in the background art section above is only used to enhance the understanding of the background of the case, and therefore may include information that does not constitute the prior art known to those with ordinary knowledge in the field.

In view of this, this case discloses a pipetting system and a calibration method thereof to achieve the pipetting system calibration and overcome the problem that the pipetting operation and system operation are affected due to the position offset of the pipette tip in the prior art.

According to one aspect of the present invention, a method for calibrating a pipetting system is provided. The calibration method includes: capturing at least one first image of a calibration point through an image capturing device; A first offset of the calibration point in the first image on a plane, and the position of the image capturing device is adjusted according to the first offset, so that the center point and the calibration point are calibrated; The image capture device captures at least a second image and a third image of the tip of a pipette of the pipetting system, wherein the third image is captured from the focus of the tip and the center point after calibration After the distance is shifted; obtain a second offset of the tip in the third image relative to the tip in the second image on the plane; respectively according to the displacement and the sharpness of the third image relative to the second image The degree of variation of the degree value and a preset mapping relationship table between the focus distance and the sharpness value to obtain a focusing distance offset of the sharpness value of the third image with respect to a predetermined sharpness value; and, according to the second The offset and the focus distance offset adjust the position of the pipette so that the tip is calibrated with the center point after calibration.

In an embodiment of the present case, the step of obtaining a first offset of a center point of the image capturing device relative to the calibration point in the first image on a plane includes: obtaining the center points respectively And the Cartesian coordinates of the calibration point in the first image on the plane, wherein the plane is parallel to the plane where a turntable of the pipetting system is located; and the Cartesian coordinates of the center point and the The rectangular coordinate of the calibration point is converted into a polar angle to obtain the included angle of the polar angle of the center point with respect to the polar angle of the calibration point; and the first offset is obtained according to the included angle.

In an embodiment of the present case, the step of obtaining the rectangular coordinates of the calibration point in the first image on the plane includes: smoothing and filtering the first image to remove Noise; binarize the first image after smoothing and filtering, and assign a value to each pixel point in the first image after smoothing and filtering; perform edge measurement on the first image after binarization , Marking a plurality of edge points of the first image after binarization; performing contour calibration on the plurality of edge points of the first image after edge measurement to form at least one contour; for each of the contours Perform roundness detection to screen out contours whose roundness value is less than a preset value; perform area screening on the screened contours, and screen out a contour whose area meets the area of the calibration point; and obtain a screened out contour The rectangular coordinates of the centroid of the contour on the plane are used as the rectangular coordinates of the calibration point in the first image on the plane.

In an embodiment of the present case, the step of obtaining a second offset of the tip in the third image relative to the tip in the second image on a plane includes: obtaining the first The center of the tip in the three images is a first rectangular coordinate of the plane, and the center of the tip in the second image is a second rectangular coordinate of the plane, wherein the plane is perpendicular to The direction in which the focusing distance is located; according to the first rectangular coordinates and the second rectangular coordinates, the center of the tip in the third image is obtained relative to the center of the tip in the second image , Where the rectangular coordinate uses a graphic element as a standard unit; and, according to the corresponding relationship between a graphic element and a distance, the graphic element offset is converted into a distance offset, which is used as the The second offset.

In an embodiment of the present case, the step of obtaining a first right-angle coordinate of the center of the tip in the third image on the plane includes: smoothly filtering the third image to remove the The noise of the third image; binarize the third image after smoothing and filtering, and assign a value to each pixel point in the third image after smoothing and filtering; Image edge measurement, marking multiple edge points of the third image after binarization; performing contour calibration on the multiple edge points of the third image after edge measurement to form at least one contour; Perform roundness detection on each of the contours, and screen out contours whose roundness value is less than a preset value; perform area screening on the screened contours, and screen out a contour whose area matches the area of the tip; and, Obtain the rectangular coordinates of the centroid of the screened contour on the plane as the rectangular coordinates of the center of the tip in the third image on the plane.

In an embodiment of the present case, the step of according to the correspondence between a picture element and the distance includes: capturing an image of an object of a preset size through the image capturing device; and, according to the image of the object The number of graphic elements and the preset size are used to obtain the corresponding relationship between the graphic elements and the distance.

In an embodiment of the present case, the mapping relationship between the focus distance and the sharpness value is based on the displacement, the deviation of the sharpness value of the third image relative to the second image, and a preset focusing distance and sharpness value. , The step of obtaining a focus distance offset of the sharpness value of the third image relative to a preset sharpness value includes: according to the direction of the displacement and the third image relative to the second image Obtaining the position of the sharpness value of the third image in the mapping relationship table, wherein the sharpness value in the mapping relationship table has a Gaussian distribution; and, according to the variation amount of the sharpness value, the position of the sharpness value of the third image in the mapping relationship table is obtained; The position of the sharpness value of the image in the mapping relationship table, obtaining the focusing distance offset of the focusing distance corresponding to the sharpness value of the third image with respect to the focusing distance corresponding to a preset sharpness value, The preset sharpness value is the highest sharpness value in the mapping relationship table.

In an embodiment of the present case, the calibration method further includes setting the imaging depth of the image capture device so that the detectable range of the focus distance offset is within a preset distance range, wherein In the step of the third image, the displacement is greater than the preset distance range.

According to one aspect of the present invention, a pipetting system is provided. The pipetting system includes: an image capture device; and a processor configured to execute a plurality of executable instructions: The image capturing device captures at least one first image of a calibration point; obtaining a first offset of a center point of the image capturing device relative to the calibration point in the first image on a plane , Adjust the position of the image capturing device according to the first offset, so that the center point and the calibration point are calibrated; and the image capturing device is calibrated to capture the pipetting system At least a second image and a third image of the tip of a pipette, wherein the third image is captured after the focus distance between the tip and the calibrated center point is shifted; the third image is obtained A second offset of the tip in the image relative to the tip in the second image relative to the plane; respectively according to the displacement and the sharpness of the third image relative to the second image The degree value variation and a preset mapping relationship table between the focus distance and the sharpness value to obtain a focusing distance offset of the sharpness value of the third image relative to a preset sharpness value; and, according to The second offset and the focus distance offset adjust the position of the pipette so that the tip is calibrated with the center point after calibration.

In an embodiment of the present case, the pipetting system further includes: a turntable with the image capturing device.

Compared with the prior art, the beneficial effects of this case include at least:

The image of the calibration point is captured by the image capture device, and the first offset of the center point of the image capture device relative to the calibration point in the plane dimension is obtained, so as to realize the calibration of the image capture device in the plane dimension, so that the calibrated image The capture device is used for the positioning of the subsequent calibration steps; the image capture device after the calibration is used to capture the image of the tip of the pipette at least twice to obtain the second offset of the tip of the pipette in the plane dimension and the focus The focus distance offset in the distance dimension realizes the calibration of the tip of the pipette in the plane dimension and the focus distance dimension; thus, this case uses an image capture device to achieve the pipetting system calibration, so as to realize the subsequent pipetting operation and system Operational control.

In order to have a better understanding of the above and other aspects of the present invention, the following specific examples are given in conjunction with the accompanying drawings to describe in detail as follows:

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms, and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided so that the case will be comprehensive and complete, and the concept of the example embodiments will be fully conveyed to those with ordinary knowledge in the art. The same reference numerals in the figures indicate the same or similar structures, and thus their repeated description will be omitted.

Figure 1 shows the main structure of a pipetting system in an embodiment. In some embodiments, the method for calibrating the pipetting system in this case can be used to calibrate the pipetting system shown in Figure 1. Referring to FIG. 1, the pipetting system includes a turntable 10 and a pipetting device 30. The pipetting device 30 is, for example, a vertical syringe pump, and the pipetting operation is realized through the pipette 31. The turntable 10 is provided with a pipette replacement area 101 and a pipetting area 102.

Figure 2 shows the main structure of the pipetting device 30 in the pipetting system. The pipetting device 30 is connected to the sliding rail 300, and the pipetting device 30 drives the pipette 31 to move along the sliding rail 300 to realize the pipetting operation. Furthermore, the pipetting system of this embodiment further includes an image capturing device 21 installed on the turntable 10 and a fixed calibration point 11. Using the image capturing device 21 and the calibration point 11, the turntable 10 and the pipette 31 of the pipetting system are calibrated.

Figure 3 shows the steps of a method for calibrating a pipetting system in an embodiment. Referring to Figure 3, the calibration method of the pipetting system in this embodiment includes: step S10, capturing at least one first image of a calibration point through an image capturing device; step S20, obtaining the image capturing device A first offset of a center point relative to the calibration point in the first image on a plane, and the position of the image capturing device is adjusted according to the first offset so that the center point and the calibration point are aligned Calibration; step S30, capturing at least a second image and a third image of the tip of a pipette of the pipetting system by the image capturing device after calibration, wherein the third image is captured from the tip and After the calibrated focusing distance of the center point is shifted; step S40, obtaining a second offset of the tip in the third image relative to the tip in the second image on the plane; step S50, respectively According to the displacement, the variation of the sharpness value of the third image relative to the second image, and a preset mapping relationship table between the focus distance and the sharpness value, the sharpness value of the third image relative to a predetermined Set a focus distance offset of the sharpness value; and step S60, adjust the position of the pipette according to the second offset and the focus distance offset, so that the tip is calibrated with the center point after calibration .

In this embodiment, the image of the calibration point is captured by the image capturing device, and the first offset of the center point of the image capturing device with respect to the calibration point in the plane dimension is obtained, so as to realize the calibration of the image capturing device in the plane dimension, so that the calibration The latter image capture device is used for positioning in subsequent calibration steps; the image capture device after calibration is used to capture images of the tip of the pipette at least twice to obtain the second offset of the tip of the pipette in the plane dimension And the focus distance offset in the focus distance dimension, to achieve the calibration of the tip of the pipette in the plane dimension and the focus distance dimension; thus, this embodiment uses an image capture device to achieve the calibration of the pipetting system, thereby achieving subsequent Control of pipetting operation and system operation.

Among them, in this embodiment, the image capturing device is a microscope camera. In other embodiments, the image capturing device is a miniature camera, as long as it has an image capturing function. The calibration point is a fixed reference point. In this embodiment, the calibration point belongs to the pipetting system. In other embodiments, the calibration point does not belong to the pipetting system.

FIG. 4 shows a schematic diagram of the image capturing device capturing the first image of the calibration point in an embodiment. Referring to Figure 4, the calibration point 11 in this embodiment is located above the image capturing device 21. Before capturing the image of the calibration point 11 in step S10, move the image capturing device 21 to the calibration point 11 to enter the image capturing device 21 within the capture area. In other embodiments, the relative position between the calibration point and the image capture device is adjusted, as long as the calibration point can enter the capture area of the image capture device, the image capture device can be calibrated with reference to the calibration point. Can.

Figure 5 shows the main steps of obtaining a first offset in an embodiment. Referring to FIG. 5, the step of obtaining a first offset of a center point of the image capturing device relative to the calibration point in the first image on a plane in step S20 includes: S202, obtaining the The rectangular coordinates of the center point and the calibration point in the first image on the plane, wherein the plane is parallel to the plane where a turntable of the pipetting system is located; S204, the rectangular coordinates of the center point and the calibration point respectively The rectangular coordinate is converted into a polar angle to obtain the included angle of the polar angle of the center point with respect to the polar angle of the calibration point; and S206, obtaining the first offset amount according to the included angle.

Combining Fig. 4 and Fig. 5, the plane is marked as the X-Y plane, and the X-Y plane is parallel to the plane where the image capturing device 21 is located. The plane on which the image capturing device 21 is located is an operating platform for storing and receiving liquid in the pipetting system. For example, in a pipetting system equipped with a turntable, the plane on which the image capturing device 21 is located is the turntable of the pipetting system. In step S202, an X-Y rectangular coordinate system is first established according to the X-Y plane, and then the rectangular coordinates of the center point 211 and the calibration point 11 in the first image in the X-Y rectangular coordinate system are obtained respectively. Since the position of the center point 211 is known after the image capturing device 21 is installed, in some embodiments, the XY rectangular coordinate system is established with the center point 211 as the origin, and the first image is projected to XY The rectangular coordinate system obtains the rectangular coordinates of the calibration point 11 in the first image to facilitate the calculation of the first offset. In some embodiments, an image of the center point 211 is prestored, and the image of the center point 211 and the first image are respectively projected to the X-Y rectangular coordinate system to obtain the rectangular coordinates of the center point 211 and the rectangular coordinates of the calibration point 11.

Among them, the rectangular coordinates of the calibration point 11 are obtained by the center circle contour search algorithm. Figure 6 shows the main steps of obtaining the rectangular coordinates of the calibration point in an embodiment. Referring to Figure 6, the step of obtaining the rectangular coordinates of the calibration point in the first image in step S202 on the plane includes: S2021 , Perform smoothing filtering on the first image to remove the noise of the first image; S2022, Binarize the smoothed and filtered first image to be each pixel point in the first image after smoothing and filtering Assignment; S2023. Perform edge measurement on the first image after binarization, and mark multiple edge points of the first image after binarization; S2024, perform edge measurement on the multiple edges of the first image after edge measurement Perform contour calibration on the edge points to form at least one contour; S2025, perform roundness detection on each contour, and screen out contours whose roundness value is less than a preset value; S2026, perform area screening on the screened contour, and screen out The contour area conforms to a contour of the area of the calibration point; and S2027. Obtain the rectangular coordinates of the centroid of the screened contour on the plane as the rectangular coordinates of the calibration point on the plane in the first image.

Specifically, the binarization in step S2022 is to set the pixel points in the first image after smoothing and filtering that exceed a preset threshold to 1, and vice versa. In this embodiment, the preset threshold is set according to needs, and is not limited here. The edge measurement in step S2023 is to use the Canny edge detection algorithm to mark the edge points of the first image after binarization. The Canny edge detection algorithm is a multi-level edge detection algorithm used to identify the actual edges in the image. The contour calibration of step S2024 connects the connected edge points detected in the previous step to form a contour. Since there may be multiple contours in the first image, multiple groups of connected edge points will be detected during edge measurement. Multiple outlines will be marked. The roundness detection in step S2025 calculates the roundness value of each contour marked in the previous step by the formula "T=4π*S/C ² ", and screens out contours with low roundness values, leaving a contour close to a circle. The preset value used for judging the roundness value is set according to needs, and is not limited here. In the formula "T=4π*S/C ² ", T is the roundness value of a contour, S is the area of the contour, and C is the circumference of the contour. The area S and perimeter C of a contour are obtained by software measurement. The area of the calibration point is known. The area screening of step S2026 calculates the area of each contour left in the previous step, and screens out a contour whose contour area is closest to the area of the calibration point, and then obtains the centroid of the contour through step S2027 The rectangular coordinates on the XY plane are used as the rectangular coordinates of the calibration point 11. Among them, the centroid of a contour is the geometric center of the contour.

In the same way, when the rectangular coordinates of the center point 211 need to be obtained, the center circle contour search algorithm is also used, which will not be explained here.

After obtaining the rectangular coordinates of the center point 211 and the rectangular coordinates of the calibration point 11, the rectangular coordinates of the center point 211 and the rectangular coordinates of the calibration point 11 are respectively converted into polar angles through step S204. Taking the conversion of the rectangular coordinates of the calibration point 11 into a polar angle as an example, the polar angle θ _{11 of the} calibration point 11 is obtained by the following formula:

Among them, (x ₁₁ , y ₁₁ ) are the rectangular coordinates of the calibration point 11 in the XY rectangular coordinate system, and the XY rectangular coordinate system uses a primitive point as the unit coordinate. In the same way, according to the rectangular coordinates (x ₂₁₁ , y ₂₁₁ ) of the center point 211, the polar angle θ _{211 of the} center point 211 is obtained, which will not be further described here. Further, the center point of the polar angle [theta] 211 ₂₁₁ subtracts the polar angle [theta] 11 of the calibration point _11, to obtain the polar angle [theta] ₂₁₁ with respect to the center point 211 of the calibration point polar angle [theta] 11 ₁₁ Delta] [theta angle as a first offset quantity. In the embodiment where the pipetting system is configured with the turntable, as shown in Figure 4, the included angle Δθ is a rotation angle of the turntable along the direction of rotation of the turntable.

Subsequently, the position of the image capturing device 21 is adjusted according to the first offset, so that the center point 211 is calibrated with the calibration point 11 to compensate for the first offset. For example, the turntable of the pipetting system is controlled to rotate Δθ so that the center point 211 is aligned with the calibration point 11 in a direction perpendicular to the X-Y plane.

Figure 7 shows a schematic diagram of the image capturing device capturing a second image of the tip of a pipette in an embodiment, and Figure 8 shows the image capturing device capturing a second image of the tip of the pipette in an embodiment A schematic representation of the third image. Referring to Figures 7 and 8, the pipette 31 in this embodiment is located above the image capturing device 21. Before capturing the image of the tip 311 of the pipette 31, move the pipette 31 to the pipette The tip 311 of the 31 enters the capturing area of the image capturing device 21. In other embodiments, the relative position between the pipette and the image capture device can be adjusted, as long as the tip of the pipette can enter the capture area of the image capture device, and the tip of the pipette refers to the image The conditions under which the capture device is calibrated. The pipette 31 is installed on a pipetting device 30 of the pipetting system, such as a vertical syringe pump. Wherein, when capturing the second image in Figure 7, the center of the tip 311 of the pipette 31 and the center point 211 of the image capturing device 21 are aligned in the direction of the Z coordinate axis perpendicular to the XY plane, that is, The pipette 31 conforms to the original assembled state, and in the original assembled state, the center of the tip 311 is projected to the origin of the XY rectangular coordinate system. Alternatively, the calibrated center point 211 is used as the origin of the XYZ three-dimensional coordinate system in Figures 7 and 8. Since the center point 211 has been calibrated relative to the calibration point 11, the center point 211 is used as the origin of the XYZ three-dimensional coordinate system. Point, which is conducive to the positioning of the subsequent calibration steps. When the third image was captured in Figure 8, the pipette 31 was re-installed, and the focus distance between the tip 311 and the calibrated center point 211 was shifted, and the tip 311 and the calibrated center point There may also be some displacement between 211 and the XY plane.

Figure 9 shows the main steps of obtaining a second offset in an embodiment. Referring to FIG. 9, the step of obtaining a second offset of the tip in the third image relative to the tip in the second image on the plane in step S40 includes: S402, obtaining the third image respectively A first rectangular coordinate of the center of the tip in the image on the plane, and a second rectangular coordinate of the center of the tip in the second image on the plane, wherein the plane is perpendicular to the direction of the focusing distance; S404. Obtain the pixel offset of the center of the tip in the third image relative to the center of the tip in the second image according to the first rectangular coordinate and the second rectangular coordinate, where the rectangular coordinate is A graphic element is used as a standard unit; and S406, according to the corresponding relationship between a graphic element and the distance, the graphic element offset is converted into a distance offset as the second offset.

In combination with Figures 7-9, the direction of the focus distance is the Z-axis direction, and the second offset is the center of the tip 311 in the third image relative to the center of the tip 311 in the second image, that is The offset Δx and Δy of the calibrated center point 211 on the XY plane. Since the X-Y rectangular coordinate system uses a primitive point as the unit coordinate, the obtained coordinate offsets Δx and Δy are the primitive offsets. Further, according to the corresponding relationship between the graphic element and the distance, the distance offset corresponding to the graphic element offset can be obtained.

In step S402, the first right-angle coordinate of the center of the tip in the third image is obtained by using a center circle contour search algorithm. Figure 10 shows the main steps of obtaining a first right-angle coordinate of the center of the tip in the third image in an embodiment. Referring to Figure 10, the center of the tip in the third image is obtained in step S402. The step of a first rectangular coordinate of the plane includes: S4021, smoothing and filtering the third image to remove the noise of the third image; S4022, binarizing the smoothed and filtered third image, which is a smoothing filter Assign a value to each pixel point in the third image afterwards; S4023, perform edge measurement on the third image after binarization, and mark multiple edge points of the third image after binarization; S4024, Perform contour calibration on the plurality of edge points of the third image after edge measurement to form at least one contour; S4025. Perform roundness detection on each contour, and screen out contours whose roundness value is less than a preset value; S4026, Perform area screening on the screened contour, and screen out a contour whose contour area matches the area of the tip; and S4027. Obtain the rectangular coordinates of the center of the screened contour on the plane as the third image. The center of the tip is in the rectangular coordinates of the plane. The specific principle of each step of the center circle contour search algorithm is the same as the process of obtaining the rectangular coordinates of the calibration point 11 in the above-mentioned embodiment, so the description will not be repeated.

As described above, in some embodiments, the second rectangular coordinate of the center of the tip 311 in the second image on the X-Y plane is the origin of the X-Y rectangular coordinate system. In other embodiments, when it is necessary to obtain the second rectangular coordinates of the center of the tip 311 in the second image on the X-Y plane, the center circle contour search algorithm is also used, which will not be further described here.

In step S406, an object of a known size is captured by the image capturing device 21 to obtain the corresponding relationship between the image element and the distance. Figure 11 shows the main steps for obtaining the correspondence between a picture element and a distance in an embodiment. Referring to Fig. 11, the step S406 according to the correspondence between a picture element and the distance includes: S4062, through the image The capturing device captures an image of an object of a preset size; and S4064, according to the number of image elements of the image of the object and the preset size, obtain the corresponding relationship between the image element and the distance. For example, if an image of an object known to be 100 μm long is captured by the image capturing device 21, and the number of image elements of the image of the object is 10, then a corresponding relationship of 1 image element equal to 10 μm is obtained. According to the correspondence between the image element and the distance, the image element offset of the tip 311 in the third image relative to the tip 311 in the second image is converted into an offset distance.

For example, in an example, referring to Figures 7 and 8, the second rectangular coordinates of the center of the tip 311 in the second image calculated by the center circle contour search algorithm on the XY plane are (297, 249) , The first right-angle coordinate of the center of the tip 311 in the third image on the XY plane is (240, 257). Then, the offset of the tip 311 from the X coordinate axis in the third image is Δx=-57, and the offset from the Y coordinate axis is Δy=8. Further according to the corresponding relationship between the image element and the distance, the tip 311 in the third image is obtained with an offset of 570 μm on the X axis and an offset of 80 μm on the Y axis.

The second image and the third image are also used to obtain the focus distance offset of the tip 311. Figure 12 shows the main steps for obtaining the focus distance offset in an embodiment. Referring to Figure 12, in this embodiment, step S50 is based on the displacement and the third image relative to the second image. The variation of the sharpness value and a mapping relationship table between a preset focusing distance and sharpness value, and the step of obtaining a focusing distance offset of the sharpness value of the third image with respect to a predetermined sharpness value includes: S502. Obtain the position of the sharpness value of the third image in the mapping table according to the direction of the displacement and the amount of variation of the sharpness value of the third image with respect to the second image, where the sharpness value of the third image is in the mapping table The sharpness value has a Gaussian distribution; and S504, according to the position of the sharpness value of the third image in the mapping relationship table, obtain the focusing distance corresponding to the sharpness value of the third image relative to a preset sharpness value The focus distance offset corresponding to the focus distance, where the preset sharpness value is the highest sharpness value in the mapping relationship table.

Among them, the mapping relationship table between the focus distance and the sharpness value is preset. Before calibrating the pipetting system, first use the image capture device to shoot the cutting-edge images multiple times, calculate the sharpness value at different focusing distances, and obtain a mapping table that can reflect the correspondence between the focusing distance and the sharpness value. . For example, in an embodiment, the mapping relationship between the focus distance and the sharpness value is shown in Table 1 below:

Table 1: The mapping relationship between focus distance and sharpness value: Focus distance/μm 6220 6240 6260 6280 6300 6320 6340 Clarity value 0.862379 0.865822 0.890014 1 0.974314 0.859491 0.8526

Among them, the sharpness value is the sharpness value after normalization. Since there is an optimal focusing distance between the tip and the center point of the image capturing device, when the distance between the tip and the center point is equal to the optimal focusing distance, the sharp image captured by the image capturing device has the highest definition value. When the distance between the tip and the center point becomes larger or smaller with respect to the optimal focusing distance, the sharpness value of the sharp image captured by the image capturing device will decrease relative to the highest sharpness value. In other words, the sharpness value of the image has a Gaussian distribution with respect to the focus distance. Under this rule, the definition value in the acquired mapping table is Gaussian. Therefore, when obtaining the focus distance between the tip and the center point, it is necessary to shoot the image of the tip with the shifted focus distance at least twice to obtain a specific position of the sharpness value in the mapping relationship table. In some embodiments, the pipette is reinstalled multiple times until the exact focusing distance between the tip and the center point can be obtained. It should be noted that in this embodiment, only part of the focus distance and sharpness values are shown in the above mapping relationship table. In other embodiments, in the mapping relationship table, there are multiple left and right sides of the highest sharpness value. The sharpness value is approximately symmetrically distributed.

Referring to Figures 7 and 8, the displacement is a known quantity along the Z coordinate axis. By shifting the focus distance between the tip 311 and the center point 211, the sharpness value of the image of the tip 311 captured twice is changed to obtain the focusing distance corresponding to the sharpness value of the third image of the tip 311 , As the actual distance between the tip 311 and the center point 211. Taking the direction of the Z coordinate axis shown in Figures 7 and 8 as an example, when the tip 311 moves a certain distance along the Z coordinate axis with respect to the center point 211 of the image capturing device 21, the tip 311 and the calibrated The focus distance between the center points 211 increases; when the tip 311 moves down a certain distance along the Z coordinate axis relative to the center point 211 of the image capturing device 21, the focus distance between the tip 311 and the calibrated center point 211 Decrease.

其中，Src指的是原始圖像的二維矩陣，而dst就是以src分別對x及y作二階偏微再加總的二維矩陣，目的是藉由求x方向與y方向的變化量找出影像的邊緣值，也就是清晰度值。 The amount of variation of the sharpness value of the third image with respect to the second image refers to the difference between the sharpness value of the third image and the sharpness value of the second image. Use the existing method to obtain the sharpness value of the image. Specifically, there are many algorithms for evaluating the sharpness value of an image. In the spatial domain, it is mainly to examine the spatial contrast of the image, that is, the gradient difference of gray-scale features between adjacent pixels; in the frequency domain, It mainly examines the frequency components of the image. The image with sharp focus has more high frequency components, and the image with blurred focus has more low frequency components. In one embodiment, the Laplace operator is used to obtain the sharpness values of the second image and the third image, and the Laplace operator calculates the gradients in the X-axis direction and the Y-axis direction respectively. The clearer the image in the same scene , The higher the gradient value. The formula of the Laplace operator is as follows:

Among them, Src refers to the two-dimensional matrix of the original image, and dst is the two-dimensional matrix obtained by subtracting x and y respectively with src, and then adding up the two-dimensional matrix. The purpose is to find the amount of change in the x and y directions. The edge value of the output image, that is, the sharpness value.

In one embodiment, the preset sharpness value is the highest sharpness value 1 in the above mapping relationship table, and the corresponding optimal focusing distance is 6280 μm. According to the calculation of the Laplace operator, the sharpness value of the second image of the tip is 0.865822, then the focus distance is increased, the pipette is moved up by 20μm, and the third image of the tip is obtained by shooting again, and the third image is calculated by the Laplace operator The clarity value of is 0.890014. According to the characteristic that the sharpness value increases with the increase of the focusing distance and the law of Gaussian distribution, it can be known that the focusing distance between the tip and the center point when shooting the second image and the third image is less than the optimal focusing distance. For optimal focusing distance, it is determined that the sharpness value 0.890014 of the third image in the mapping relationship table is located to the left of the highest sharpness value 1, and the focusing distance corresponding to the sharpness value 0.890014 of the third image is 6260 μm. Of course, when moving the pipette, the amount of displacement should also be controlled to prevent the subsequent steps from being affected by excessive or small displacement. In some embodiments, the pipette is reinstalled multiple times to obtain a more accurate focusing distance between the tip and the center point. Next, obtain a focus distance of 6260 μm corresponding to the sharpness value of 0.890014 of the third image with a focus distance offset of 20 μm relative to the focusing distance of 6280 μm corresponding to the preset sharpness value 1, as shown in Figure 8. Δz.

。 Further, in one embodiment, the calibration method of the pipetting system further includes setting the imaging depth of the image capturing device so that the detectable range of the focus distance offset is within a preset distance range, wherein In the step of taking the third image, the displacement is greater than the preset distance range. Use the following formula to set the imaging depth of field d _{tot of the} image capture device:

.

Among them, NA is the numerical aperture of the objective lens of the image capturing device, M is the magnification of the objective lens, NA and M are both known quantities. In this embodiment, an objective lens with NA being 0.3 and M being 2.66 is used. λ is the wavelength of light wave, usually λ=0.55μm. n is the refractive index of the medium between the pipette and the objective lens. In this embodiment, the medium is air, and the refractive index is n=1. And e is the minimum distance that the image capturing device can distinguish, which is a known quantity. In this embodiment, e=2.2 μm. Therefore, in this embodiment, the imaging depth of field d _{tot of the} image capturing device is calculated as 8.86 μm. When the imaging depth of field is 8.86 μm, the focus distance offset greater than 8.86 μm is detectable, that is, the detectable range of the focus distance offset in this embodiment is greater than 8.86 μm. Therefore, in the step of capturing the third image, when reinstalling the pipette, the focus distance between the tip of the pipette and the center point of the image capturing device needs to be displaced more than 8.86μm, so that the image can be captured. Whether the variation of the sharpness value between the second image and the third image of the tip captured by the device can be detected.

Further, after the second offset and the focus distance offset are obtained, the position of the pipette is adjusted to compensate for the second offset and the focus distance offset, so that the tip of the pipette is aligned with the calibrated center point Align in the three-dimensional directions of the X coordinate axis, the Y coordinate axis and the Z coordinate axis to complete the calibration of the pipette.

In summary, the calibration method of the pipetting system in the above embodiment captures the image of the calibration point by the image capture device, and obtains the first offset of the center point of the image capture device with respect to the calibration point on the XY plane, so as to realize the image Calibration of the turntable of the pipetting system where the capture device is located on the XY plane, so that the calibrated image capture device is used for positioning in the subsequent calibration steps; the tip of the pipette is captured at least twice by the calibrated image capture device The second offset of the tip of the pipette on the XY plane and the offset of the focus distance on the Z coordinate axis are obtained to realize the calibration of the tip of the pipette on the XY plane and the Z coordinate axis; The method for calibrating the pipetting system utilizes an image capturing device to realize the calibration of the rotation direction of the turntable of the pipetting system and the calibration of the three-dimensional direction of the pipette, so as to realize subsequent pipetting operations and system operation control.

This case also discloses a pipetting system. Figure 13 shows the main modules of a pipetting system in an embodiment. With reference to Figure 13, the pipetting system 1 in this embodiment includes: an image capture device 21; and a processor 41. 41 is configured to be implemented by executing a plurality of executable instructions: capturing at least one first image of a calibration point by the image capturing device 21; obtaining a center point of the image capturing device 21 relative to the first image A first offset of the calibration point on a plane, and adjust the position of the image capture device 21 according to the first offset so that the center point is calibrated with the calibration point; the image capture after calibration The capturing device 21 captures at least a second image and a third image of the tip of a pipette 31 of the pipetting system 1, wherein the third image is captured from the focus distance between the tip and the center point after calibration After the displacement occurs; obtain a second offset of the tip in the third image relative to the tip in the second image on the plane; respectively according to the displacement, the third image relative to the second image The variation of the sharpness value and a preset mapping relationship table between the focus distance and the sharpness value to obtain a focusing distance offset of the sharpness value of the third image relative to a preset sharpness value; and, according to The second offset and the focus distance offset adjust the position of the pipette 31 so that the tip is calibrated with the center point after calibration. Among them, in some embodiments, the executable instructions are stored in the processor 41, and in some embodiments, the executable instructions are stored in a memory of the pipetting system 1. For the specific execution process and principle of the executable instructions, please refer to the description of the above described embodiment of the calibration method of the pipetting system, and the description will not be repeated here.

In one embodiment, the processor 41 is, for example, a CPU (central processing unit, central processing unit). The processor 41 is respectively connected to the image capturing device 21 and the pipette 31 of the pipetting system 1 to realize the calculation of the first offset, the second offset, and the focus distance offset, and according to the first offset The amount, the second offset, and the focus distance offset respectively control the image capture device 21 and the pipette 31 to perform position adjustment, so as to realize the calibration of the pipetting system 1.

In summary, the pipetting system of the above embodiment captures the image of the calibration point through the image capture device, obtains the first offset of the center point of the image capture device relative to the calibration point in the plane dimension, and realizes that the image capture device is The calibration of the plane dimension enables the calibrated image capture device to be used for the positioning of the subsequent calibration steps; the image of the tip of the pipette is captured by the calibrated image capture device at least twice to obtain the tip of the pipette on the plane The second offset of the dimension and the offset of the focus distance in the focus distance dimension realize the calibration of the tip of the pipette in the plane dimension and the focus distance dimension; thus, this case uses an image capture device to achieve the pipetting system Calibration to control subsequent pipetting operations and system operation.

In summary, although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Those with ordinary knowledge in the technical field to which the present invention belongs can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to those defined by the attached patent application scope.

1: Pipetting system 10: Turntable 11: Calibration point 101: Pipette replacement area 102: Pipetting area 21: Image capture device 211: Center point 30: Pipetting device 31: Pipette 300: Slide rail 41: Processor Δθ: included angle Δx, Δy, Δz: offset

Figure 1 shows a schematic structural diagram of a pipetting system in an embodiment of the present case; Figure 2 shows a schematic structural diagram of a pipetting device in an embodiment of the present case; Figure 3 shows a schematic flow chart of a method for calibrating a pipetting system in an embodiment of the present case; FIG. 4 shows a schematic diagram of an image capturing device capturing a first image of a calibration point in an embodiment of the present invention; Figure 5 shows a schematic flow chart of obtaining a first offset in an embodiment of the present case; Figure 6 shows a schematic diagram of the process of obtaining the rectangular coordinates of the calibration point in an embodiment of the present case; FIG. 7 shows a schematic diagram of the image capturing device capturing a second image of the tip of a pipette in an embodiment of the present invention; FIG. 8 shows a schematic diagram of the image capturing device capturing a third image of the tip of the pipette in an embodiment of the present invention; Figure 9 shows a schematic flow chart of obtaining a second offset in an embodiment of the present case; Figure 10 shows a schematic flow chart of obtaining a first right-angle coordinate of the center of the tip in the third image in an embodiment of the present case; Figure 11 shows a schematic flow chart of obtaining the correspondence between a graphic element and a distance in an embodiment of the present case; Figure 12 shows a schematic diagram of the flow of obtaining a focus distance offset in an embodiment of the present case; and Figure 13 shows a schematic diagram of a module of a pipetting system in an embodiment of the present case.

S10~S60: Each step

Claims (10)

A method for calibrating a pipetting system includes: Capturing at least one first image of a calibration point by an image capturing device; Obtain a first offset of a center point of the image capture device relative to the calibration point in the first image on a plane, and adjust the image capture device according to the first offset , So that the center point and the calibration point are calibrated; At least a second image and a third image of the tip of a pipette of the pipetting system are captured by the calibrated image capture device, wherein the third image is captured from the tip and calibration After the subsequent focus distance of the center point is shifted; Obtaining a second offset of the tip in the third image relative to the tip in the second image on the plane; Obtain the sharpness of the third image according to the displacement, the variation of the sharpness value of the third image relative to the second image, and a preset mapping relationship table between the focus distance and sharpness value. A focus distance offset relative to a preset sharpness value; and The position of the pipette is adjusted according to the second offset and the focus distance offset, so that the tip is calibrated with the center point after calibration. According to the calibration method described in item 1 of the scope of patent application, the obtaining of a first offset of a center point of the image capturing device relative to the calibration point in the first image on a plane The steps include: Obtaining the rectangular coordinates of the center point and the calibration point in the first image on the plane, respectively, wherein the plane is parallel to a plane where a turntable of the pipetting system is located; Respectively converting the rectangular coordinates of the center point and the rectangular coordinates of the calibration point into polar angles to obtain the angle between the polar angle of the center point and the polar angle of the calibration point; and The first offset is obtained according to the included angle. According to the calibration method described in item 2 of the scope of patent application, the step of obtaining the rectangular coordinates of the calibration point in the first image on the plane includes: Smoothing and filtering the first image to remove noise in the first image; Binarize the smoothed and filtered first image, and assign a value to each pixel point in the smoothed and filtered first image; Performing edge measurement on the first image after binarization, and mark a plurality of edge points of the first image after binarization; Performing contour calibration on the plurality of edge points of the first image after edge measurement to form at least one contour; Perform roundness detection on each of the contours, and screen out contours whose roundness value is less than a preset value; Performing area screening on the sieved contour, and sifting out a contour whose contour area conforms to the area of the calibration point; and Obtain the rectangular coordinates of the centroid of the screened contour on the plane as the rectangular coordinates of the calibration point in the first image on the plane. According to the calibration method described in item 1 of the scope of patent application, the step of obtaining a second offset of the tip in the third image relative to the tip in the second image on a plane include: Respectively obtain a first rectangular coordinate of the center of the tip in the third image on the plane, and a second rectangular coordinate of the center of the tip in the second image on the plane, wherein The plane is perpendicular to the direction in which the focusing distance is located; According to the first rectangular coordinates and the second rectangular coordinates, the pixel offset of the center of the tip in the third image relative to the center of the tip in the second image is obtained, where The rectangular coordinates use a graphic element as a standard unit; and According to the corresponding relationship between a graphic element and the distance, the graphic element offset is converted into a distance offset as the second offset. According to the calibration method described in item 4 of the scope of patent application, the step of obtaining a first right-angle coordinate of the center of the tip in the third image on the plane includes: Smoothing and filtering the third image to remove the noise of the third image; Binarize the third image after smoothing and filtering, and assign a value to each pixel point in the third image after smoothing and filtering; Performing edge measurement on the third image after the binarization, and marking a plurality of edge points of the third image after the binarization; Performing contour calibration on the plurality of edge points of the third image after edge measurement to form at least one contour; Perform roundness detection on each of the contours, and screen out contours whose roundness value is less than a preset value; Performing area screening on the sieved contour, and the sieved contour area conforms to a contour of the area of the tip; and Obtain the rectangular coordinates of the centroid of the screened contour on the plane as the rectangular coordinates of the center of the tip in the third image on the plane. For the calibration method described in item 4 of the scope of the patent application, the step according to the corresponding relationship between a picture element and the distance includes: Capturing an image of an object of a preset size by the image capturing device; and According to the number of image elements of the image of the object and the preset size, the corresponding relationship between the image elements and the distance is obtained. According to the calibration method described in item 1 of the scope of patent application, the said displacement is based on the deviation of the sharpness value of the third image with respect to the second image, and a preset focusing distance and sharpness. Value mapping table, the step of obtaining a focus distance offset of the sharpness value of the third image relative to a preset sharpness value includes: Obtain the position of the definition value of the third image in the mapping table according to the direction of the displacement and the amount of variation of the definition value of the third image relative to the second image, wherein the The sharpness value in the mapping relationship table has a Gaussian distribution; and According to the position of the sharpness value of the third image in the mapping relationship table, the focusing distance corresponding to the sharpness value of the third image relative to the focusing distance corresponding to a preset sharpness value is obtained The distance offset, wherein the preset sharpness value is the highest sharpness value in the mapping relationship table. The calibration method described in item 1 of the scope of patent application further includes setting the imaging depth of the image capturing device so that the detectable range of the focus distance offset is within a preset distance range, wherein In the step of taking the third image, the displacement is greater than the preset distance range. A pipetting system includes: An image capture device; and A processor, the processor is configured to be implemented by executing a plurality of executable instructions: Capturing at least one first image of a calibration point by the image capturing device; Obtain a first offset of a center point of the image capture device relative to the calibration point in the first image on a plane, and adjust the image capture device according to the first offset , So that the center point and the calibration point are calibrated; At least a second image and a third image of the tip of a pipette of the pipetting system are captured by the calibrated image capture device, wherein the third image is captured from the tip and calibration After the subsequent focus distance of the center point is shifted; Obtaining a second offset of the tip in the third image relative to the tip in the second image on the plane; Obtain the sharpness of the third image according to the displacement, the variation of the sharpness value of the third image relative to the second image, and a preset mapping relationship table between the focus distance and sharpness value. A focus distance offset relative to a preset sharpness value; and The position of the pipette is adjusted according to the second offset and the focus distance offset, so that the tip is calibrated with the center point after calibration. The pipetting system described in item 9 of the scope of patent application further includes: A turntable with the image capturing device.

Images