CN113528583A

CN113528583A - Automatic microinjection method, device, system, equipment and storage medium

Info

Publication number: CN113528583A
Application number: CN202110801066.0A
Authority: CN
Inventors: 徐青松; 池子强
Original assignee: University of Macau
Current assignee: University of Macau
Priority date: 2021-07-15
Filing date: 2021-07-15
Publication date: 2021-10-22

Abstract

The application provides an automatic microinjection method, device, system, equipment and storage medium, and relates to the technical field of automation. The method can comprise the following steps: acquiring an image of a first target injection object in an image to be injected, wherein the image to be injected is an image of a plurality of injection objects in a culture dish acquired by image acquisition equipment, and the first target injection object is an initial injection object in the image to be injected; performing image processing on the image of the first target injection object to obtain a processed image of the first target injection object; determining a first target injection target position in the first target injection object according to the processed image of the first target injection object; and controlling the injector to inject the first target injection site. The method can realize automatic injection operation, and improve injection efficiency and injection accuracy.

Description

Automatic microinjection method, device, system, equipment and storage medium

Technical Field

The application relates to the technical field of automation, in particular to an automatic microinjection method, device, system, equipment and storage medium.

Background

The zebra fish (Danio rerio) is highly similar to human genes, the zebra fish is low in breeding cost and rapid in breeding, and larvae of the zebra fish have a plurality of typical vertebrate characteristics, so that biological and medical experiments carried out by the zebra fish as a carrier have important significance for human scientific research. Researchers often use microinjection to directly deliver substances to target sites and organs in zebrafish experiments, and even can be used for injecting substances such as cells and targeted drugs.

In the prior art, a "micropipette holder" is typically used to control the position of the larvae for administering the medication to the larvae.

However, the method is slow in speed, the average treatment time for controlling the rotation of a single zebra fish juvenile fish and the injection is dozens of seconds, and the method is difficult to adapt to the requirement of high-flux injection.

Disclosure of Invention

An object of the application is to provide an automatic microinjection method, device, system, equipment and storage medium aiming at the defects in the prior art, so as to solve the problems that the injection efficiency of zebra fish larvae medicine is low and the high-flux injection requirement is difficult to meet in the prior art.

In order to achieve the above purpose, the technical solutions adopted in the embodiments of the present application are as follows:

in a first aspect, an embodiment of the present application provides an automated microinjection method, which is applied to an upper computer in an automated microinjection system, where the automated microinjection system includes: the device comprises image acquisition equipment, a culture dish, an injector and the upper computer; the method comprises the following steps:

acquiring an image of a first target injection object in an image to be injected, wherein the image to be injected is an image of a plurality of injection objects in the culture dish acquired by the image acquisition equipment, and the first target injection object is an initial injection object in the image to be injected;

performing image processing on the image of the first target injection object to obtain a processed image of the first target injection object;

determining a first target injection target position in the first target injection object according to the processed image of the first target injection object;

and controlling the injector to inject the first target injection target.

Optionally, after controlling the injector to inject the first target injection target, the method further comprises:

acquiring an image of a second target injection object in the image to be injected according to a pre-planned injection track, wherein the second target injection object is a next injection object of the first target injection object;

performing image processing on the image of the second target injection object to obtain a processed image of the second target injection object;

determining a second target injection target position in the second target injection object according to the processed image of the second target injection object;

and controlling the injector to inject the second target injection target position according to the second target injection target position, and stopping injecting until all the injection objects in the culture dish are injected.

Optionally, the acquiring an image of a first target injection object in the image to be injected includes:

identifying a location of a first target injection subject in the image to be injected;

and controlling the image acquisition equipment to acquire the image of the first target injection object according to the position of the first target injection object in the image to be injected.

Optionally, the system further comprises: the culture dish is arranged on the positioning platform;

the controlling the image acquisition equipment to acquire the image of the first target injection object according to the position of the first target injection object in the image to be injected comprises the following steps:

controlling the positioning platform to move according to the position of the first target injection object so as to move the first target injection object in the culture dish to be under the visual field of the image acquisition equipment;

and acquiring the image of the first target injection object, which is acquired and sent by the image acquisition equipment according to the position of the first target injection object.

Optionally, before the determining the first target injection target in the first target injection subject according to the processed image of the first target injection subject, the method further comprises:

and if the culture dish is judged to have the inclination, performing inclination correction on the processed image of the first target injection object to obtain a corrected image of the first target injection object.

Optionally, the performing tilt correction on the processed image of the first target injection object to obtain a corrected image of the first target injection object includes:

determining the position of a preset edge in the processed image of the first target injection object;

determining an inclination angle according to the position of the preset edge;

and according to the inclination angle and the transformation matrix, carrying out coordinate transformation on each vertex in the processed image of the first target injection object to obtain a corrected image of the first target injection object.

Optionally, the determining a first target injection target in the first target injection subject according to the processed image of the first target injection subject includes:

and determining a first target injection target position in the first target injection object by adopting an identification algorithm corresponding to the first target injection target position according to the processed image of the first target injection object.

Optionally, the controlling the injector to inject the first target injection target includes:

controlling the positioning platform to move the first target position of the first target injection object to an injection position corresponding to the injector according to the first target injection target position;

and sending a driving signal to the injection pump so as to drive the injector to inject to the first target injection target position through the injection pump.

Optionally, after sending a driving signal to the syringe pump to drive the syringe by the syringe pump to inject into the first target injection target, the method further includes:

sending a syringe exit signal to the displacement operator to control the syringe to exit from the first target injection site.

Optionally, the performing image processing on the image of the first target injection object to obtain a processed image of the first target injection object includes:

and carrying out image binarization processing and filtering processing on the image of the first target injection object to obtain a processed image of the first target injection object.

In a second aspect, an embodiment of the present application further provides an automatic microinjection apparatus, which is applied to an upper computer in an automatic microinjection system, where the automatic microinjection system includes: the device comprises image acquisition equipment, a culture dish, an injector and the upper computer; the device comprises: the device comprises an acquisition module, a processing module, a determination module and a control module;

the acquisition module is used for acquiring an image of a first target injection object in an image to be injected, wherein the image to be injected is an image of a plurality of injection objects in the culture dish acquired by the image acquisition equipment, and the first target injection object is an initial injection object in the image to be injected;

the processing module is used for carrying out image processing on the image of the first target injection object to obtain a processed image of the first target injection object;

the determining module is used for determining a first target injection target position in the first target injection object according to the processed image of the first target injection object;

the control module is used for controlling the injector to inject the first target injection target.

Optionally, the acquiring module is further configured to acquire an image of a second target injection object in the image to be injected according to a pre-planned injection trajectory, where the second target injection object is a next injection object of the first target injection object;

the processing module is further configured to perform image processing on the image of the second target injection object to obtain a processed image of the second target injection object;

the determining module is further used for determining a second target injection target position in the second target injection object according to the processed image of the second target injection object;

the control module is further used for controlling the injector to inject the second target injection target position according to the second target injection target position, and stopping injection until all injection objects in the culture dish are injected.

Optionally, the obtaining module is specifically configured to

the acquisition module is particularly used for

Optionally, the apparatus further comprises: a correction module;

and the correction module is used for performing inclination correction on the processed image of the first target injection object to obtain a corrected image of the first target injection object if the culture dish is judged to have inclination.

Optionally, the correction module, in particular for

determining an inclination angle according to the position of the preset edge;

Optionally, the determining means is specifically configured to

Optionally, the system further comprises: an injection pump;

the control module is particularly used for

Optionally, the control module is further used for

Optionally, the processing module is specifically configured for

In a third aspect, embodiments of the present application further provide an automated microinjection system, including: the device comprises image acquisition equipment, a positioning platform, a culture dish, an injector, an injection pump, a displacement manipulator and an upper computer;

the upper computer is in communication connection with the image acquisition device, the positioning platform, the injection pump and the displacement manipulator respectively, and is used for executing the steps of the method provided in the first aspect.

In a fourth aspect, an embodiment of the present application provides an electronic device, including: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium communicating via the bus when the electronic device is operated, the processor executing the machine-readable instructions to perform the steps of the method as provided in the first aspect when executed.

In a fifth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, performs the steps of the method as provided in the first aspect.

The beneficial effect of this application is:

the application provides an automated microinjection method, apparatus, system, device, and storage medium, the method may include: acquiring an image of a first target injection object in an image to be injected, wherein the image to be injected is an image of a plurality of injection objects in a culture dish acquired by image acquisition equipment, and the first target injection object is an initial injection object in the image to be injected; performing image processing on the image of the first target injection object to obtain a processed image of the first target injection object; determining a first target injection target position in the first target injection object according to the processed image of the first target injection object; and controlling the injector to inject the first target injection site. In the method, the first target injection target position can be identified from the first target injection object after image processing is carried out on the basis of the acquired image of the first target injection object, wherein the identification of the first target injection target position can be realized through an image identification algorithm, the identification efficiency and accuracy can be improved, and the injector can be controlled to inject on the basis of the identified first target injection target position, so that the automatic injection operation is completed, and the injection efficiency and the injection accuracy are improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.

FIG. 1 is a schematic diagram of an automated microinjection system according to an embodiment of the present application;

FIG. 2 is a first schematic flow chart of an automated microinjection method according to an embodiment of the present disclosure;

FIG. 3 is a schematic view of a microarray culture dish according to an embodiment of the present invention;

FIG. 4 is a second schematic flow chart of an automated microinjection method according to an embodiment of the present application;

FIG. 5 is a schematic view of an injection trajectory provided by an embodiment of the present application;

FIG. 6 is a third schematic flow chart of an automated microinjection method provided in an embodiment of the present application;

FIG. 7 is a fourth schematic flow chart of an automated microinjection method provided by an embodiment of the present application;

FIG. 8 is a fifth schematic flow chart of an automated microinjection method provided by an embodiment of the present application;

FIG. 9 is a schematic diagram of an image correction provided in an embodiment of the present application;

FIG. 10 is a sixth schematic flow chart of an automated microinjection method provided by an embodiment of the present application;

FIG. 11 is a schematic diagram of an automatic microinjection process provided by an embodiment of the present application;

FIG. 12 is a schematic view of an automated microinjection apparatus according to an embodiment of the present application;

fig. 13 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Icon: 1-an upper computer; 2-culture dish; 3-high definition camera; 4-biological microscope; 5-a syringe; 6-a displacement operator; 7-an injection pump; 8-positioning the platform.

Detailed Description

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for illustrative and descriptive purposes only and are not used to limit the scope of protection of the present application. Additionally, it should be understood that the schematic drawings are not necessarily drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and steps without logical context may be performed in reverse order or simultaneously. One skilled in the art, under the guidance of this application, may add one or more other operations to, or remove one or more operations from, the flowchart.

In addition, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.

In order to enable the skilled person to use the present disclosure, the following embodiments are given in connection with the specific application scenario "zebrafish larva drug injection". It will be apparent to those skilled in the art that the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the application. Although the present application is described primarily in the context of zebrafish larva drug injection, it should be understood that this is only one exemplary embodiment. The present application may be applied to any other scenario. For example, the present application may be applied to drug injection and the like of any other living organisms.

It should be noted that in the embodiments of the present application, the term "comprising" is used to indicate the presence of the features stated hereinafter, but does not exclude the addition of further features.

FIG. 1 is a schematic diagram of an automated microinjection system according to an embodiment of the present application; alternatively, the methods provided herein may be applied to the automated microinjection system, which, as shown in fig. 1, may include: the system comprises an upper computer 1, a culture dish 2, an image acquisition device (wherein the image acquisition device can be composed of a high definition camera 3 and a biological microscope 4), an injector 5, a displacement manipulator 6 (wherein an XYZ multi-degree-of-freedom micromanipulator can be adopted), an injection pump 7 and a positioning platform 8 (wherein an XY micro-displacement platform can be adopted), wherein the upper computer can be respectively in communication connection with the image acquisition device, the displacement manipulator, the injection pump and the positioning platform.

Optionally, a plurality of target objects waiting for microinjection can be placed in the culture dish, the culture dish is placed on the positioning platform, the upper computer can control the movement of the positioning platform, so as to move the culture dish placed on the positioning platform, so as to move the target object to be injected currently in the culture dish to the visual field of the image acquisition equipment, the image acquisition equipment can shoot the image of the target object to be injected currently, and is sent to an upper computer which can carry out image processing identification and the like according to the image of the target object to be injected, and finally, the target position to be injected is determined from the target object to be injected, the upper computer can move the injector by controlling the displacement manipulator, so as to move the injector to the target position to be injected, the upper computer controls the injection pump to drive the injector to inject the medicine to the target position to be injected, and therefore automatic medicine injection is completed.

After injection of the current target object to be injected is completed, the upper computer can continue to control the positioning platform to move so as to move the next target object to be injected in the culture dish to the visual field of the image acquisition equipment, and therefore the steps are executed in a circulating mode until the injection of all the target objects in the culture dish is completed.

The method can realize high-flux and high-efficiency microinjection by combining the provided automatic microinjection system with an automatic microinjection method.

Next, the implementation of the automated microinjection method of the present application will be described in detail by way of a number of specific embodiments.

FIG. 2 is a first schematic flow chart of an automated microinjection method according to an embodiment of the present disclosure; FIG. 3 is a schematic view of a microarray culture dish according to an embodiment of the present disclosure. The execution subject of the method may be an upper computer in the system, as shown in fig. 2, the method may include:

s201, acquiring an image of a first target injection object in the image to be injected, wherein the image to be injected is an image of a plurality of injection objects in a culture dish acquired by an image acquisition device, and the first target injection object is an initial injection object in the image to be injected.

First, the design and fabrication of the culture dish involved in the present method is briefly described before. The following example is developed by taking the application of the method in a drug injection scene of zebra fish larvae as an example.

Zebrafish embryonic egg cells and larvae are both micron-sized. Aiming at the contour of the zebra fish larva, the traditional 3D printing technology cannot meet the requirement of high precision, so that the zebra fish microarray is processed by adopting the photoetching technology. Photolithography was performed on standard silicon wafers 10 cm in diameter. First, a zebrafish array is designed using software such as AutoCAD, as shown in fig. 3 (a). In order to meet the size requirement of the culture dish, the zebra fish larvae of which the outline shapes are two days after fertilization are placed in a square area of 6cm multiplied by 6cm according to the standard interval. Each larva model was designed based on existing larva size data, as shown in fig. 3 (b). The microarray petri dish arrangement was designed by standard photolithography using a Quintel Q-2001CT mask aligner to align SU-8100 silicon wafers. The depth of each groove was 300 μm (as shown in fig. 3 (c)). Then, an inverted mold was made by coating a Polydimethylsiloxane (PDMS) material on a silicon wafer. Specifically, PDMS and a curing agent were mixed uniformly in a ratio of 10:1, and the colloidal liquid was poured onto a photo-etched silicon wafer, and the silicon wafer was carefully peeled off after removing bubbles and allowing to stand at 65 ℃ for 4 hours. At this time, the larva shape on the PDMS mold was convex. And (2) melting agarose in a culture dish at the temperature of 90-95 ℃, then immersing the PDMS reverse mold into the culture dish, regulating the temperature to 4-38 ℃, standing for 30 minutes, and then taking down the reverse mold to obtain the microarray culture dish containing the zebra fish larva-shaped groove.

Alternatively, a culture dish manufactured based on the above design may be used to place a plurality of injection subjects in the culture dish, respectively, and wait for injection.

Optionally, a plurality of injection objects are placed in the culture dish, the image to be injected may refer to an image formed by the plurality of injection objects in the culture dish, that is, a complete image of the plurality of injection objects in the culture dish is displayed, the first target injection object may refer to a starting injection object in the image to be injected, the first target injection object may be preset, taking the culture dish shown in (a) in fig. 3 as an example, the first target injection object may refer to an injection object located in an upper left corner, that is, an injection object in a first row and a first column from left, currently, the first target injection object is not limited to the injection object in the upper left corner, and in actual application, the preset may be performed according to an actual situation.

Alternatively, based on the set first target injection subject, an image of the first target injection subject may be acquired.

S202, image processing is carried out on the image of the first target injection object, and the processed image of the first target injection object is obtained.

Generally, the acquired images of the first target injection object are all color images and are stored in RGB (U23), and first, the images of the first target injection object may be subjected to image processing to distinguish the first target injection object from the background and extract the outline of the first target injection object for subsequent operations.

And S203, determining a first target injection target position in the first target injection object according to the processed image of the first target injection object.

Generally, when injecting a medicine into an injection subject, a target injection position is determined first, and the injection is not performed randomly. Optionally, image recognition may be performed based on the processed image of the first target injection subject, so as to determine the first target injection target from the first target injection subject, where the first target injection target may be any part of the first target injection subject, and is determined specifically according to requirements.

And S204, controlling the injector to inject the first target injection target.

Optionally, based on the determined first target injection target site, the injector may be controlled to inject the drug into the first target injection target site to complete the injection of the first target injection subject.

Optionally, the overall implementation of the method is automatically completed based on the signal control of the upper computer, so that on one hand, the injection is accurate, and on the other hand, the injection efficiency is high.

In summary, the automated microinjection method provided in this embodiment includes: acquiring an image of a first target injection object in an image to be injected, wherein the image to be injected is an image of a plurality of injection objects in a culture dish acquired by image acquisition equipment, and the first target injection object is an initial injection object in the image to be injected; performing image processing on the image of the first target injection object to obtain a processed image of the first target injection object; determining a first target injection target position in the first target injection object according to the processed image of the first target injection object; and controlling the injector to inject the first target injection site. In the method, the first target injection target position can be identified from the first target injection object after image processing is carried out on the basis of the acquired image of the first target injection object, wherein the identification of the first target injection target position can be realized through an image identification algorithm, the identification efficiency and accuracy can be improved, and the injector can be controlled to inject on the basis of the identified first target injection target position, so that the automatic injection operation is completed, and the injection efficiency and the injection accuracy are improved.

FIG. 4 is a second schematic flow chart of an automated microinjection method according to an embodiment of the present application; fig. 5 is a schematic view of an injection trajectory provided in an embodiment of the present application. Optionally, in step S203, after controlling the injector to inject the first target injection target, the method of the present application may further include:

s401, according to a pre-planned injection track, an image of a second target injection object in the image to be injected is obtained, and the second target injection object is the next injection object of the first target injection object.

In some embodiments, after the injection of the first target injection object is completed, the culture dish may be controlled to move according to the pre-planned injection trajectory to obtain an image of the second target injection object in the image to be injected.

Alternatively, as shown in fig. 5, images of the injection subjects may be sequentially acquired and the injection subjects may be sequentially injected according to the injection trajectory as shown in fig. 5. After the first row completes the injection from top to bottom, the next row on the right can be shifted to, the next row completes the injection from bottom to top, and the images of the target injection object are sequentially acquired according to the injection track indicated by the arrow (namely, the injection is performed from top to bottom in the odd rows, and the injection is performed from bottom to top in the even rows).

It is worth mentioning that due to certain differences in body types and contours of zebra fish larvae, the success rate of simply moving the culture dish according to the standard spacing between the injection objects to obtain the position of the next injection object is low, and the requirement of injection is difficult to meet. In this embodiment, the image processing program can be designed by using more machine vision functions provided by the LabVIEW software, so that the next injection object can be accurately determined, and the accuracy of the determined target injection position in the next injection object can be improved.

S402, image processing is carried out on the image of the second target injection object, and the processed image of the second target injection object is obtained.

And S403, determining a second target injection target position in the second target injection object according to the processed image of the second target injection object.

S404, controlling the injector to inject the second target injection target position according to the second target injection target position, and stopping injecting until all injection objects in the culture dish are injected.

Alternatively, based on the acquired image of the second target injection subject, the same processing as the image of the first target injection subject may be taken, and finally the second target injection target in the second target injection subject is determined to be injected. The injection is terminated until all the injection subjects in the culture dish have completed.

Under ideal conditions, in a complete operation cycle, automatic microinjection of M × N injection objects can be realized, wherein the number of grooves for placing the injection objects in the culture dish can be designed according to different requirements, and the number of the injection objects to be placed in the manufactured culture dish can also be placed according to requirements.

As shown in fig. 5, M is set to 8 and N is set to 10 in this embodiment, that is, injection of injection subjects in 8 rows and 10 columns is realized, and therefore, 80 injection subjects can be injected in one cycle.

FIG. 6 is a third schematic flow chart of an automated microinjection method provided in an embodiment of the present application; optionally, in step S201, acquiring an image of the first target injection object in the image to be injected may include:

s601, identifying the position of the first target injection object in the image to be injected.

In some embodiments, based on the acquired image to be injected, the coordinates of each injection object in the image to be injected may be identified to determine therefrom the position of the first target injection object, where the identification of the coordinates may be achieved by a simple image recognition method.

S602, controlling the image acquisition equipment to acquire the image of the first target injection object according to the position of the first target injection object in the image to be injected.

Optionally, based on the determined position of the first target injection subject, the upper computer may control the image acquisition device to acquire an image of the first target injection subject.

FIG. 7 is a fourth schematic flow chart of an automated microinjection method provided by an embodiment of the present application; in step S602, controlling the image capturing device to capture an image of the first target injection object according to the position of the first target injection object in the image to be injected may include:

s701, controlling the positioning platform to move according to the position of the first target injection object so as to move the first target injection object in the culture dish to the view of the image acquisition equipment.

Optionally, based on the determined position of the first target injection object, the upper computer may send the position information to the positioning platform to control the positioning platform to move the culture dish placed thereon, so as to move the first target injection object in the culture dish to a position under the field of view of the image acquisition device, and optionally, to a position under a biological microscope in the image acquisition device.

S702, acquiring the image of the first target injection object, which is acquired and sent by the image acquisition equipment according to the position of the first target injection object.

Alternatively, the image capturing device may capture an image of the first target injection object under the biological microscope by using a high-definition camera, and generally, when the first target injection object is moved to the visual field of the image capturing device, the first target injection object is placed at the midpoint of the visual field as much as possible, so that the image of the first target injection object captured by the image capturing device can be clearer and more accurate.

Optionally, the image acquisition device may send the acquired image of the first target injection object to an upper computer, wherein the upper computer may further include a display, and the image sent to the upper computer may be displayed in the display at the same time, so as to facilitate viewing by a user.

Optionally, in step S203, before determining the first target injection target in the first target injection subject according to the processed image of the first target injection subject, the method of the present application may further include: and if the culture dish is judged to have the inclination, performing inclination correction on the processed image of the first target injection object to obtain a corrected image of the first target injection object.

In some embodiments, if the culture dish placed on the positioning platform is inclined, the identification of the first target injection target in the first target injection object may be affected, so that when the culture dish is judged to be inclined, the acquired processed image of the first target injection object may be subjected to inclination correction.

FIG. 8 is a fifth schematic flow chart of an automated microinjection method provided by an embodiment of the present application; fig. 9 is a schematic view of image correction according to an embodiment of the present application. Optionally, in the above step, performing tilt correction on the processed image of the first target injection object to obtain a corrected image of the first target injection object, may include:

s801, determining the position of a preset edge in the processed image of the first target injection object.

In general, the processed first target injection object image obtained after image processing contains less and clear information, and the efficiency and speed of performing tilt correction on the image are high.

In the previous embodiments, taking the injection object as the zebra fish larva as an example, as shown in fig. 9, the back of the zebra fish larva may be approximated to a straight line, and the image scanning may be performed from left to right to determine the position of the preset edge in the processed image of the first target injection object, where the preset edge may refer to the straight line where the back is located.

S802, determining the inclination angle according to the position of the preset edge.

Based on the determined position of the preset edge, an included angle between the preset edge and the X coordinate axis can be obtained as an inclination angle alpha, so that the angle of the inclination angle is obtained.

And S803, performing coordinate transformation on each vertex in the processed image of the first target injection object according to the inclination angle and the transformation matrix to obtain a corrected image of the first target injection object.

Alternatively, when the tilt correction is performed on the image, it is necessary to rotate the processed image of the first target injection subject counterclockwise by θ ═ 90 ° - α to perform the correction.

As shown in fig. 9, it is assumed that coordinates (x, y) are target points in the processed image of the first target injection subject, and (x ', y') are corrected target point coordinates. Point (Cx, Cy) is the center of rotation (lowest point in y-axis coordinates in the larval image after scanning). After rotating counterclockwise by an angle θ, (x ', y') can be calculated by the following formula:

alternatively, all vertices in the processed image of the first target injection subject may be transformed simultaneously, and the above may be expanded as:

through the transformation, the coordinates of each vertex in the processed image of the first target injection object are converted, and the corrected image of the first target injection object is obtained.

Alternatively, in step S203, determining the first target injection target in the first target injection subject according to the processed image of the first target injection subject may include: and determining the first target injection target position in the first target injection object by adopting an identification algorithm corresponding to the first target injection target position according to the processed image of the first target injection object.

In some embodiments, different recognition algorithms can be designed for different injection target positions according to the characteristics of the image and the shape of the zebra fish larvae, so as to determine the target injection target position from the target injection object, in this embodiment, the yolk of the zebra fish is taken as the target first target injection target position for example.

Taking the image of the first target injection object in the vertical state after correction in fig. 9 as an example, the image can be scanned from right to left to obtain a waveform of the zebra fish outline, two waveforms with the highest peak values are collected and respectively judged as a head and a yolk, so as to obtain a coordinate range of the abdomen of the zebra fish, and the specific coordinates of the yolk are determined by calculating the middle position of the highest waveform.

And then, taking the eyes of the zebra fish as a target injection target for example, respectively scanning from right to left to obtain a first waveform, wherein the first waveform can be the waveform of the abdomen of the zebra fish, and then scanning from top to bottom to obtain a second waveform, the second waveform can be the waveform of the top of the head of the zebra fish, a rectangular area is determined by the first waveform and the second waveform, the eyes of the zebra fish can be located in the rectangular area, and further, the specific positions of the eyes can be more accurately identified in the rectangular area.

Corresponding recognition algorithms can be provided for different target injection sites to determine the location of the target injection site.

FIG. 10 is a sixth schematic flow chart of an automated microinjection method provided by an embodiment of the present application; optionally, in step S204, controlling the injector to inject the first target injection target may include:

s1001, according to the first target injection target position, controlling the positioning platform to move the first target position of the first target injection object to an injection position corresponding to the injector.

Optionally, based on the determined first target injection target position, the upper computer may send the first target injection target position to the positioning platform, so as to move the first target position of the first target injection object to an injection position corresponding to the injector by controlling the movement of the positioning platform, where the injection position may be a position where the injector is set in advance to perform drug injection. Meanwhile, the needle head of the injector can be moved to the injection position by controlling the displacement manipulator, wherein the injection position can be positioned near the first target injection target position, on one hand, the needle head of the injector needs to be controlled to be positioned at a proper height, the distance between the needle head and the first target injection target position is within a preset range, and on the other hand, the needle head is ensured to be positioned under the visual field of a microscope as far as possible.

S1002, sending a driving signal to the injection pump so as to drive the injector to inject to the first target injection target position through the injection pump.

Optionally, the host computer may send a driving signal to the injection pump to drive the injection pump and control the dosage of the injection medicine, so that the injection pump may drive the injector to inject into the first target injection target site to complete the injection operation.

Optionally, in step S1002, after sending a driving signal to the syringe pump to drive the syringe by the syringe pump to inject into the first target injection target, the method of the present application may further include: a syringe withdrawal signal is sent to the displacement operator to control withdrawal of the syringe from the first target injection site.

In some embodiments, after the injection of the first target injection target site in the first target injection subject is completed, the upper computer may further send a syringe exit signal to the displacement operator to exit the needle of the syringe from the first target injection target site by controlling the displacement operator and restore the syringe to the initial position to prepare for the injection work of the next injection subject.

Optionally, in step S202, performing image processing on the image of the first target injection object to obtain a processed image of the first target injection object, which may include: and carrying out image binarization processing and filtering processing on the image of the first target injection object to obtain a processed image of the first target injection object.

In order to distinguish the zebra fish from the background, the image binarization of the first target injection object is realized by adopting gray threshold transformation. Specifically, the threshold is set to T, pixels having a gradation value smaller than T are set to 0, and pixels having a gradation value larger than T are set to 1. Preliminary tests show that the effect is ideal when the threshold value is equal to 0.8-0.9 times the average gray value. Considering that some noise points, such as impurities in agarose, cell membranes of embryos remaining after incubation of zebra fish larvae and the like, still exist in the image at this time, the image needs to be filtered so as to remove small noise points in the image. Then, an erosion function is used for eliminating isolated pixels with higher brightness relative to the background in the image, so that the target contour is refined. And filling cavities or cracks inside particles in the image by adopting an expansion function, and expanding the target contour to ensure that the image boundary is more round. The most efficient region in the image is then selected, the contours extracted, and the image appropriately enlarged. And finally, obtaining a processed zebra fish larva image, namely obtaining a processed image of the first target injection object.

Fig. 11 is a schematic diagram of an automatic microinjection process according to an embodiment of the present application. The automated injection procedure implemented by the present application is described below by way of a complete example.

A batch of 2 day old zebrafish juvenile fish was cultured under laboratory conditions, awaiting microinjection. MS-222 (ethyl m-aminobenzoate methanesulfonate) diluted by distilled water is used for anesthetizing the zebra fish juvenile fish, and the anesthetized larvae are easy to manipulate. The larvae were then pipetted into the prepared petri dishes and sequentially placed into standard wells with a brush or terry. If the direction of the young fish is consistent with the direction of the groove of the microarray culture dish, the young fish can be easily pushed into the groove by the brush. Otherwise, the fish larvae would need to be turned upside down, which takes some time to ensure that the yolk is facing the needle. Another method is to use a pipette to re-suck and place the zebrafish juvenile fish while adjusting the direction. This arrangement of placement operations helps to increase the speed of automated injections. Next, the culture dish was placed on the positioning platform, the focus of the biomicroscope of the image acquisition device was adjusted to obtain a clear image, and the first juvenile fish in the upper left corner was moved into view. At this time, the coordinates returned by the positioning platform are recorded as the initial position. Simultaneously, the number of lines and columns of injections is set according to the number of larvae and the array of placements. In this example, the syringe is angled at 45 ° to the plane of the dish. The needle head is arranged at a proper position by adjusting the displacement manipulator where the injector is located, and micro-injection operations such as puncture and injection are waited to be executed.

An automated microinjection procedure for a complete batch of zebrafish is shown in fig. 11. The point a represents the target injection target (e.g. the first target injection target) of the target injection object determined by the method, the point b represents the position of the injector performing the injection operation (i.e. the injection position may be located on the extension line of the injection needle), and the horizontal line on the right side of the point b represents the needle head of the injector. In fig. 11(a), point a represents a first target injection target site of a first target injection subject on which a drug injection is currently to be performed, point b represents an injection site of the injector, and the positioning platform is driven to translate the point a of the first target injection target site of the first target injection subject to the point b for puncture injection by calculating the distance between the point a and the point b, or the injector is driven to move from the point b of the injection site to the point a of the first target injection target site of the first target injection subject by driving the displacement manipulator. As shown in fig. 11 (b). The syringe was driven using the displacement manipulator to move to point b and then the needle of the syringe was controlled to penetrate the yolk. To better reveal the penetration situation, there is no point b in fig. 11(c), and then the injection pump is driven and the injected dose is controlled, completing the injection operation. Thereafter, the needle is withdrawn by controlling the displacement operator, as shown in fig. 11 (d). Finally, the apparatus was returned to the original position, ready for moving the next zebrafish larva to the field of vision for microinjection (see fig. 11(e)), the moving distance and the injection trajectory being as shown in fig. 5. In this way, the entire injection operation task can be automatically completed.

The following describes apparatuses, devices, storage media, and the like for performing the automated microinjection method provided in the present application, and specific implementation processes and technical effects thereof are referred to above, and will not be described again below.

Fig. 12 is a schematic diagram of an automated microinjection apparatus according to an embodiment of the present application, which performs a function corresponding to the steps performed by the method described above. The device can be understood as the above-mentioned upper computer, or a computer, a controller, and the like, and as shown in fig. 12, the device may include: an acquisition module 120, a processing module 121, a determination module 122, and a control module 123;

the acquiring module 120 is configured to acquire an image of a first target injection object in an image to be injected, where the image to be injected is an image of a plurality of injection objects in a culture dish acquired by an image acquisition device, and the first target injection object is an initial injection object in the image to be injected;

the processing module 121 is configured to perform image processing on the image of the first target injection object to obtain a processed image of the first target injection object;

a determining module 122, configured to determine a first target injection target in the first target injection subject according to the processed image of the first target injection subject;

and the control module 123 is used for controlling the injector to inject the first target injection target.

Optionally, the obtaining module 120 is further configured to obtain an image of a second target injection object in the image to be injected according to a pre-planned injection trajectory, where the second target injection object is a next injection object of the first target injection object;

the processing module 121 is further configured to perform image processing on the image of the second target injection object to obtain a processed image of the second target injection object;

a determining module 122, further configured to determine a second target injection target in the second target injection subject according to the processed image of the second target injection subject;

the control module 123 is further configured to control the injector to inject the second target injection target location according to the second target injection target location, and stop injecting until all the injection subjects in the culture dish are injected.

Optionally, the obtaining module 120 is specifically configured for

Identifying a location of a first target injection subject in an image to be injected;

Optionally, the obtaining module 120 is specifically configured for

Controlling the positioning platform to move according to the position of the first target injection object so as to move the first target injection object in the culture dish to the visual field of the image acquisition equipment;

Optionally, the apparatus further comprises: a correction module;

and the correction module is used for performing inclination correction on the processed image of the first target injection object to obtain a corrected image of the first target injection object if the culture dish is judged to have the inclination.

Optionally, a correction module, in particular for

determining an inclination angle according to the position of the preset edge;

and performing coordinate transformation on each vertex in the processed image of the first target injection object according to the inclination angle and the transformation matrix to obtain a corrected image of the first target injection object.

Optionally, a determination module 122, in particular for

And determining the first target injection target position in the first target injection object by adopting an identification algorithm corresponding to the first target injection target position according to the processed image of the first target injection object.

Optionally, a control module 123, in particular for

Controlling the positioning platform to move the first target of the first target injection object to an injection position corresponding to the injector according to the first target injection target;

and sending a driving signal to the injection pump so as to drive the injector to inject to the first target injection site through the injection pump.

Optionally, a control module 123 for

A syringe withdrawal signal is sent to the displacement operator to control withdrawal of the syringe from the first target injection site.

Optionally, a processing module 121, in particular for

The above-mentioned apparatus is used for executing the method provided by the foregoing embodiment, and the implementation principle and technical effect are similar, which are not described herein again.

These above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

The modules may be connected or in communication with each other via a wired or wireless connection. The wired connection may include a metal cable, an optical cable, a hybrid cable, etc., or any combination thereof. The wireless connection may comprise a connection over a LAN, WAN, bluetooth, ZigBee, NFC, or the like, or any combination thereof. Two or more modules may be combined into a single module, and any one module may be divided into two or more units. It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to corresponding processes in the method embodiments, and are not described in detail in this application.

It should be noted that the above modules may be one or more integrated circuits configured to implement the above methods, for example: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. For another example, the modules may be integrated together and implemented in the form of a System-on-a-chip (SOC).

Fig. 13 is a schematic structural diagram of an electronic device according to an embodiment of the present application, where the electronic device may be the above-mentioned upper computer, or a computing device with a data processing function, such as a computer, a processor, and a controller.

The apparatus may include: a processor 801 and a memory 802.

The memory 802 is used for storing programs, and the processor 801 calls the programs stored in the memory 802 to execute the above-mentioned method embodiments. The specific implementation and technical effects are similar, and are not described herein again.

Wherein the memory 802 stores program code that, when executed by the processor 801, causes the processor 801 to perform various steps in methods according to various exemplary embodiments of the present application described in the "exemplary methods" section above in this description.

The Processor 801 may be a general-purpose Processor, such as a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, and may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present Application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in a processor.

Memory 802, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The Memory may include at least one type of storage medium, and may include, for example, a flash Memory, a hard disk, a multimedia card, a card-type Memory, a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Programmable Read Only Memory (PROM), a Read Only Memory (ROM), a charged Erasable Programmable Read Only Memory (EEPROM), a magnetic Memory, a magnetic disk, an optical disk, and so on. The memory is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such. The memory 802 in the embodiments of the present application may also be circuitry or any other device capable of performing a storage function for storing program instructions and/or data.

Optionally, the present application also provides a program product, such as a computer readable storage medium, comprising a program which, when being executed by a processor, is adapted to carry out the above-mentioned method embodiments.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.

The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to perform some steps of the methods according to the embodiments of the present application. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

Claims

1. An automated microinjection method, applied to an upper computer in an automated microinjection system, the automated microinjection system comprising: the device comprises image acquisition equipment, a culture dish, an injector and the upper computer; the method comprises the following steps:

and controlling the injector to inject the first target injection target.

2. The method of claim 1, wherein after the controlling the syringe to inject the first target injection target, the method further comprises:

3. The method of claim 1, wherein the acquiring an image of a first target injection subject in the image to be injected comprises:

4. The method of claim 3, wherein the system further comprises: the culture dish is arranged on the positioning platform;

5. The method according to any one of claims 1-4, wherein before determining the first target injection target in the first target injection subject based on the processed image of the first target injection subject, the method further comprises:

6. The method of claim 5, wherein the tilt correcting the processed image of the first target injection subject to obtain a corrected image of the first target injection subject comprises:

determining an inclination angle according to the position of the preset edge;

7. The method according to claim 1, wherein the determining a first target injection target in the first target injection subject from the processed image of the first target injection subject comprises:

8. The method of claim 4, wherein the system further comprises: an injection pump; the controlling the injector to inject the first target injection target includes:

9. The method of claim 8, wherein after sending a drive signal to the syringe pump to drive the syringe by the syringe pump to inject the first target injection site, the method further comprises:

10. The method according to any one of claims 1 to 4, wherein the image processing the image of the first target injection subject to obtain a processed image of the first target injection subject comprises:

11. An automated microinjection apparatus, applied to an upper computer of an automated microinjection system, the automated microinjection system comprising: the device comprises image acquisition equipment, a culture dish, an injector and the upper computer; the device comprises: the device comprises an acquisition module, a processing module, a determination module and a control module;

12. An automated microinjection system, comprising: the device comprises image acquisition equipment, a positioning platform, a culture dish, an injector, an injection pump, a displacement manipulator and an upper computer;

the upper computer is in communication with the image acquisition device, the positioning platform, the syringe pump, and the displacement manipulator, respectively, and is configured to perform the steps of any of the methods of claims 1-10.

13. An electronic device, comprising: a processor, a storage medium and a bus, the storage medium storing program instructions executable by the processor, the processor and the storage medium communicating via the bus when the electronic device is running, the processor executing the program instructions to perform the steps of the method according to any one of claims 1 to 10 when executed.

14. A computer-readable storage medium, characterized in that the storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 10.