WO2023142215A1

WO2023142215A1 - Method for automatically picking up nanowires by micro-nano operation robot on basis of dynamic motion primitives

Info

Publication number: WO2023142215A1
Application number: PCT/CN2022/078316
Authority: WO
Inventors: 杨湛; 韩驰瑞; 巢沛栋; 房梁; 张略; 孙立宁
Original assignee: 苏州大学
Priority date: 2022-01-27
Filing date: 2022-02-28
Publication date: 2023-08-03
Also published as: CN114549269A

Abstract

The present invention relates to a method for automatically picking up nanowires by a micro-nano operation robot on basis of dynamic motion primitives. The method comprises: acquiring a demonstration trajectory of a micro-nano operation robot; determining whether there is motion-in-depth information on the demonstration trajectory of the micro-nano operation robot, wherein if the determination result is "yes", the trajectory is motion in depth, and if the determination result is "no", the trajectory is motion in plane, and dividing the demonstration trajectory of the micro-nano operation robot into a plurality of simple meta-tasks, wherein a division criterion for the meta-tasks is that division is performed according to whether motion in depth is performed; and coding the meta-tasks and establishing a meta-task library, and the micro-nano operation robot calling the meta-tasks in the meta-task library and moving according to trajectories of the meta-tasks. In the present invention, with regard to the operating environment and motion features of a micro-nano operation robot, a complex operation trajectory is divided into a plurality of simple meta-task trajectories, thereby greatly reducing the learning difficulty for the micro-nano operation robot. The method is applicable to robot operation in a micro environment.

Description

A method for automatically picking up nanowires by a micro-nano manipulation robot based on dynamic motion primitives

technical field

The invention relates to the technical field of robot automation, in particular to a method for automatically picking up nanowires by a micro-nano operating robot based on dynamic motion primitives.

Background technique

In a microscopic environment, use a scanning electron microscope (Scanning Electron Microscope, SEM) to build a micro-nano operating robot or operating system to complete the picking of nanowires. These robots are assembled in the cavity of the SEM, and their overall dimensions are very small, usually on the order of a few centimeters. The operating objects of the micro-nano manipulator are usually at the micron and nanometer level. In order to ensure normal operation, the motion accuracy needs to be guaranteed at the nanometer level. Using the SEM as a vision sensor, using a vision-based contact detection method, the contact between the probe tip and the substrate can be detected, and the relative vertical position of the probe tip and the nano-object to be manipulated can also be determined, which can be based on the SEM image. Establish a visual servo control system with a feed-forward control controller for the closed-loop control of multiple micro-nano manipulators, which can bring the probe to the target position accurately, through this visual feedback-based servo control method, some nanometer Operate robots for automated manipulation tasks.

At present, the design scheme of macro-robot automation mainly adopts the trajectory-based demonstration learning method, which generally needs to segment the trajectory to improve the learning efficiency of the trajectory. Usually, the automatic segmentation trajectory is mainly divided into supervised segmentation and unsupervised methods. Among them, there is a method in the supervised segmentation method It is a segmentation technology based on shape matching, contacting the motion trajectory model trained in advance, and segmenting the teaching trajectory through the geometric shape and other features between the trajectory; one of the segmentation demonstration trajectory in the unsupervised segmentation method is to divide the robot end effector The contact keyframe with the operating object is used as the split point to complete the split of the track. However, the above-mentioned general demonstration motion trajectory segmentation methods are basically used in the field of macro-robots. There are certain differences between their motion characteristics and physical changes with the operating objects and micro-nano manipulators. In the environment, the objects to be manipulated and information to be collected require the help of a scanning electron microscope, and general segmentation methods are not suitable for direct application to micro-nano manipulation robots.

Contents of the invention

Therefore, the technical problem to be solved by the present invention is to overcome the problems existing in the prior art, and to propose a method for automatically picking up nanowires by a micro-nano manipulating robot based on dynamic motion primitives.

In order to solve the above-mentioned technical problems, the present invention provides a method for automatically picking up nanowires by a micro-nano operating robot based on dynamic motion primitives, including the following steps:

S10: Obtain the demonstration track of the operator operating the micro-nano operation robot to complete the nanowire picking task;

S20: Using the evaluation function to judge whether there is depth motion information on the demo trajectory of the micro-nano manipulator robot, if the judgment result is yes, then this section of trajectory is a depth movement, if the judgment result is no, then this section of trajectory is a planar movement, And the demonstration trajectory of the micro-nano manipulator is divided into a plurality of simple meta-tasks, wherein the division criterion of the meta-tasks is divided according to whether to carry out deep motion;

S30: Encode the meta-task and establish a meta-task library. When there is a nanowire automatic picking task, the micro-nano operation robot calls the meta-task in the meta-task library, moves according to the trajectory of the meta-task, and completes Nanowire automatic pickup tasks.

In one embodiment of the present invention, in S10, obtaining the demo track of the micro-nano manipulator includes:

Obtaining the movement video of the micro-nano operation robot, and intercepting multiple frames of images on the movement video;

A low-pass Gaussian filter is selected to filter each frame of image, and the filtered image is etched and binarized successively to obtain the outlines of the cantilever beam and the nanowire in the image.

In one embodiment of the present invention, in S20, using the evaluation function to judge whether there is depth motion information on the demo trajectory of the micro-nano manipulator includes:

The Tenengrad gradient function is used as the evaluation function, and the Tenengrad gradient function is used to calculate the sharpness changes of the cantilever beam when it is stationary and in depth motion, and the polynomial regression model is used for characterization, and the corresponding slope is obtained to judge whether the cantilever beam is in depth. Motion or stillness.

In one embodiment of the present invention, in S20, the demonstration trajectory of the micro-nano manipulator robot is divided into a plurality of simple meta-tasks, wherein the division criterion of the meta-tasks includes:

Segmenting a segment of the track for deep motion on the demo track, defining the motion as an ascending element task, before performing the ascending element task, moving the cantilever beam directly below the nanowire, and defining the motion as a positioning element task;

After controlling the rise of the cantilever beam, increasing the short-distance swing motion of the cantilever beam in the Y direction, and defining this motion as a contact judgment meta-task, judging the contact state between the cantilever beam and the nanowire according to the contact judgment meta-task , if the contact state of the cantilever beam and the nanowire is line contact, then control the cantilever beam to pick up the nanowire, and define this motion as a separation element task, if the contact state of the cantilever beam and the nanowire is point contact, Then define the contact repair meta-task to repair the point contact until the contact state between the cantilever beam and the nanowire is line contact.

In an embodiment of the present invention, there is at least one contact repair meta-task.

In one embodiment of the present invention, in S30, encoding the meta-task includes:

The trajectory of the meta-task is filtered, and Platts dynamic time warping is used to perform trajectory alignment processing on the trajectory of the filtered meta-task, and multiple trajectories are normalized to the same time step;

A Gaussian mixture model is used to characterize the motion characteristics of the meta-task after trajectory alignment processing, and a mixture of Gaussian regression generalization is used to generate the meta-task demonstration trajectory;

Meta-task demonstration trajectories are encoded using dynamic motion primitives.

In an embodiment of the present invention, the meta-task includes a divided meta-task and an introduced empty walk demonstration meta-task.

In one embodiment of the present invention, the formula of the Platts dynamic time warping is as follows:

in,

is the i-th track to be aligned, i=1,2,...,m,

is the time regularization matrix.

In addition, the present invention also provides a computer device, including a memory, a processor, and a computer program stored on the memory and operable on the processor, and the processor implements the steps of the above-mentioned method when executing the program.

Moreover, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the above-mentioned method are realized.

The above technical solution of the present invention has the following advantages compared with the prior art:

The present invention proposes a method for automatically picking up nanowires by a micro-nano operating robot based on dynamic motion primitives. Aiming at the operating environment and motion characteristics of the micro-nano operating robot, it establishes a criterion for dividing element tasks, and divides complex operating trajectories into multiple A simple meta-task trajectory greatly reduces the learning difficulty of micro-nano manipulator robots, and is suitable for robot operations in micro-environment.

Description of drawings

In order to make the content of the present invention easier to understand clearly, the present invention will be described in further detail below according to the specific embodiments of the present invention in conjunction with the accompanying drawings.

Fig. 1 is a schematic flowchart of the method for automatically picking up nanowires by a micro-nano manipulating robot based on dynamic motion primitives in the present invention.

Fig. 2 is the sharpness change and polynomial regression characterization results of the AFM cantilever beam of the present invention in different states, wherein Fig. 2 (a) is the sharpness change and polynomial regression characterization results of the AFM cantilever beam at rest; Fig. 2 (b) is the AFM Characterization results of sharpness change and polynomial regression during the depth motion of the cantilever beam.

Fig. 3 is three contact states of the nanowire of the present invention and the AFM cantilever beam.

Fig. 4 is a motion track diagram of the demonstration end effector of the micro-nano manipulation robot picking up nanowires of the present invention.

Fig. 5 is a schematic diagram of the meta-tasks of the depth motion divided by the present invention.

Fig. 6 is a schematic diagram illustrating the division of a complete nanowire pick-up task according to the present invention.

Figure 7(a) is the original trajectory diagram of the positioning meta-task, and Figure 7(b) is the original trajectory diagram of the positioning meta-task after filtering.

Fig. 8 is an effect diagram of aligning multiple positioning unit task trajectories in the present invention, the left side is before alignment, and the right side is after alignment.

Fig. 9 is the meta-task demonstration trajectory after learning in the present invention, the upper half of which is GMM characterizing the motion characteristics of positioning meta-task, and the lower half is the most stable trajectory graph generated by GMR generalization.

Fig. 10 is a trajectory diagram of the DMP recurring positioning meta-task demonstration in the present invention.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, so that those skilled in the art can better understand the present invention and implement it, but the examples given are not intended to limit the present invention.

Please refer to shown in Fig. 1, present embodiment provides a kind of micro-nano operation robot based on dynamic motion primitive to pick up nanowire method automatically, comprises the following steps:

In a method for automatically picking up nanowires by a micro-nano operation robot based on dynamic motion primitives disclosed in the present invention, for S10 of the above-mentioned embodiment, the micro-nano operation robot is built in the SEM, and the end effector is an atomic force microscope (Atom Force Microscope). Microscope, AFM) probe, but what can be seen in the SEM image is the cantilever beam of the AFM probe. The operator manipulates the control handle and the controller to control the movement of the micro-nano manipulating robot. The cantilever beam of the AFM probe can be recognized through the image, and the trajectory of the cantilever beam of the probe can be calculated to obtain the demonstration trajectory of the end effector. The main processing process is: obtain the motion video of the operation demonstration, intercept multiple frames of images on the motion video, filter each frame of image with a low-pass Gaussian filter, and use the morphological operation on the filtered image to convert the finer nano The wires are etched away so that the probes can also be identified when they overlap the nanowires. The image was then binarized using automatic histogram thresholding, enabling the profile of the probe cantilever and nanowires to be captured in the image. Due to the disorder of the nanowires, its profile changes greatly, but the profile of the probe cantilever does not change significantly, so the cantilever of the probe can be identified by polygonal approximation. The shape of the cantilever is very similar to an isosceles trapezoid, so choose the X coordinate of the short side of the cantilever as the X coordinate of the cantilever, and the Y coordinate of the centroid of the profile as the Y coordinate of the cantilever.

In the method for automatically picking up nanowires by a micro-nano manipulating robot based on dynamic motion primitives disclosed in the present invention, for S20 of the above-mentioned embodiment, since the image captured by the above-mentioned SEM is a two-dimensional grayscale image, the micro-nano manipulating robot The depth direction, that is, the Z-axis motion information cannot be accurately obtained, so the method adopted in the present invention is to analyze the Z-axis motion of the end effector AFM cantilever by judging the change of the definition of the cantilever beam. Since the range captured by the center of the field of view of the SEM is limited, and the size of the cantilever beam is far beyond the shooting range of the SEM, the area of the cantilever beam that can be seen during the operation is always changing, so it is not easy to use the whole picture. The definition change of the AFM cantilever beam area is calculated, so the present invention expands according to the matched AFM tip area, and selects a fixed size area to calculate the definition change of the AFM cantilever beam, that is, the present invention uses the Tenengrad gradient function as the evaluation Clarity evaluation function. The Tenengrad gradient function uses the Sobel operator to extract the gradient value of the horizontal and vertical directions of the image pixel, which is defined as the sum of the squares of the gradient of the pixel, and sets the sensitivity of a threshold adjustment function for the gradient. The expression is:

Among them, T is a given edge detection threshold, G(x, y) is the gradient at the pixel point (x, y), and Gx and Gy are the convolution of Soble horizontal and vertical edge detection operators at the pixel point respectively.

Moreover, the SEM image is an image containing two-dimensional information, and the depth information of the Z axis cannot be directly obtained. And researchers can't judge the relative information of the Z axis when manipulating the micro-nano robot, so the plane operation and the depth operation are separated, which is a good segmentation point, and the segmented segments can also be easily given Reasonable semantic interpretation. First, by analyzing the turning point between the depth motion and the plane motion, and using it as the key point of segmentation, the depth motion is segmented first. The plane movement occurs on the XY plane, and the XY trajectory changes drastically at this time, while the depth movement occurs on the Z axis. At this time, the XY trajectory does not change theoretically, but because the three driving mechanisms of the micro-nano manipulator are connected together Yes, each axis alone will drive other axes to vibrate when it moves, and building a robot operating system in the SEM electron microscope will interfere with the SEM image. These two reasons will make the AFM cantilever move in the Z-axis, XY plane position changes slightly. After many experiments and measurements, the shaking range of the AFM cantilever is within 15 pixels. Therefore, it can be determined whether the cantilever beam is in a state of deep motion according to the variation range of the X-axis and Y-axis trajectories. However, this also requires distinguishing between the Z-axis motion of the cantilever and the true rest state. When the AFM cantilever beam moves in the Z axis, the distance between its upper surface and the SEM objective lens will change, resulting in a change in the definition of the cantilever beam in the SEM image. Therefore, by evaluating the clarity through the above-mentioned Tenengrad gradient function, it is possible to calculate the changes in the clarity of the AFM cantilever when it is stationary and in deep motion, and use polynomial regression to characterize the two changes, such as As shown in Fig. 2, for example, the |k| of the definition evaluation change of the cantilever beam at rest is less than 0.003, while the |k| Perform Z-axis motion or stand still.

After dividing the depth motion and plane motion, the trajectory of plane motion is often more complicated, which needs to be further divided. Due to the particularity of the micro-nano operation, the present invention divides a segment of the track for deep motion on the demonstration track, and defines the motion as an ascending meta-task. Before performing the ascending meta-task, the cantilever beam is moved directly below the nanowire , define this movement as a positioning element task; after controlling the cantilever beam to rise, what you can see from the image is the top view of the nanowire and the cantilever beam, and there is an overlapping state between them, but their positional relationship in the Z-axis direction we If it cannot be known, it is impossible to accurately judge whether it is in contact with the nanowire. Therefore, the present invention increases the short-distance swing motion of the cantilever beam in the Y direction, and defines this motion as a contact judgment meta-task (a priori meta-task). The contact judgment meta-task judges the contact state between the cantilever beam and the nanowire. After the contact judgment, the contact state between the nanowire and the cantilever beam can be obtained, which is divided into three contact states: line contact, point contact a, point contact Contact b, as shown in Figure 3. If the contact state of the cantilever beam and the nanowire is line contact, then control the cantilever beam to pick up the nanowire, and define the motion as a separation element task; if the contact state of the cantilever beam and the nanowire is point contact, this When the contact force between the two is very small, it may not be enough to pull (pick up) the nanowire from the substrate. Corresponding correction operations are required for these two kinds of point contacts, so the present invention defines these two contact repairs as two priori meta-tasks according to the difference between the two types of line contacts, which are respectively contact repair meta-task a and contact repair unit Task b. However, contact judgment must be continued after contact repair, so contact repair is an action that occurs between two contact judgments. Based on this temporal feature, the present invention can further divide these two meta-tasks, so as to obtain multiple relatively simple meta-tasks, which greatly reduces the difficulty for robots to learn these meta-tasks.

In a method for automatically picking up nanowires by a micro-nano operating robot based on dynamic motion primitives disclosed in the present invention, for S30 in the above embodiment, after the meta-tasks are divided, the present invention encodes the trajectories of these meta-tasks, so that the micro-nano Manipulating the robot learns and reproduces the motion of these meta-tasks. However, the divided meta-tasks may not be representative enough, so the present invention also introduces the empty-walk demonstration meta-task of the micro-nano manipulator to increase the teaching ability of the meta-task demonstration.

In the specific coding, since the SEM image will be disturbed by various factors and the jitter of the micro-nano manipulator, the calculated movement trajectory of the AFM cantilever beam will have many small-scale jitters, so the moving average filter can be used to analyze the movement trajectory. filter processing. There may be differences in the time length between multiple trajectories of the same meta-task divided, but the follow-up processing needs trajectories with the same time length, so the present invention adopts Procrustes Dynamic Time Warping (PDTW) ) for trajectory alignment. PDTW is modified on the basis of the original dynamic time warping. It can align multiple trajectories at the same time, and regularize multiple trajectories to the same time step. By minimizing:

in,

is the i-th track to be aligned, i=1,2,...,m,

is the time regularization matrix.

Gaussian Mix Model (GMM) is a typical demonstration learning algorithm, and the trajectory learning model based on GMM can well maintain the shape characteristics of the demonstration trajectory of the AFM cantilever beam. The present invention uses K-means clustering method and Bayesian Information Criterion (BIC) criterion to determine the initial value of parameter, and adopts EM algorithm to carry out parameter estimation. After the parameters are calculated, Gaussian Mix Regression (GMR) is used to perform regression processing on the represented GMM to generate the most reliable trajectory for each meta-task. And the most stable trajectory generated by GMR is encoded using dynamic motion primitive DMP. The dynamic motion primitive transforms an easy-to-understand simple attractor system into a desired attractor system using a learnable forcing function term, which can work for point attractors and limit cycle attractors of almost any complexity, Has good coordination and stability. In the method of the present invention, DMP is used to encode the optimal meta-task trajectory, and a meta-task library is established. The subsequent automatic execution task is that the micro-nano operation robot calls the meta-task DMP corresponding to the meta-task library, which is generated according to the DMP plan trajectory for movement. One of the one-dimensional DMP equations is:

Among them, the first half of the first formula is the derivation equation of the spring damping model. When α=4β, the trajectory will reach the target quickly and stably; f is the interference item. According to the teaching trajectory data, use local weighted regression to find the weight of the basis function ψi ωi, reproduces the shape of the teaching trajectory; the third formula is the canonical system, x will decay to 0 before the end of the motion, so as not to affect the position of the target, so DMP can reproduce the generalized GMR very accurately track.

According to the operating environment and motion characteristics of the micro-nano manipulating robot, the present invention establishes a criterion for dividing meta-tasks, and divides complex operation trajectories into a plurality of simple meta-task trajectories, which greatly reduces the learning difficulty of micro-nano manipulating robots, and is applicable to micro-manipulators. environment for robot operation.

The present invention uses the sharpness evaluation function to calculate the change of the sharpness of the end effector of the micro-nano manipulator robot during the deep motion, to analyze the movement of the micro-nano manipulator robot during the deep motion, and can distinguish between motion and static states.

The present invention clusters different trajectories of the same divided meta-task, and performs code learning for each type of trajectories, and uses different meta-task trajectories in different scenarios to better complete the operation task.

Compared with the traditional demonstration learning, the present invention uses the segmented simple meta-task trajectory as teaching, obtains the most stable teaching trajectory through a series of optimization processes, and then uses DMP to encode and learn this trajectory, greatly reducing The error between the trajectory generated by the DMP planning and the demonstration trajectory is minimized.

The content of the method for automatically picking up nanowires by a micro-nano manipulating robot based on dynamic motion primitives proposed by the present invention will be described in detail below through a specific implementation example.

First, the operator manually controls the handle to complete the task of picking up the nanowires, and obtains a demonstration motion video. For each frame of image intercepted from the motion video, there are many noise points. After comparison and verification, the low-pass Gaussian filter is used for processing, and then a circular nucleus with a size of 1 is selected to corrode the nanowires in the SEM image. Then calculate the pixel values of all pixels in the image. Since the background accounts for most of the area in the image, the pixel value with the largest ratio is counted as the threshold, and the binary segmentation is performed, and the AFM cantilever beam and the nanowire substrate are identified. Contour, the contour of the AFM cantilever can be identified by the difference between the contours, and the position of the AFM cantilever can be calculated. By identifying the position of the AFM cantilever beam in the continuous frame images, according to the time of the image frames, the continuous motion track of the XY plane of the micro-nano manipulator robot can be obtained, as shown in Figure 4.

Then, according to the division criterion of the defined meta-task, the demonstration trajectory of the operator manually picking up the nanowire is divided: firstly, it is divided according to whether the depth movement is performed, and it can be found that the XY axis trajectory during the period from 32s to 252s is basically unchanged , and the jitter does not exceed 15 pixels, and the Teneggrad function is used to calculate the sharpness change during this period, and then the polynomial regression model is used to characterize it, and the slope is 0.036, which is determined to be a deep movement during this period. This section is segmented out and named as rising meta-task according to its motion characteristics, as shown in Figure 5. Control the cantilever beam to move directly below the target nanowire before the ascending meta-task, which is the basis for the ascending meta-task. According to this motion characteristic, the present invention names this movement as the positioning meta-task, as shown in FIG. 6 . After the ascent meta task, it is necessary to judge the contact state of the nanowire and the AFM cantilever beam. At this time, the secondary electron image collected from the SEM shows the top view of the nanowire and the AFM cantilever beam. There is an overlapping state between them, but we cannot know their positional relationship in the Z-axis direction. Therefore, the short-distance swing motion of the cantilever beam in the Y direction can be increased. Through this motion, it is observed whether the nanowires follow the cantilever beam to move to judge what kind of contact state they are in. The present invention defines the motion as a contact judgment meta-task. It is a back-and-forth motion, and the trajectory is also special. Using the particularity of the contact judgment trajectory, it is divided from the demonstration trajectory. As shown in Figure 6, a total of three contact judgment meta-tasks are divided. Between the two contact judgment meta-tasks is the contact repair meta-task, which is divided according to this temporal feature, as shown in Figure 6. The result of the last contact judgment must be determined as wire contact. At this time, it is necessary to control the AFM cantilever beam to pull the nanowire from the substrate. This movement is defined as the separation element task in the present invention, as shown in FIG. 6 . In this way, the present invention divides a complete nanowire picking task demonstration into six simple meta-tasks, so that learning these six meta-tasks for the micro-nano manipulation robot is definitely much easier than learning a complex complete demonstration task.

Finally, after the meta-tasks are divided, it is necessary to let the micro-nano manipulator robot learn the motor skills of these meta-tasks, that is, to encode the meta-tasks, and first to filter the meta-task trajectories. Because the SEM image will be disturbed by various factors and the jitter of the nano-manipulator robot, the calculated movement trajectory of the AFM cantilever beam will have many small jitters. The moving average filter is used to filter the movement trajectory. The effect is shown in Fig. 7 in. The time length between the divided task trajectories of the same class may be different, but subsequent processing requires trajectories with the same time length, so Platts Dynamic Time Warping (PDTW) is used for trajectory alignment processing. The effect of aligning multiple positioning meta-task trajectories is shown in Figure 8. Then GMM is used to characterize the motion properties of the meta-task, and GMR generalization is adopted to generate the most stable meta-task demonstration trajectory, as shown in Fig. 9. Finally, DMP is used to encode the optimal demonstration trajectory generated by GMR. DMP can not only reproduce the demonstration trajectory very well, but also can generalize the learning demonstration trajectory. In the actual trajectory planning, the starting point and the target point will not change. Affecting the generated trajectory, the effect of DMP reproducing the positioning meta-task demonstration trajectory is shown in Figure 10.

Corresponding to the above method embodiment, the embodiment of the present invention also provides a computer device, including:

memory for storing computer programs;

A processor, which is used to implement the steps of the above-mentioned method for automatically picking up nanowires by a micro-nano manipulating robot based on dynamic motion primitives when executing a computer program.

In the embodiment of the present invention, the processor may be a central processing unit (Central Processing Unit, CPU), an application-specific integrated circuit, a digital signal processor, a field programmable gate array, or other programmable logic devices.

The processor can call the program stored in the memory. Specifically, the processor can execute the operations in the embodiment of the method for automatically picking up nanowires by a micro-nano manipulation robot based on dynamic motion primitives.

The memory is used to store one or more programs, the programs may include program codes, and the program codes include computer operation instructions.

In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device or other volatile solid-state storage devices.

Corresponding to the above method embodiment, the embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned dynamic motion primitive-based micro Steps in the method for automatically picking up nanowires by a nanomanipulator robot.

Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowcharts and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

Apparently, the above-mentioned embodiments are only examples for clear description, and are not intended to limit the implementation. For those of ordinary skill in the art, on the basis of the above description, other changes or changes in various forms can also be made. It is not necessary and impossible to exhaustively list all the implementation manners here. However, the obvious changes or changes derived therefrom are still within the scope of protection of the present invention.

Claims

A method for automatically picking up nanowires by a micro-nano operating robot based on dynamic motion primitives, characterized in that it includes the following steps:

S10: Obtain the demonstration track of the operator operating the micro-nano operation robot to complete the nanowire picking task;

S20: Using the evaluation function to judge whether there is depth motion information on the demo trajectory of the micro-nano manipulator robot, if the judgment result is yes, then this section of trajectory is a depth movement, if the judgment result is no, then this section of trajectory is a planar movement, And the demonstration trajectory of the micro-nano manipulator is divided into a plurality of simple meta-tasks, wherein the division criterion of the meta-tasks is divided according to whether to carry out deep motion;

S30: Encode the meta-task and establish a meta-task library. When there is a nanowire automatic picking task, the micro-nano operation robot calls the meta-task in the meta-task library, moves according to the trajectory of the meta-task, and completes Nanowire automatic pickup tasks.
The method for automatically picking up nanowires by a micro-nano operating robot based on dynamic motion primitives according to claim 1, wherein, in S10, obtaining the demonstration trajectory of the micro-nano operating robot includes:

Obtaining the movement video of the micro-nano operation robot, and intercepting multiple frames of images on the movement video;

A low-pass Gaussian filter is selected to filter each frame of image, and the filtered image is etched and binarized successively to obtain the outlines of the cantilever beam and the nanowire in the image.
The method for automatically picking up nanowires by a micro-nano manipulating robot based on dynamic motion primitives according to claim 2, wherein in S20, an evaluation function is used to judge whether there is depth motion information on the demonstration track of the micro-nano manipulating robot include:

The Tenengrad gradient function is used as the evaluation function, and the Tenengrad gradient function is used to calculate the sharpness changes of the cantilever beam when it is stationary and in depth motion, and the polynomial regression model is used for characterization, and the corresponding slope is obtained to judge whether the cantilever beam is in depth. Motion or stillness.
The method for automatically picking up nanowires by a micro-nano operating robot based on dynamic motion primitives according to claim 3, wherein in S20, the demonstration trajectory of the micro-nano operating robot is divided into a plurality of simple meta-tasks, Among them, the division criteria of meta-tasks are divided according to whether to carry out in-depth motion, including:

Segmenting a segment of the track for deep motion on the demo track, defining the motion as an ascending element task, before performing the ascending element task, moving the cantilever beam directly below the nanowire, and defining the motion as a positioning element task;

After controlling the rise of the cantilever beam, increasing the short-distance swing motion of the cantilever beam in the Y direction, and defining this motion as a contact judgment meta-task, judging the contact state between the cantilever beam and the nanowire according to the contact judgment meta-task , if the contact state of the cantilever beam and the nanowire is line contact, then control the cantilever beam to pick up the nanowire, and define this motion as a separation element task, if the contact state of the cantilever beam and the nanowire is point contact, Then define the contact repair meta-task to repair the point contact until the contact state between the cantilever beam and the nanowire is line contact.
The method for automatically picking up nanowires by a micro-nano manipulating robot based on dynamic motion primitives according to claim 4, wherein there is at least one contact repair element task.
The method for automatically picking up nanowires by a micro-nano operating robot based on dynamic motion primitives according to claim 1, wherein in S30, encoding the meta-task includes:

The trajectory of the meta-task is filtered, and Platts dynamic time warping is used to perform trajectory alignment processing on the trajectory of the filtered meta-task, and multiple trajectories are normalized to the same time step;

A Gaussian mixture model is used to characterize the motion characteristics of the meta-task after trajectory alignment processing, and a mixture of Gaussian regression generalization is used to generate a meta-task demonstration trajectory;

Meta-task demonstration trajectories are encoded using dynamic motion primitives.
The method for automatically picking up nanowires by a micro-nano manipulating robot based on dynamic motion primitives according to claim 6, wherein the meta-tasks include divided meta-tasks and introduced empty-walk demonstration meta-tasks.
The method for automatically picking up nanowires by a micro-nano operating robot based on dynamic motion primitives according to claim 6, wherein the formula of Platts dynamic time warping is as follows:

in,
is the i-th track to be aligned, i=1,2,...,m,
is the time regularization matrix.
A computer device, comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, characterized in that, when the processor executes the program, it realizes any one of claims 1 to 8 method steps.
A computer-readable storage medium, on which a computer program is stored, wherein the program implements the steps of the method according to any one of claims 1 to 8 when the program is executed by a processor.