WO2024022459A1

WO2024022459A1 - Positioning alignment method and apparatus for micro mechanical arm, and electronic device

Info

Publication number: WO2024022459A1
Application number: PCT/CN2023/109673
Authority: WO
Inventors: 宋亮; 卓清山; 陈聪
Original assignee: 中欧智薇(上海)机器人有限公司
Priority date: 2022-07-29
Filing date: 2023-07-27
Publication date: 2024-02-01
Also published as: CN115153855A; CN115153855B

Abstract

Disclosed in embodiments of the present invention are a positioning alignment method and apparatus for a micro mechanical arm, and an electronic device. The method comprises: determining an initial pose of the micro mechanical arm in a folded state and center positions of a plurality of targets, the initial pose comprising an initial position and an initial posture; determining a target area of the micro mechanical arm in a motion plane according to the initial position and the center positions of the plurality of targets; controlling the micro mechanical arm to move from the initial position to a preset position of the target area, and fixing the micro mechanical arm at the preset position; and controlling the micro mechanical arm to be converted from the folded state to an unfolded state by taking the preset position as a base point, sequentially moving to the center positions of reachable targets, and carrying out alignment on the reachable targets. The present invention solves the technical problem that due to the fact that a micro mechanical arm in the prior art needs to be frequently moved and fixed manually for positioning, the application time and the risk are increased, and achieves the technical effects of reducing the number of moving and fixing times and reducing the application time and risk.

Description

A positioning and alignment method, device and electronic equipment for a micro robotic arm

Technical field

The present invention relates to the field of micro-robot positioning technology, and in particular to a positioning and alignment method, device and electronic equipment for a micro-robot.

Background technique

Current surgical robot solutions generally use large-scale robotic arms. Large-scale robotic arms have the advantages of flexible joints, high degrees of freedom, and wide range of motion. However, they are expensive to use and complex to operate, which can easily cause delays in surgical time and are not conducive to a large number of surgeries using robots. universal. The emergence of micro-robots using local positioning technology can make up for the shortcomings of complex and high-cost use of large robotic arms, but at the same time it has the advantage of accurate positioning accuracy. Compared with large-scale robotic arms, micro-manipulators are smaller in size and are generally used in local area positioning. For example, in the field of neurosurgery where the positioning area is relatively small and the positioning accuracy is high. Taking the application of micro-robots in traditional orthopedic surgery scenarios as an example, the user needs to first move the micro-robot to a position close to the surgical area, then fix the micro-robot, and then start the micro-robot for fine positioning to achieve precise positioning. However, in such application scenarios, if there are multiple positioning positions or targets, frequent manual movement and fixation is required before starting the micro-robot, which increases manual operations, risks, and operation time, ultimately limiting the application. scope.

No effective solution has yet been proposed for the above problems.

Contents of the invention

Embodiments of the present invention provide a positioning and alignment method, device and electronic equipment for a micro-robot to at least solve the problem in related technologies that the positioning of the micro-robot requires frequent manual movement and fixation, resulting in increased application time and risk. question.

According to an aspect of an embodiment of the present invention, a positioning and alignment method for a micro-manipulator is provided, including: determining the initial posture of the micro-manipulator in a stowed state and the center positions of multiple targets, wherein the initial The posture includes an initial position and an initial posture; according to the initial position and the center positions of multiple targets, the target area of the micro-manipulator in the motion plane is determined, where the target area is the target area of the micro-manipulator. The instrument clamping front end can reach at least two target areas when the micro-manipulator is in the initial posture; the micro-manipulator is controlled to move from the initial position to a predetermined position in the target area, and the The micro-manipulator is fixed at the predetermined position, and , the predetermined position is any position within the target area; the micro-manipulator is controlled to use the predetermined position as a base point to convert from the stowed state to the unfolded state, and move to the reachable target in sequence. center position and align the reachable target.

Optionally, determining the initial posture of the micro-manipulator in the stowed state includes: detecting a position mark attached to the micro-manipulator in the stowed state; determining based on the position mark The instrument holds the initial position and the initial posture of the front end in the physical space coordinate system, and uses the initial position and the initial posture as the initial posture.

Optionally, determining the center positions of multiple targets includes: acquiring a planning image containing multiple targets; comparing the first marker point in the planning image with the second marker point in the physical space coordinate system. Register to obtain a registration matrix, where the coordinates of the first marker point in the image coordinate system correspond to the coordinates of the second marker point in the physical space coordinate system one-to-one; according to the registration matrix Perform coordinate transformation on multiple targets to obtain coordinates of multiple targets in the physical space coordinate system, and use the coordinates of multiple targets in the physical space coordinate system as multiple targets. central location.

Optionally, obtaining a planning image containing a plurality of the targets includes: obtaining three-dimensional image data of a plurality of the targets; planning a plurality of the targets according to the three-dimensional image data to obtain the planning image, wherein , the target in the planning image includes an instrument entry point and a target end point.

Optionally, determining the target area of the micro-manipulator in the motion plane based on the initial position and the center positions of multiple targets includes: obtaining the maximum movement radius of the micro-manipulator; The motion plane of the arm determines the intersection point between the motion plane and multiple target extension lines, wherein the target extension line is an extension line connecting the target instrument entry point and the target end point to form a line segment; determine the initial The distance between the position and each intersection point; the target whose distance is less than or equal to the maximum movement radius is recorded in the list as the reachable target; the intersection point corresponding to the reachable target As the center, the maximum movement radius is used as the radius to generate a circular area of all reachable targets in the list in the motion plane; based on the circular area of all reachable targets in the motion plane, Generate a common intersection area and use the common intersection area as the target area.

Optionally, controlling the micro-manipulator to move from the initial position to a predetermined position in the target area includes: displaying the initial position and the target area in real time; based on the real-time display of the initial position and the target area, moving the micromanipulator from the initial position to a predetermined position in the target area.

Optionally, moving to the center position of the reachable targets in sequence includes: based on the motion plane of the micro-manipulator, determining the target extension corresponding to the motion plane and at least two reachable targets. The intersection point of the lines, wherein the target extension line is the extension line connecting the target instrument entry point and the target end point to form a line segment; obtain the The movement path of the micro-manipulator, wherein the movement path of the micro-manipulator includes an alignment sequence of at least two reachable targets, the alignment sequence is based on the distance between the predetermined position and each of the intersection points Determine; according to the movement path of the micro-manipulator, control the micro-manipulator to move to the reachable center position of the target.

Optionally, controlling the movement of the micro-manipulator to the central position of the reachable target includes: determining the movement range of the robot arm required to move the micro-manipulator to the central position of the reachable target; according to The movement range of the micro-manipulator moves the micro-manipulator to a central position of the reachable target.

Optionally, after moving the micro-manipulator arm to the reachable center position of the target according to the movement range of the micro-manipulator arm, the method further includes: obtaining the position of the micro-manipulator arm after completing the movement. posture; determine the angle difference and distance difference between the target line segment and the posture, wherein the target line segment is a line segment connected to the entry point of the instrument that can reach the target and the target end point; in the angle difference If the angle difference is less than the angle difference threshold and the distance difference is less than the distance difference threshold, it is determined that the micro-manipulator has reached the center position of the reachable target.

Optionally, after aligning the reachable target, the method further includes: detecting whether the micro-manipulator is fixed in the target area; If so, control the micro-manipulator to move from the center position of the currently reachable target to the center position of the next reachable target, and align the next reachable target; If the micro-manipulator is not fixed in the target area, the micro-manipulator is returned to the stowed state, and the target area of the micro-manipulator is re-determined based on the remaining reachable targets. , fix the micro-manipulator at the predetermined position of the target area, control the micro-manipulator to convert from the stowed state to the unfolded state with the predetermined position as the base point, and move to the remaining reachable areas in sequence. The center position of the target and align the remaining reachable targets.

According to another aspect of the embodiment of the present invention, a positioning and alignment device for a micro-manipulator is also provided, including: a first determination unit for determining the initial posture and multiple targets of the micro-manipulator in a stowed state. center position, wherein the initial pose includes an initial position and an initial posture; a second determination unit configured to determine the position of the micro-manipulator in the motion plane based on the initial position and the center positions of multiple targets. A target area, wherein the target area is an area where the instrument holding front end of the micro-manipulator can reach at least two of the targets when the micro-manipulator is in the initial posture; the first control unit, with In order to control the micro-manipulator to move from the initial position to a predetermined position in the target area, the micro-manipulator is fixed at the predetermined position, wherein the predetermined position is any one in the target area. Position; the second control unit is used to control the micro-manipulator to convert from the stowed state to the unfolded state with the predetermined position as the base point, move in sequence to the center position of the reachable target, and control the reachable target. to align with the target.

According to another aspect of an embodiment of the present invention, an electronic device is also provided, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute any of the above. The method steps described steps.

According to another aspect of the embodiment of the present invention, a computer-readable storage medium is also provided. The computer-readable storage medium includes a stored program, wherein when the program is running, the location of the computer-readable storage medium is controlled. The device performs any of the method steps described above.

In the embodiment of the present invention, the initial posture of the micro-manipulator in the folded state and the center positions of multiple targets are determined, where the initial posture includes the initial position and the initial posture; according to the initial position and the centers of the multiple targets Position, determine the target area of the micro-manipulator in the motion plane, where the target area is an area where the instrument clamping front end of the micro-manipulator can reach at least two targets when the micro-manipulator is in the initial posture; control the micro-manipulator from the initial position The position moves to a predetermined position in the target area, and the micro-manipulator is fixed at the predetermined position, where the predetermined position is any position in the target area; the micro-manipulator is controlled to convert from the stowed state to the unfolded state based on the predetermined position, and then Move to the center of the reachable target and aim at the reachable target. That is to say, the embodiment of the present invention needs to control the micro-manipulator to move to the target area of the motion plane, fix the micro-manipulator at a predetermined position, and then control the micro-manipulator to move to the center of the reachable target in sequence, thereby achieving The micro-manipulator aligns multiple reachable targets, thereby solving the technical problem in related technologies that the positioning of the micro-manipulator requires frequent manual movement and fixation, resulting in increased application time and risk, and reduces the number of moves and fixations. , the technical effect of reducing application time and risk.

Description of drawings

The drawings described here are used to provide a further understanding of the present invention and constitute a part of this application. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the attached picture:

Figure 1 is a flow chart of a positioning and alignment method for a micromanipulator provided by an embodiment of the present invention;

Figure 2 is a schematic diagram of the intersection between the motion plane of the micro-manipulator and the target according to an embodiment of the present invention;

Figure 3 is a schematic diagram of the common intersection area of the circle formed by all targets provided by the embodiment of the present invention;

Figure 4 is a schematic diagram showing that the micro-manipulator provided by the embodiment of the present invention has been aligned with the target 1;

FIG. 5 is a schematic diagram of a positioning and alignment device for a micromanipulator provided by an embodiment of the present invention.

Detailed ways

In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only These are some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the scope of protection of the present invention. around.

It should be noted that the terms "first", "second", etc. in the description, claims and drawings of the present invention are used to distinguish different objects, rather than to limit a specific order.

According to one aspect of an embodiment of the present invention, a method for positioning and aligning a micro-manipulator is provided. It should be noted that the steps shown in the flow chart of the accompanying drawings can be implemented in a computer system such as a set of computer-executable instructions. are performed, and, although a logical order is shown in the flowchart diagrams, in some cases the steps shown or described may be performed in a different order than herein.

Figure 1 is a flow chart of a positioning and alignment method for a micromanipulator provided by an embodiment of the present invention. As shown in Figure 1, the method includes the following steps:

Step S102, determine the initial posture of the micro-manipulator in the stowed state and the center positions of multiple targets, where the initial posture includes the initial position and the initial posture;

The above-mentioned micro-manipulator is a component of a micro-robot. The micro-manipulator includes but is not limited to a lower actuator arm and an upper actuator arm. The lower actuator arm is provided with an instrument clamping front end and a position mark. The above-mentioned multiple targets are the positioning and alignment targets of the micro-manipulator, and these targets are located in small local areas. During the specific implementation process, the micro-manipulator arm in the retracted state can be placed above the area where multiple targets are located, and then the initial posture of the micro-manipulator arm in the retracted state is obtained. In addition, position and posture or position and posture are also called poses.

Step S104: Determine the target area of the micro-manipulator in the motion plane based on the initial position and the center positions of the multiple targets, where the target area is the instrument clamping front end of the micro-manipulator that can reach at least 1 when the micro-manipulator is in the initial posture. Two target areas;

The above-mentioned target area is a part of the movement plane of the micro-manipulator, which belongs to the movement plane of the micro-manipulator; the above-mentioned micro-manipulator can cover multiple targets in the target area of the movement plane, that is, the micro-manipulator is fixed in the target area. Multiple targets that can be reached; optionally, multiple targets are located on the same side relative to the micro-manipulator. The target area is reachable by the instrument clamping front end of the micro-manipulator when the micro-manipulator is in the initial position. The area of multiple targets; the multiple targets are located on different sides relative to the micro-manipulator. The target area is the area with the most targets that can be reached by the instrument clamping front end of the micro-manipulator when the micro-manipulator is in the initial position.

Step S106, control the micro-manipulator to move from the initial position to a predetermined position in the target area, and fix the micro-manipulator at the predetermined position, where the predetermined position is any position in the target area;

Since the target area includes multiple positions, and any one position can achieve the goal of reaching the most targets, the micro-manipulator can be controlled to move from the initial position to any position in the target area, and then the micro-manipulator can be fixed at that position. A rough positioning can be achieved.

Step S108, control the micro-manipulator to switch from the folded state to the unfolded state with the predetermined position as the base point, and move sequentially Go to the center of the reachable target and align the reachable target.

Optionally, controlling the micro-manipulator to move sequentially to the center position of the reachable target is related to the movement path of the micro-manipulator. Since the movement path of the micro-manipulator includes the alignment sequence of at least two reachable targets, the alignment sequence of the at least two reachable targets can be flexibly adjusted according to the needs of the application scenario; for example, The alignment sequence of multiple reachable targets is preset as the movement path of the micro-manipulator, and then the micro-manipulator is controlled to move to the center of each reachable target in sequence according to the movement path; for another example, the micro-manipulator can be The movement plane of the arm is determined by the intersection point of the movement plane and the target extension line corresponding to multiple reachable targets. The target extension line is the extension line of the line segment connected to the target instrument entry point and the target end point, and then the target extension line is connected to the predetermined position according to the predetermined position. The distance between each intersection determines the alignment sequence of multiple reachable targets as the movement path of the micro-manipulator. Finally, the micro-manipulator is controlled to move to the center of each reachable target in sequence according to the movement path; here it can be reached The alignment order of multiple targets includes but is not limited to the sorting result obtained by sorting the multiple reachable targets from small to large or from small to large according to the distance between the predetermined position and each intersection point.

It should be noted that in the above implementation steps of the present invention, performing a rough positioning of the micro-manipulator can achieve alignment of at least one reachable target. Optionally, if the target area of the micro manipulator can cover all targets, only one rough positioning can be performed to achieve alignment of all targets; if the target area of the micro manipulator only covers part of the targets, multiple times are required. By executing the above method steps, multiple rough positionings are required to achieve alignment of all targets. Compared with the existing technology, the above method can reduce the number of rough positioning, thereby reducing the time-consuming and operational risks of manual rough positioning.

It should be noted that the application scenarios of the above method include but are not limited to orthopedic surgery and other positioning areas with small and Scenarios with high positioning accuracy requirements.

In an optional implementation, determining the initial posture of the micro-manipulator arm in the stowed state includes: detecting the position mark attached to the micro-manipulator arm in the stowed state; determining the instrument clamp according to the position mark Hold the initial position and initial attitude of the front end in the physical space coordinate system, and use the initial position and initial attitude as the initial pose.

Since the corresponding position mark is attached to the micro manipulator, the instrument holding front end of the micro manipulator can be detected by the detection equipment, and then the position mark attached to the micro manipulator is detected when the micro manipulator is in the folded state, and then based on the position mark To calculate the initial position and initial posture of the instrument clamping front end in the physical space coordinate system, and the initial posture also includes the initial position and initial posture.

In the above embodiment of the present invention, the initial position and initial attitude of the instrument clamping front end in the physical space coordinate system are calculated using the position mark attached to the micro-manipulator, so that the retracted state of the micro-manipulator can be accurately obtained the initial pose below.

In an optional implementation, determining the center positions of multiple targets includes: acquiring a planning image containing multiple targets; comparing the first marker point in the planning image with the second marker point in the physical space coordinate system. Register to obtain a registration matrix, in which the coordinates of the first marker point in the image coordinate system correspond to the coordinates of the second marker point in the physical space coordinate system one-to-one; perform coordinate transformation on multiple targets according to the registration matrix, The coordinates of multiple targets in the physical space coordinate system are obtained, and the coordinates of the multiple targets in the physical space coordinate system are used as the center positions of the multiple targets.

The first marker point in the above planned image is the location of the physical marker point used when collecting the image. The location of the physical marker point can be at the center of the target or at the edge of the area where multiple targets are located. There is no limit here. . In the specific implementation process, it is necessary to first obtain a planning image containing multiple targets, and then register the first marker point in the planning image with the second marker point in the physical space coordinate system to obtain the registration matrix, and then according to the registration The quasi-matrix performs coordinate transformation on multiple targets to obtain the coordinates of the multiple targets in the physical space coordinate system, and uses the coordinates of the multiple targets in the physical space coordinate system as the center positions of the multiple targets.

In the above-described embodiment of the present invention, the coordinates of multiple targets in the physical space coordinate system are obtained through coordinate transformation using a registration matrix, and then the center positions of the multiple targets can be obtained.

In an optional implementation, obtaining a planning image containing multiple targets includes: obtaining three-dimensional image data of multiple targets; planning the multiple targets based on the three-dimensional image data to obtain a planning image, wherein in the planning image The target includes the instrument entry point and the target end point.

It should be noted that the above-mentioned instrument entry point and target end point are represented by three-dimensional coordinates. The instrument entry point and target end point of any target in the three-dimensional image data can reflect the center position of the corresponding target in the three-dimensional space of the image.

In the above-described embodiment of the present invention, multiple targets are planned through three-dimensional image data of multiple targets, and corresponding planning images are obtained to facilitate subsequent registration processing and coordinate transformation.

In an optional implementation, determining the target area of the micro-manipulator in the motion plane based on the initial position and the center positions of the multiple targets includes: obtaining the maximum movement radius of the micro-manipulator; based on the motion plane of the micro-manipulator , determine the intersection points of the motion plane and multiple target extension lines, where the target extension line is the extension line connecting the target instrument entry point and the target end point to form a line segment; determine the distance between the initial position and each intersection point; set the distance to be less than or equal to the maximum movement The target of the radius is recorded in the list as a reachable target; using the intersection corresponding to the reachable target as the center and the maximum movement radius as the radius, all reachable targets in the list are generated in the circular area of the motion plane; according to all The reachable target is in the circular area of the motion plane, a common intersection area is generated, and the common intersection area is used as the target area.

The above maximum movement radius is the maximum movement distance of the micro-manipulator from the stowed state to the expansion in a single direction under fixed conditions; the above list is used to save reachable targets.

In order to more accurately obtain the target area of the micro-manipulator in the motion plane, it is first necessary to obtain the maximum movement radius of the micro-manipulator, and then determine the intersection points of the motion plane and multiple target extension lines according to the motion plane of the micro-manipulator, and determine the initial The distance between the position and each intersection point, and then the target whose distance is less than or equal to the maximum movement radius is recorded in the list as the reachable target, and then the intersection point corresponding to the reachable target is used as the center and the maximum movement radius is used as the radius to generate a list All reachable targets are in the circular area of the motion plane. Finally, a common intersection area is generated based on all reachable targets in the circular area of the motion plane. This common intersection area is also the target area.

In the above-described embodiment of the present invention, the target area of the micro-manipulator in the motion plane can be obtained more accurately, thereby fixing the micro-manipulator to the target area, which can effectively reduce the number of rough positioning.

In an optional implementation, controlling the micro-manipulator to move from the initial position to a predetermined position in the target area includes: displaying the initial position and the target area in real time; and moving the micro-manipulator from the original position and the target area based on the real-time display of the initial position and the target area. The initial position is moved to a predetermined position in the target area.

In the above embodiments of the present invention, the initial position and the target area are displayed in real time through visualization, and the micro-manipulator is controlled to move from the initial position to a predetermined position in the target area based on the visual display content, thereby realizing the visualization of the rough positioning of the micro-manipulator. Operation, the initial position and target area are displayed more intuitively, which can reduce misoperation and grasp the execution dynamics of rough positioning in a timely manner.

In an optional implementation, moving to the center position of the reachable targets in sequence includes: determining the intersection point of the motion plane and target extension lines corresponding to at least two reachable targets according to the motion plane of the micro-manipulator, Among them, the target extension line is the extension line of the line segment connecting the target instrument entry point and the target end point; the movement path of the micro-manipulator is obtained, wherein the movement path of the micro-manipulator includes the alignment sequence of at least two targets that can be reached, The alignment sequence is determined based on the distance between the predetermined position and each intersection point; according to the movement path of the micro-manipulator, the micro-manipulator is controlled to move to the center position of the reachable target.

Optionally, the at least two reachable targets can be sorted according to the distance between the predetermined position and each intersection point to obtain the alignment order of the at least two reachable targets. For example, the at least two reachable targets can be sorted according to the distance from small to large. The targets are sorted to obtain the alignment sequence of at least two targets that can be reached, and the alignment sequence is used as the movement path of the micro-manipulator. For example, the alignment sequence obtained according to distance is: target 1, target 2 and target 3, then the micro-manipulator will first move to target 1, then move from target 1 to target 2, and then move from target 2 to target 3, and control the micro-manipulator to move to target 1, target 2 and The center position of target 3.

In the above-mentioned embodiments of the present invention, by obtaining the optimal movement path of the micro-manipulator and causing the micro-machine to achieve alignment of multiple targets according to the movement path, the selection of the alignment target is optimized and the target alignment efficiency is improved. .

In an optional implementation, controlling the movement of the micro-manipulator to the central position of the reachable target includes: determining the movement range of the micro-manipulator required to move the micro-manipulator to the center position of the reachable target; according to the micro-machine The arm movement range moves the micro-manipulator to the center of the reachable target.

The above-mentioned robot arm movement range includes the movement distance of the upper actuator arm and/or the movement distance of the lower actuator arm;

In the above-described embodiment of the present invention, the movement range of the robot arm required for the micro-manipulator to be deployed from the stowed state to the center position of the reachable target is used to control the movement of the micro-manipulator to the center position of the reachable target. This allows the micro-manipulator to accurately move to the center of the reachable target.

In an optional implementation, after moving the micro-manipulator arm to the center position of the reachable target according to the movement range of the micro-manipulator arm, the above method further includes: obtaining the pose of the micro-manipulator arm after completing the movement; determining The angle difference and distance difference between the target line segment and the posture, where the target line segment is a line segment connected to the entry point of the instrument that can reach the target and the target end point; when the angle difference is less than the angle difference threshold and the distance difference is less than the distance In the case of a difference threshold, it is determined that the micro-manipulator has reached the center position of the reachable target.

The above pose includes the position and attitude of the micro-manipulator after completing its movement, where the position is represented by coordinates in the physical space coordinate system, and the attitude is represented by a rotation vector;

In order to confirm whether the micro-manipulator arm has moved in place, after moving the micro-manipulator arm to the center position of the reachable target, the pose of the micro-manipulator arm after completing the movement can be obtained, and then the angle difference between the target line segment and the pose can be calculated and distance difference, and then determine whether the angle difference is less than the angle difference threshold and whether the distance difference is less than the distance difference threshold. If the angle difference is less than the angle difference threshold and the distance difference is less than the distance difference threshold, then the micro When the robotic arm has reached the center position of the reachable target, it indicates that the micro robotic arm has moved into position.

In addition, if the angle difference is greater than or equal to the angle difference threshold or the distance difference is greater than or equal to the distance difference threshold, it indicates that the micro manipulator has not moved in place, and the movement of the micro manipulator needs to be appropriately adjusted until the micro manipulator has been moved into position.

It should be noted that the above-mentioned angle difference threshold and distance difference threshold can be set according to the needs of the application scenario; Optionally, the above angle difference threshold is 0.1 degrees, and the distance difference threshold is 0.3 mm.

In an optional implementation, after aligning the reachable target, the above method further includes: detecting whether the micro-manipulator is fixed in the target area; if the micro-manipulator is fixed in the target area, control The micro-manipulator moves from the center position of the currently reachable target to the center position of the next reachable target, and aligns the next reachable target; when the micro-manipulator is not fixed in the target area, then Return the micro-manipulator to the stowed state, re-determine the target area of the micro-manipulator based on the remaining reachable targets, fix the micro-manipulator at a predetermined position in the target area, and control the micro-manipulator to transition from the stowed state with the predetermined position as the base point. In the expanded state, move to the center of the remaining reachable targets in sequence and align the remaining reachable targets.

Since the micro manipulator is prone to artificially moving its position after the target is aligned, the previous rough positioning of the micro manipulator cannot reach the remaining reachable targets. Therefore, in the present invention, after the micro manipulator is aligned with the target, It will detect whether the micro-manipulator is still fixed in the target area; if the micro-manipulator is fixed in the target area, the micro-manipulator can be controlled to move from the center position of the reachable target to the center position of the next reachable target. And align the next reachable target; if the micro-manipulator is not fixed in the target area, the micro-manipulator can be returned to the stowed state, and the target area of the micro-manipulator is re-determined according to the remaining reachable targets, and the micro-manipulator can be moved to the target area. The robotic arm is fixed at a predetermined position in the target area, and then controls the micro robotic arm to convert from the stowed state to the unfolded state based on the predetermined position, and then move to the center of the remaining reachable targets in sequence, and align the remaining reachable targets. allow.

In the above-described embodiment of the present invention, it is determined whether the previous rough positioning is still effective by detecting whether the micro-manipulator is still fixed in the target area. When the previous rough positioning is still effective, continue to align the remaining reachable targets. , otherwise it is necessary to re-perform rough positioning, that is, use the remaining reachable targets to re-determine the target area of the micro-manipulator to achieve positioning and alignment of the remaining reachable targets, thus avoiding invalid operations on the remaining reachable targets. .

Taking orthopedic surgery as an example, an optional embodiment of the present invention will be described in detail below.

Step 1. Rough positioning. The corresponding specific implementation steps are as follows:

Step 1.1. Use surgical planning software to plan the surgical target accordingly (generally, planning can be based on three-dimensional image data such as CT, MRI or ultrasound images). The result of the planning is 2 points, each point is represented by (x, y, z) express. The two points represent the target end point and the equipment entry point respectively. The two points can ultimately be represented by a line segment or a vector;

Step 1.2: Use the navigation registration method to register the planning image with the physical spatial location. Generally, the position of the physical marker point is used when collecting the image so that it can be scanned into the image so that it can be recognized by the algorithm. Then, the registration matrix is obtained by registering the physical marker points in the physical space coordinate system and the marker points in the image coordinate system. The registration matrix is expressed in a 3×3 matrix;

Step 1.3. Transform all surgical targets according to the registration matrix to obtain the coordinates of the corresponding targets in the physical space coordinate system;

Step 1.4. Attach corresponding position marks to the micro-manipulator. The instrument holding front end of the micro-manipulator can be a passive optical reflective mark or an electromagnetic sensor so that it can be detected by the optical or electromagnetic positioning system, thereby obtaining the instrument holding front end of the micro-manipulator. Position and attitude in the physical space coordinate system (including current position and rotation vector);

Step 1.5. When the micro-manipulator needs to be placed in the approximate area of the surgical area, the current position and target position of the manipulator (corresponding to the center position of the above target) are calculated based on the screen. Calculate the area that can cover the most targets at the same time and give the corresponding logo;

The optional calculation method is as follows: Assume that the maximum moving radius of the micro-manipulator from the stowed state to unfolding to a single direction under the current fixed conditions is x, where the value of The hardware parameters of the robotic arm are as follows;

Step (a): In the retracted state of the micro-manipulator, according to the motion plane of the micro-manipulator, find the intersection point between the plane and each target (the extension of the line segment connecting the two points of the target). Calculate the distance between the current position of the micro-manipulator and the intersection point. If it is less than x, then the target is included in the list of reachable targets.

It should be noted that the above-mentioned micro robot arm is divided into a lower actuator arm and an upper actuator arm. The movement of the lower actuator arm is the movement of the robot arm in the current plane, and the movement of the upper actuator arm can also By moving in its plane, the angle of the clamped instrument is changed by changing the positional relationship with the lower actuator arm; Figure 2 is a schematic diagram of the intersection of the motion plane of the micro-manipulator provided by the embodiment of the present invention and the target, as shown As shown in Figure 2, in order to move the micro-manipulator to the target trajectory, which is the line connecting the two points of the target, move the position B of the instrument held by the lower actuator arm to the target connection line B'. Among them, B' is the intersection point of the plane where the lower actuator arm is located and the line connecting the two target points. Then the position A of the instrument held by the upper actuator arm is also moved to the above-mentioned connecting line A', where, A ′ is the intersection point of the plane where the upper actuator arm is located and the line connecting the two points of the target; the movement distance of the upper actuator arm is the movement of the mechanical arm, recorded as y′, and the movement of the upper actuator arm reaches the target when the lower actuator arm reaches The latter can be satisfied without too much consideration; and the moving distance of the lower actuator arm is the moving range of the mechanical arm mentioned above, which is x′. If x′<x, it is considered to be included in the reachable range. target list;

Step (b): Place all the targets in the reachable list, with the intersection point just calculated as the center and x as the radius, to form a circular area on the motion plane of the micro-manipulator. The common intersection area of the circle formed by all targets is the area that can cover the most targets at the current position and posture of the micro-manipulator. By analogy, the common intersection areas of other circles are areas that can cover the corresponding targets at the same time;

Step (c): Depending on the number of intersecting targets, different color prompts can be provided for the corresponding areas. Figure 3 is a schematic diagram of the common intersecting area of the circle formed by all targets provided by the embodiment of the present invention, as shown in Figure 3. In the common intersection area of the circle formed by target 1, target 2, target 3 and target 4, the dark area can cover the most area, while other areas cover relatively few targets.

Step 1.6. When moving the micro-manipulator, use the position mark attached to the micro-manipulator to move the micro-manipulator in real time. The screen visually displays its position (circular dark area), and prompts the user based on the current position, the number of targets that the current micro-manipulator can cover, so that the user can understand the micro-manipulator and the above-mentioned target area that can be covered The positional relationship guides the user to place the micro-manipulator to cover as many target areas as possible, and then fix it.

Step 2. Automatically move from the rough positioning position to the center position of a certain target

From the rough positioning position (the micro-manipulator is in the retracted state), the movement range x' and y' of the micro-manipulator required to move the micro-manipulator to the target position (vector) are calculated in real time, and x' and y' are calculated by sending instructions. If y′ is given to the micro-manipulator platform, the movement of the micro-manipulator can be completed. Figure 4 is a schematic diagram of the micro manipulator provided by the embodiment of the present invention that has been aligned with the target 1. As shown in Figure 4, the micro manipulator has covered the target 1; during the movement of the micro manipulator, the position of the micro manipulator on the screen can be updated in real time. position (indicated by a dark circular area) to let the user know the progress of the movement and give prompts when the movement is in place.

Furthermore, the method for determining that the micro-manipulator has covered the target is as follows:

Through the position and attitude of the micro-manipulator that has completed the movement, for example, the current position and rotation vector are obtained according to the position sensor, and the angle difference and distance difference between the line connecting the two points of the target are calculated. If the angle difference and distance difference are less than a certain threshold (such as 0.3mm and 0.1 degrees), the micro-manipulator is considered to have covered the target.

Step 3. Move from the center of one target to the center of another target

When moving from one target to another, it is necessary to determine whether the position and posture of the micro-manipulator has changed since the last time the micro-manipulator was placed and locked. If the position has changed, it may have previously been considered that the micro-manipulator The initial placement area where the arm can cover multiple targets is no longer valid. At this time, it may not be possible to directly reach the corresponding target position through the automatic movement of the micro-manipulator. It is necessary to consider the possibility that the micro-manipulator is moving to a certain target position. Later, the position was artificially moved. In this case, although the rough fixation of the micro-manipulator at the beginning ensured that the micro-manipulator could reach several targets at the same time. However, due to human movement, the micro-manipulator may not be able to reach several targets that were previously thought to be reachable, so recalculation is required. In the specific implementation process, the following methods can be used:

Step (1): The position and posture of the micro manipulator at a certain target can be represented by a matrix C, which can be obtained based on the position and posture A of the micro manipulator in the stowed state combined with the movement posture of the micro manipulator itself. The micro manipulator arm The own movement pose can be considered as a local coordinate system transformation, represented by matrix B. Then there is C=B·A;

Step (2): Calculate the position and attitude A of the micro-manipulator at the last positioning based on the position matrix C of the micro-manipulator at a certain target, then A=B ^-1 C;

Step (3): After obtaining the position and posture A, calculate the distance range of the movement posture from the position and posture A to the target that needs to be reached. If the movement distance and posture are within the one-directional movement range of the micro-manipulator starting from A, the micro-manipulator directly executes the corresponding movement posture instructions and reaches the corresponding target in one go. Otherwise, the corresponding target is considered unreachable. Pass The screen prompts that the micro-manipulator must return to the retracted state and perform rough positioning again.

According to another aspect of the embodiment of the present invention, a positioning and alignment device for a micro-manipulator is also provided. Figure 5 is a schematic diagram of a positioning and alignment device for a micro-manipulator provided by an embodiment of the present invention. As shown in Figure 5, The positioning and alignment device of the micro manipulator includes: a first determination unit 52 , a second determination unit 54 , a first control unit 56 and a second control unit 58 . The positioning and alignment device of the micro robotic arm will be described in detail below.

The first determination unit 52 is used to determine the initial posture of the micro-manipulator in the stowed state and the center positions of multiple targets, where the initial posture includes the initial position and the initial posture;

The second determination unit 54 is connected to the above-mentioned first determination unit 52 and is used to determine the target area of the micro-manipulator in the motion plane according to the initial position and the center positions of the multiple targets, where the target area is the instrument holder of the micro-manipulator. The holding front end can reach the area of at least two targets when the micro manipulator is in the initial position;

The first control unit 56 is connected to the above-mentioned second determination unit 54 and is used to control the micro-manipulator to move from the initial position to a predetermined position in the target area and fix the micro-manipulator at the predetermined position, where the predetermined position is within the target area. any position;

The second control unit 58 is connected to the above-mentioned first control unit 56 and is used to control the micro-manipulator to convert from the stowed state to the unfolded state with the predetermined position as the base point, move to the center position of the reachable target in sequence, and control the reachable target. target to align.

In the embodiment of the present invention, the positioning and alignment device of the micro-manipulator is used to determine the initial posture of the micro-manipulator in the stowed state and the center positions of multiple targets, where the initial posture includes the initial position and the initial posture; Determine the target area of the micro-manipulator in the motion plane based on the initial position and the center positions of multiple targets, where the target area is that the instrument clamping front end of the micro-manipulator can reach at least two targets when the micro-manipulator is in the initial posture. area; control the micro-manipulator to move from the initial position to a predetermined position in the target area, and fix the micro-manipulator at the predetermined position, where the predetermined position is any position within the target area; control the micro-manipulator to use the predetermined position as the base point by The folded state is converted to the unfolded state, and the system moves to the center of the reachable target in sequence, and aims at the reachable target. That is to say, the embodiment of the present invention needs to control the micro-manipulator to move to the target area of the motion plane, fix the micro-manipulator at a predetermined position, and then control the micro-manipulator to move to the center of the reachable target in sequence, thereby achieving The micro-manipulator aligns multiple reachable targets, thereby solving the technical problem in related technologies that the positioning of the micro-manipulator requires frequent manual movement and fixation, resulting in increased application time and risk, and reduces the number of moves and fixations. , the technical effect of reducing application time and risk.

It should be noted here that the above-mentioned first determination unit 52, second determination unit 54, first control unit 56 and second control unit 58 correspond to steps S102 to S108 in the method embodiment, and the above-mentioned units are the same as the corresponding steps. The implementation examples and application scenarios are the same, but are not limited to the content disclosed in the above method embodiments.

In an optional implementation, the above-mentioned first determination unit 52 includes: a detection module for detecting the position mark attached to the micro-manipulator when the micro-manipulator is in the stowed state; a first determination module for detecting the position according to the location of the micro-manipulator. The mark determines the initial position and initial posture of the instrument clamping front end in the physical space coordinate system, and takes the initial position and initial posture as the initial posture.

In an optional implementation, the above-mentioned first determination unit 52 includes: a first acquisition module, used to acquire a planning image containing multiple targets; a registration module, used to compare the first marker point in the planning image with The second marker point in the physical space coordinate system is registered to obtain a registration matrix, in which the coordinates of the first marker point in the image coordinate system correspond to the coordinates of the second marker point in the physical space coordinate system; transformation The module is used to perform coordinate transformation on multiple targets according to the registration matrix, obtain the coordinates of multiple targets in the physical space coordinate system, and use the coordinates of the multiple targets in the physical space coordinate system as the center positions of the multiple targets.

In an optional implementation, the above-mentioned first acquisition module includes: a first acquisition sub-module for acquiring three-dimensional image data of multiple targets; and a first processing sub-module for processing multiple targets based on the three-dimensional image data. Perform planning to obtain a planning image, where the target in the planning image includes the instrument entry point and the target end point.

In an optional implementation, the above-mentioned second determination unit 54 includes: a second acquisition module, used to acquire the maximum movement radius of the micro-manipulator; and a second determination module, used to determine based on the motion plane of the micro-manipulator. The intersection of the motion plane and multiple target extension lines, where the target extension line is the extension line of the line segment connected by the target instrument entry point and the target end point; the third determination module is used to determine the distance between the initial position and each intersection point; the fourth The determination module is used to record the targets whose distance is less than or equal to the maximum movement radius as reachable targets into the list; the first generation module is used to use the intersection corresponding to the reachable target as the center and the maximum movement radius as the radius. Generate all reachable targets in the list in the circular area of the motion plane; the second generation module is used to generate a common intersection area based on all reachable targets in the circular area of the motion plane, and use the common intersection area as the target area.

In an optional implementation, the above-mentioned first control unit 56 includes: a display module for displaying the initial position and the target area in real time; a first processing module for displaying the initial position and the target area of the miniature in real time. The robotic arm moves from the initial position to a predetermined position in the target area.

Optionally, the above-mentioned second control unit 58 includes: a fifth determination module, configured to determine, according to the movement plane of the micro-manipulator, the intersection point of the movement plane and target extension lines corresponding to at least two reachable targets, wherein the target extension The line is an extension line connecting the target instrument entry point and the target end point to form a line segment; the third acquisition module is used to acquire the movement path of the micro-manipulator, wherein the movement path of the micro-manipulator includes pairs of at least two targets that can be reached. The alignment sequence is determined based on the distance between the predetermined position and each intersection point; the second processing module is used to control the micro-manipulator to move to the center position of the reachable target according to the movement path of the micro-manipulator.

In an optional implementation, the above-mentioned second processing module includes: a first determining sub-module for determining the micro The movement range of the manipulator arm required to move the manipulator arm to the center position of the reachable target; the second processing submodule is used to move the micro manipulator arm to the center position of the reachable target according to the movement range of the micro manipulator arm.

In an optional implementation, the above-mentioned second processing module also includes: a second acquisition sub-module, configured to acquire the The pose of the micro-manipulator after completing its movement; the second determination sub-module is used to determine the angle difference and distance difference between the target line segment and the pose, where the target line segment is the entry point of the instrument that can reach the target and the target end point connected into line segments; the third determination sub-module is used to determine that the micro-manipulator has reached the center position of the reachable target when the angle difference is less than the angle difference threshold and the distance difference is less than the distance difference threshold.

In an optional implementation, the above-mentioned second control unit 58 also includes: a detection module for detecting whether the micro-manipulator is fixed in the target area after aligning the target reachable by the micro-manipulator; a third The processing module is used to control the micro-manipulator to move from the center position of the currently reachable target to the center position of the next reachable target when the micro-manipulator is fixed in the target area, and to control the next reachable target. The target is aligned; the fourth processing module is used to return the micro-manipulator to the retracted state when the micro-manipulator is not fixed in the target area, and re-determine the target area of the micro-manipulator based on the remaining reachable targets. Fix the micro-manipulator at a predetermined position in the target area, control the micro-manipulator to convert from the stowed state to the unfolded state based on the predetermined position, move to the center position of the remaining reachable targets in sequence, and perform operations on the remaining reachable targets. alignment.

According to another aspect of the embodiment of the present invention, an electronic device is also provided, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to perform any of the above method steps .

According to another aspect of the embodiment of the present invention, a computer-readable storage medium is also provided. The computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any of the above. The method steps of the item.

It should be noted that the above-mentioned computer-readable storage medium can be located in any computer terminal in the computer terminal group in the computer network, and/or in any mobile terminal in the mobile terminal group. The above-mentioned computer-readable storage medium includes storage program of.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the scope of the present invention.

Claims

A positioning and alignment method for a micro-manipulator, which is characterized by including:

Determine the initial posture of the micromanipulator in the stowed state and the center positions of multiple targets, wherein the initial posture includes an initial position and an initial posture;

According to the initial position and the center positions of multiple targets, the target area of the micro-manipulator in the motion plane is determined, wherein the target area is the instrument clamping front end of the micro-manipulator at the position of the micro-machine. The arm can reach the area of at least two of the targets when it is in the initial position;

Controlling the micro-manipulator to move from the initial position to a predetermined position in the target area, and fixing the micro-manipulator at the predetermined position, wherein the predetermined position is any position within the target area ;

The micro-manipulator is controlled to use the predetermined position as a base point to convert from the stowed state to the unfolded state, move to the center position of the reachable target in sequence, and align the reachable target.
The method according to claim 1, characterized in that determining the initial posture of the micro-manipulator in the retracted state includes:

Detecting the position mark attached to the micro-manipulator arm when the micro-manipulator arm is in the stowed state;

The initial position and the initial posture of the instrument clamping front end in the physical space coordinate system are determined according to the position mark, and the initial position and the initial posture are used as the initial posture.
The method according to claim 1, characterized in that determining the center positions of multiple targets includes:

Obtaining a planning image containing a plurality of said targets;

Register the first marker point in the planning image with the second marker point in the physical space coordinate system to obtain a registration matrix, where the coordinates of the first marker point in the image coordinate system and the third marker point are The coordinates of the two marker points in the physical space coordinate system correspond one to one;

Perform coordinate transformation on multiple targets according to the registration matrix to obtain the coordinates of multiple targets in the physical space coordinate system, and obtain the coordinates of multiple targets in the physical space coordinate system. as a central location for multiple of said targets.
The method according to claim 3, characterized in that obtaining a planning image containing a plurality of said targets includes:

Obtain three-dimensional image data of multiple targets;

A plurality of the targets are planned according to the three-dimensional image data to obtain the planning image, wherein the targets in the planning image include an instrument entry point and a target end point.
The method according to claim 1, characterized in that, based on the initial position and the center positions of multiple targets, determining the target area of the micro-manipulator in the motion plane includes:

Obtain the maximum movement radius of the micro-manipulator;

According to the motion plane of the micro-manipulator, the intersection of the motion plane and multiple target extension lines is determined. point, wherein the target extension line is an extension line connecting the target instrument entry point and the target end point to form a line segment;

Determine the distance between the initial position and each of the intersection points;

The target whose distance is less than or equal to the maximum movement radius is recorded in the list as the reachable target;

Using the intersection point corresponding to the reachable target as the center and the maximum movement radius as the radius, generate a circular area in the motion plane for all the reachable targets in the list;

A common intersection area is generated based on all reachable circular areas of the target in the motion plane, and the common intersection area is used as the target area.
The method of claim 1, wherein controlling the micro-manipulator to move from the initial position to a predetermined position in the target area includes:

Display the initial position and the target area in real time;

According to the initial position and the target area displayed in real time, the micro-manipulator is moved from the initial position to a predetermined position in the target area.
The method according to claim 1, characterized in that sequentially moving to the central position of the reachable target includes:

According to the motion plane of the micro-manipulator, the intersection point of the motion plane and target extension lines corresponding to at least two reachable targets is determined, wherein the target extension line is the instrument entry point of the target An extension line connected to the target end point;

Obtain the movement path of the micro-manipulator, wherein the movement path of the micro-manipulator includes an alignment sequence of at least two reachable targets, and the alignment sequence is based on the predetermined position and each of the targets. The distance between the intersection points is determined;

According to the movement path of the micro-manipulator, the micro-manipulator is controlled to move to a central position of the reachable target.
The method of claim 7, wherein controlling the micro-manipulator to move to a central position of the reachable target includes:

Determine the movement range of the robot arm required to move the micro-robot arm to the central position of the reachable target;

According to the movement range of the micro-manipulator, the micro-manipulator is moved to a central position of the reachable target.
The method according to claim 8, characterized in that, after moving the micro-manipulator arm to a central position of the reachable target according to the movement range of the micro-manipulator arm, the method further includes:

Obtain the posture of the micro-manipulator after completing its movement;

Determine the angle difference and distance difference between the target line segment and the posture, wherein the target line segment is a line segment connected by the entry point of the instrument that can reach the target and the target end point;

When the angle difference is less than the angle difference threshold and the distance difference is less than the distance difference threshold, it is determined that the micro-manipulator has reached the center position of the reachable target.
The method according to any one of claims 1 to 9, characterized in that, after aligning the reachable target, the method further includes:

Detect whether the micro-manipulator is fixed in the target area;

When the micro-manipulator is fixed in the target area, the micro-manipulator is controlled to move from the center position of the currently reachable target to the center position of the next reachable target, and Aim at the next reachable said target;

When the micro-manipulator is not fixed in the target area, the micro-manipulator is returned to the stowed state, and the target of the micro-manipulator is re-determined based on the remaining reachable targets. area, fix the micro-manipulator at the predetermined position of the target area, control the micro-manipulator to use the predetermined position as the base point to convert from the stowed state to the unfolded state, and move sequentially to the remaining reachable the center position of the target, and align the remaining reachable targets.
A positioning and alignment device for a micro-manipulator, which is characterized by including:

A first determination unit configured to determine the initial posture of the micromanipulator in the stowed state and the center positions of multiple targets, where the initial posture includes an initial position and an initial posture;

A second determination unit, configured to determine the target area of the micro-manipulator in the motion plane according to the initial position and the center positions of multiple targets, wherein the target area is the instrument holder of the micro-manipulator. The holding front end can reach the area of at least two of the targets when the micro-manipulator is in the initial posture;

A first control unit, configured to control the micro-manipulator to move from the initial position to a predetermined position in the target area, and to fix the micro-manipulator at the predetermined position, wherein the predetermined position is the Any location within the target area;

The second control unit is used to control the micro-manipulator to switch from the stowed state to the unfolded state with the predetermined position as the base point, move to the center position of the reachable target in sequence, and control all reachable targets. Align with the above targets.
An electronic device, characterized by including:

processor;

Memory used to store instructions executable by the processor;

Wherein, the processor is configured to perform the method steps of any one of claims 1 to 10.
A computer-readable storage medium, characterized in that the computer-readable storage medium includes a stored program, Wherein, when the program is running, the device where the computer-readable storage medium is located is controlled to perform the method steps described in any one of claims 1 to 10.