WO2022141153A1

WO2022141153A1 - Ultrasonic positioning puncture system and storage medium

Info

Publication number: WO2022141153A1
Application number: PCT/CN2020/141231
Authority: WO
Inventors: 黄善灯; 柏龙; 陈晓红; 潘鲁锋; 柳建飞
Original assignee: 诺创智能医疗科技(杭州)有限公司
Priority date: 2020-12-30
Filing date: 2020-12-30
Publication date: 2022-07-07

Abstract

An ultrasonic positioning puncture system and a storage medium. The ultrasonic positioning puncture system comprises a computer device (101), a first series mechanical arm (102) and an ultrasonic detection apparatus (103). The computer device (101) comprises a processor (1012), and when running a computer program, the processor (1012) executes the following steps: acquiring first attitude information of an ultrasonic detection plane, second attitude information of the first series mechanical arm (102), and first position information of a target point in the ultrasonic detection plane; determining, according to the first attitude information and the second attitude information, the position conversion relationship between a reference coordinate system of the ultrasonic positioning puncture system and a coordinate system in which the ultrasonic detection plane is located (S203); and determining, according to the position conversion relationship and the first position information, second position information of the target point in the reference coordinate system (S204), thereby improving the accuracy of a puncture surgery.

Description

Ultrasound positioning puncture system and storage medium

technical field

The present application relates to the field of robotic arm control, and in particular, to an ultrasonic positioning puncture system and a storage medium.

Background technique

In the traditional puncture operation, under the guidance of image scanning equipment such as ultrasound, CT, and MRI, the doctor manually sends small surgical instruments (such as biopsy needles) into the patient's body to detect or treat the lesion. The nearby 2D or 3D scan images determine the appropriate needle entry point and needle entry direction, and then manually complete the puncture operation based on experience.

Computer-aided navigation techniques and robotics were introduced into puncture procedures. The working principle of the puncture robot with the surgical navigation system is as follows: first, the computer 3D synthesizes the scanned 2D image to form a 3D image near the lesion, and then the doctor judges the target position and the appropriate needle insertion path through the 3D image and inputs it into the navigation system. , and then the navigation system calculates the current state and target point of the robot manipulator arm and plans a trajectory. Finally, the manipulator arm completes the puncture positioning according to the planned trajectory, and then manually completes the needle insertion through the needle insertion mechanism or the doctor, avoiding pure manual operation. errors caused. However, the existing navigation technology usually locates and registers the puncture needle and medical imaging equipment through optical positioning, and performs puncture with the aid of medical images, but there are large errors in the process of optical positioning and registration, and in During the operation, the optical ball cannot be blocked, which limits the implementation of the operation.

SUMMARY OF THE INVENTION

According to various embodiments of the present application, an ultrasonic positioning and puncturing system is provided. The ultrasonic positioning and puncturing system includes a computer device, a first serial manipulator arm, and an ultrasonic detection device mounted on the end of the first serial manipulator arm. Computer equipment is electrically connected to the first serial robotic arm and the ultrasonic detection device respectively, the computer equipment includes a memory and a processor, the memory stores a computer program, and the processor runs the computer program Perform the following steps: locate the first position information of the target point in the ultrasonic detection plane projected by the ultrasonic detection device; obtain the first attitude information of the ultrasonic detection plane and the second attitude information of the first serial robot arm ; According to the first attitude information and the second attitude information, determine the position conversion relationship between the reference coordinate system of the ultrasonic positioning puncture system and the coordinate system where the ultrasonic detection plane is located; According to the position conversion relationship and the described position conversion relationship The first position information determines the second position information of the target point in the reference coordinate system.

The above ultrasonic positioning puncture system has the following advantages: by directly obtaining the coordinates of the lesion in the ultrasonic image, without the need for indirect registration through registration datums such as optical positioning, it solves the existing positioning and registration technology for puncture surgery under medical image navigation There is also the problem of error, which improves the accuracy of the puncture operation.

In some of these embodiments, the locating the first position information of the target point in the ultrasound detection plane projected by the ultrasound detection device includes: acquiring an ultrasound image acquired by the ultrasound detection device, in which the ultrasound image is Locate the position information of the target point; according to the position information of the target point in the ultrasonic image and the calibration parameters of the ultrasonic detection device, determine that the target point is in the ultrasonic detection plane projected by the ultrasonic detection device of the first location information.

In some of these embodiments, the ultrasonic detection device is rotatably connected to the end of the first serial manipulator, and the ultrasonic detection plane is formed at the end of the ultrasonic detection device and extends from the ultrasonic detection device. The end point of the ultrasonic detection plane extends along the fan plane; the ultrasonic detection plane rotates with the ultrasonic detection device, and the rotation axis of the ultrasonic detection plane passes through the end point; the first attitude information includes the rotation angle of the ultrasonic detection plane.

In some of the embodiments, the first position information includes the distance of the target point relative to the end point, and the included angle of the connecting line between the target point and the end point and the rotation axis.

In some of these embodiments, the first serial robotic arm includes a plurality of joints, the plurality of joints include: rotation joints and/or movement joints, and the second posture information includes motion amount information of each of the joints .

In some of the embodiments, the determining, according to the first attitude information and the second attitude information, the position conversion relationship between the reference coordinate system of the ultrasound positioning and puncturing system and the coordinate system where the ultrasound detection plane is located includes: : when the first serial manipulator is in the initial state, establish the reference coordinate system of the first serial manipulator and the joint coordinate system of each joint of the first serial manipulator, and when the ultrasonic detection device is in the In the initial state, a probe coordinate system is established with the end point of the ultrasonic detection device as the coordinate origin; the reference coordinate system of the first serial manipulator and the DH of the joint coordinate system of each joint of the first serial manipulator are obtained. parameters, and according to the DH parameters, determine the first position conversion relationship between the reference coordinate system of the first serial manipulator and the joint coordinate system of the end joint of the first serial manipulator; obtain the probe coordinates The position information of the coordinate origin of the system in the joint coordinate system of the terminal joint, and the attitude angle of the probe coordinate system relative to the joint coordinate system of the terminal joint, and according to the position information, attitude angle and the first a position conversion relationship, determining a second position conversion relationship between the reference coordinate system and the probe coordinate system; acquiring the rotation angle of the ultrasonic detection plane, and determining the coordinates of the ultrasonic detection plane according to the rotation angle The third position conversion relationship between the system and the probe coordinate system; according to the second position conversion relationship and the third position conversion relationship, determine the reference coordinate system of the ultrasonic positioning and puncturing system and the ultrasonic detection plane. The position conversion relationship of the coordinate system where it is located.

In some of the embodiments, the first serial manipulator includes a rotating joint and a moving joint, the Z axis of the joint coordinate system of the rotating joint is set along the rotating axis, and the Z axis of the joint coordinate system of the moving joint is set along the moving direction, The reference coordinate system and the joint coordinate system of each joint are both left-handed or right-handed systems, and when the rotary joint is connected to the previous mobile joint, the origin of the joint coordinate system of the rotary joint is the joint coordinate system of the previous mobile joint. The origin coincides.

In some of these embodiments, the directions of the X, Y, and Z axes of the probe coordinate system are the same as the directions of the X, Y, and Z axes of the joint coordinate system of the distal joint, and the Z axis of the probe coordinate system The axis is collinear with the Z-axis of the joint coordinate system of the end joint.

In some of the embodiments, the end joint of the first serial manipulator is a parallel motion platform, and the parallel motion platform includes a static platform, a moving platform, and a plurality of pieces disposed between the static platform and the moving platform The telescopic assembly, the ultrasonic detection device is set on the moving platform.

In some of the embodiments, the ultrasonic positioning and puncturing system further includes a second serial robotic arm and a puncturing instrument mounted on the end of the second serial robotic arm; the computer equipment is further connected to the second serial robotic arm, respectively. electrically connected, the processor further executes the following steps when running the computer program: taking the second position information of the target point in the reference coordinate system as the target puncture point, and controlling the second serial robotic arm The puncture instrument punctures the target puncture point.

According to various embodiments of the present application, there is provided a storage medium in which a computer program is stored, the computer program being configured to execute the following steps when running: in an ultrasound detection plane projected by the ultrasound detection device Locating the first position information of the target point; obtaining the first attitude information of the ultrasonic detection plane and the second attitude information of the first serial manipulator; according to the first attitude information and the second attitude information, determine The position conversion relationship between the reference coordinate system of the ultrasonic positioning puncture system and the coordinate system where the ultrasonic detection plane is located; according to the position conversion relationship and the first position information, it is determined that the target point is in the reference coordinate system the second location information.

Description of drawings

In order to better describe and illustrate embodiments and/or examples of those inventions disclosed herein, reference may be made to one or more of the accompanying drawings. The additional details or examples used to describe the drawings should not be construed as limiting the scope of any of the disclosed inventions, the presently described embodiments and/or examples, and the best mode presently understood of these inventions.

FIG. 1 is a structural block diagram of an ultrasonic positioning and puncturing system provided by an embodiment of the present application.

FIG. 2 is a flowchart of a working method of the ultrasonic positioning puncture system provided by the embodiment of the present application.

FIG. 3 is a flowchart of a coordinate conversion method for a multi-joint robotic arm according to an embodiment of the present application.

FIG. 4 is a schematic structural diagram of a multi-joint robotic arm provided by a preferred embodiment of the present application.

FIG. 5 is a schematic diagram of a mechanical coordinate system and a joint coordinate system of a multi-joint manipulator provided by a preferred embodiment of the present application.

FIG. 6 is a schematic diagram of a probe coordinate system and a Stewart calculation coordinate system of an ultrasonic detection device according to a preferred embodiment of the present application.

Detailed ways

The ultrasonic positioning puncture system and storage medium provided by the present application will be further described below.

FIG. 1 is a structural block diagram of an ultrasonic positioning and puncturing system provided by an embodiment of the present application. As shown in FIG. 1 , the ultrasonic positioning and puncturing system includes a computer device 101 , a first serial robotic arm 102 , and a device mounted on the end of the first serial robotic arm 102 . The ultrasonic detection device 103 and the computer equipment 101 are electrically connected to the first serial manipulator 102 and the ultrasonic detection device 103 respectively. The computer equipment 101 includes a memory 1011 and a processor 1012, and the memory 1011 stores a computer program 10111.

FIG. 2 is a flowchart of the working method of the ultrasonic positioning and puncturing system provided by the embodiment of the present application. As shown in FIG. 2 , the processor 1012 executes the following steps S201 to S204 when running a computer program.

Step S201: Locating the first position information of the target point in the ultrasonic detection plane projected by the ultrasonic detection device.

The ultrasonic detection plane projected by the ultrasonic detection device 103 is usually in the shape of a fan, and the fan center of the fan is the starting point of the ultrasonic generation. The ultrasonic detection is sent at the fan center and the echoes reflected by the substance are received to detect the target position in the ultrasonic detection plane.

Since the ultrasonic detection plane can only detect the information of one sector, and the position of the target to be punctured is located in a spatial position in the human body, to detect the accurate target position, the first serial machine equipped with the ultrasonic detection device 103 The arm 102 needs to move the ultrasonic detection device 103 to a position near the target point. When moving to a position near the target point, the movement amount of each joint of the first serial robotic arm 102 will change. It is called the second attitude information. In some of the embodiments, the first serial robotic arm includes a plurality of joints, the plurality of joints include: a rotation joint and/or a movement joint, and the second posture information includes motion amount information of each joint.

In some of these embodiments, the ultrasonic detection device is rotatably connected to the end of the first serial robotic arm 102, and the ultrasonic detection plane is formed at the end of the ultrasonic detection device and extends from the end of the ultrasonic detection device along the sector; ultrasonic detection The plane rotates following the ultrasonic detection device, and the rotation axis of the ultrasonic detection plane passes through the end point; the first attitude information includes the rotation angle of the ultrasonic detection plane.

After the ultrasonic detection device 103 is moved to a position near the target point, the ultrasonic detection plane is irradiated toward the target point. In order to display the target point in the ultrasonic detection plane, in the embodiment of the present application, the ultrasonic detection device 103 is rotated along the symmetry axis of the fan plane. , the ultrasonic detection plane rotates accordingly, so as to realize the target detection in the truncated or conical space area, and the ultrasonic image of each ultrasonic detection plane will be displayed on the electronic screen in real time. Once an image consistent with the characteristics of the target point is observed on the electronic screen, the position of the target point in the real ultrasound detection plane can be determined from the position of the target point in the ultrasound image, which is the first position information.

For example, determine the position of the target point in the ultrasonic detection plane, and locate the position information of the target point in the ultrasonic image by acquiring the ultrasonic image collected by the ultrasonic detection device in real time; then according to the position information of the target point in the ultrasonic image and the ultrasonic detection The calibration parameters of the device determine the first position information of the target point in the ultrasonic detection plane projected by the ultrasonic detection device. The calibration parameters of the ultrasonic detection device include, but are not limited to, at least one of the following: the number of array elements, the radius of curvature, the width of the linear array, the field of view (FOV), and the like.

The first position information may be represented in the form of Cartesian coordinates or polar coordinates. For example, in the case of using polar coordinates to represent, the first position information includes the distance of the target point relative to the end point, and the angle between the connecting line between the target point and the end point and the rotation axis.

Step S202: Acquire first attitude information of the ultrasonic detection plane and second attitude information of the first serial manipulator.

Step S203: According to the first attitude information and the second attitude information, determine the position conversion relationship between the reference coordinate system of the ultrasonic positioning puncture system and the coordinate system where the ultrasonic detection plane is located.

Step S204: Determine the second position information of the target point in the reference coordinate system according to the position conversion relationship and the first position information.

When the first attitude information of the ultrasonic detection plane and the second attitude information of the first serial manipulator are determined, the conversion relationship between any point on the ultrasonic detection plane and the reference coordinate system is determined. Therefore, according to the first attitude information and the second attitude information, can determine the position conversion relationship between the reference coordinate system of the ultrasonic positioning puncture system and the coordinate system where the ultrasonic detection plane is located, and then according to the position conversion relationship and the first position information, it can be determined that the target point is in the reference coordinate system. The second position information of , realizes the positioning of the target point in the reference coordinate system.

It should be noted that the steps shown in the above flow or the flow chart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical sequence is shown in the flow chart, in the In some cases, steps shown or described may be performed in an order different from that herein. For example, step S201 may also be performed after step S202 or step S203.

In some of the embodiments, according to the first attitude information and the second attitude information, determining the position conversion relationship between the reference coordinate system of the ultrasonic positioning puncture system and the coordinate system where the ultrasonic detection plane is located includes the following steps.

Step S203-1: When the first serial manipulator is in the initial state, establish the reference coordinate system of the first serial manipulator and the joint coordinate system of each joint of the first serial manipulator, and when the ultrasonic detection device is in the initial state, The probe coordinate system is established with the end point of the ultrasonic detection device as the coordinate origin.

Step S203-2: Obtain the reference coordinate system of the first serial manipulator and the DH parameters of the joint coordinate system of each joint of the first serial manipulator, and determine the reference coordinate system of the first serial manipulator to the first serial manipulator according to the DH parameter. The first position transformation relationship between the joint coordinate systems of the end joints of the serial manipulator.

Step S203-3: Obtain the position information of the coordinate origin of the probe coordinate system in the joint coordinate system of the terminal joint, and the attitude angle of the probe coordinate system relative to the joint coordinate system of the terminal joint, and according to the position information, attitude angle and the first A position conversion relationship, which determines a second position conversion relationship between the reference coordinate system and the probe coordinate system.

Step S203-4: Obtain the rotation angle of the ultrasonic detection plane, and determine a third position conversion relationship between the coordinate system where the ultrasonic detection plane is located and the probe coordinate system according to the rotation angle.

Step S203-5: According to the second position conversion relationship and the third position conversion relationship, determine the position conversion relationship between the reference coordinate system of the ultrasonic positioning and puncturing system and the coordinate system where the ultrasonic detection plane is located.

The above-mentioned step S203-2 aims to determine the first position conversion relationship between the reference coordinate system of the first serial manipulator and the joint coordinate system of the end joint of the first serial manipulator.

Aiming at the problem of complex control of the manipulator in the related art, this embodiment provides a coordinate conversion method for a multi-joint manipulator, which is applied to a manipulator including a rotating joint and a moving joint, and these rotating joints and moving joints are sequentially connected in series . In order to realize the movement of the manipulator with multiple degrees of freedom in space, the number of rotating joints and moving joints can be set according to the required degrees of freedom and motion range, that is, the number of rotating joints and moving joints can be one or more.

FIG. 3 is a flowchart of a coordinate conversion method for a multi-joint robotic arm according to an embodiment of the present application. As shown in FIG. 3 , the flowchart includes the following steps S301 to S303 .

Step S301: When the robot arm is in an initial state, a mechanical coordinate system of the robot arm and a joint coordinate system of each joint of the robot arm are established.

The control of the manipulator is usually based on DH parameters for coordinate system transformation, and DH parameters include traditional DH parameters and improved DH parameters. Two connected joints are adjacent joints. The transformation of the joint coordinate system of two adjacent joints is usually represented by a DH parameter or an improved DH parameter. Taking the DH parameter as an example, two adjacent joint coordinate systems can overlap each other by rotating θ around the Z axis and translating d, and then rotating α around the X axis and translating a. The above θ, d, α and a are DH parameters. It can be seen from this that the simpler the DH parameter is, the simpler the transformation of two adjacent joint coordinate systems will be.

In order to simplify the DH parameters, in this embodiment, the Z axis of the joint coordinate system of the rotating joint is set along the rotation axis, the Z axis of the joint coordinate system of the moving joint is set along the moving direction, and the mechanical coordinate system is the same as the joint coordinate system of each joint. It is a left-handed system or a right-handed system, and when the joint before the rotating joint is a moving joint, the origin of the joint coordinate system of the rotating joint coincides with the origin of the joint coordinate system of the moving joint.

The above-mentioned mechanical coordinate system can also be called a reference coordinate system or a world coordinate system. The mechanical coordinate system is usually set at the center of the base of the manipulator, and is located directly below the first joint of the multi-joint arm, which can make the mechanical coordinate system The transformation relationship to the first joint is as simple as possible.

In a serial manipulator, in most cases, when a rotary joint receives a command with a rotation angle of 0 or 2π, it may not be necessary to distinguish between the two, but it will not rotate according to a preset setting or rotate by 2π in a set direction. However, in some cases, it is necessary to distinguish between these two corners. In the case where the 0 or 2π rotation angle needs to be distinguished, the rotation angle of the Z axis in the DH parameter of the rotary joint is not 0 or 2π, so as to avoid confusing the rotation angle of 0 or 2π.

Step S302: Obtain the mechanical coordinate system of the manipulator and the DH parameters of the joint coordinate system of each joint of the manipulator, and determine the conversion between the mechanical coordinate system of the manipulator and the joint coordinate system of the end joint of the manipulator according to the DH parameters relation.

After the joint coordinate system of each joint is established, the DH parameters between adjacent joints can be obtained in turn, and according to the DH parameters, the conversion relationship between the mechanical coordinate system of the manipulator and the joint coordinate system of the end joint of the manipulator can be determined. .

Among them, according to the DH rule, the homogeneous transformation from the coordinates of the i-1th joint to the coordinates of the ith joint is constructed as a sequence with two rotations and two transformations, which can be expressed as follows using a matrix:

Among them, i=2,3,4,...,n; n is the total number of rotating joints and moving joints of the robotic arm. The DH parameter of the first joint of the manipulator is the transformation of the coordinate system between the first joint and the mechanical coordinate system, which is recorded as

Then the conversion relationship between the mechanical coordinate system and the joint coordinate system of the first joint:

It can be seen that the transformation relationship is exactly the same as the homogeneous transformation form from the coordinates of the i-1th joint to the coordinates of the ith joint.

in getting

and

After that, the transformation matrix from the 0th coordinate system (mechanical coordinate system) to the joint coordinate system of the Nth joint can be expressed as:

Among them, the Nth joint is the end joint.

Step S303: Perform coordinate conversion between the mechanical coordinate system and the joint coordinate system of the end joint according to the conversion relationship.

Determined in the above step S302

It represents the coordinate conversion relationship between the mechanical coordinate system and the joint coordinate system of the end joint. According to the conversion relationship, the coordinate conversion between the mechanical coordinate system and the joint coordinate system of the end joint can be realized.

The embodiments of the present application will be described and illustrated below through preferred embodiments.

FIG. 4 is a schematic structural diagram of a multi-joint mechanical arm provided by a preferred embodiment of the present application. The multi-joint mechanical arm shown in FIG. The moving joint 6 , the rotating joint 7 , the moving joint 8 , the rotating joint 9 and the moving joint 10 , in addition, the multi-joint robotic arm further includes a base 11 fixedly connected with the moving joint 1 .

The coordinate transformation method for a multi-joint robotic arm in this preferred embodiment includes steps 1 to 3, respectively corresponding to steps S301 to S303 in FIG. 3 .

Step 1: According to the rules of the world coordinate system, the mechanical coordinate system and the joint coordinate system of each joint are established at the base of the manipulator.

Referring to Figure 5, the origin of the mechanical coordinate system F ₀ -X ₀ Y ₀ Z ₀ is fixed at the base of the robot arm, the Z ₀ axis points from F ₀ to the moving joint ₁ , and the Y ₀ axis points from the base F ₀ Point to the robotic arm, and the X ₀ axis points to conform to the right-handed coordinate system.

The origin L ₁ of the joint coordinate system L ₁ -X ₁ Y ₁ Z ₁ of the mobile joint 1 is fixed on the mobile joint ₁ , and the orientation of each coordinate axis is the same as that of the corresponding axis of the mechanical coordinate system.

The origin of the joint coordinate system R ₂ -X ₂ Y ₂ Z ₂ of the rotary joint ₂ is fixed on the rotary joint 2 and coincides with L _1. The Z ₂ and Z ₁ axes point in the same direction, the X ₂ axis and the Y ₂ axis The directions are opposite to the X ₁ axis and Y ₁ axis respectively.

The joint coordinate system L ₃ -X ₃ Y ₃ Z ₃ of the mobile joint 3 is fixed at the origin L ₃ of the mobile joint 3, the Z ₃ axis points from the L ₁ point to the L ₃ point, and the X ₃ axis and the Y ₃ axis are respectively connected with the X ₂ Axis and Z ₂ axis point the same.

The joint coordinate system R ₄ -X ₄ Y ₄ Z ₄ of the rotary joint 4 is fixed at the origin R ₄ of the rotary joint 4 and coincides with the point L ₃ (Fig. 5 is a clear indication of the joint coordinate system of the mobile joint 3 and the rotary joint 4). The joint coordinate system of L ₃ and R ₄ are marked separately, the same below), the Z ₄ axis points to the opposite direction to the Y ₃ axis, initially, the X ₄ axis and the Y ₄ axis point to the X ₃ axis and the Z ₃ axis respectively. on the contrary.

The joint coordinate system R ₅ -X ₅ Y ₅ Z ₅ of the rotary joint 5 is fixed at the origin R ₅ of the rotary joint 5. The direction of the Z ₅ axis and the Z ₄ axis are the same. Initially, the X ₅ axis and the Y ₅ axis point respectively. The X ₄ -axis and Y ₄ -axis point opposite.

The joint coordinate system L ₆ -X ₆ Y ₆ Z ₆ of the mobile joint 6 is fixed at the origin L ₆ of the mobile joint 6. The Z ₆ axis points from the R ₅ point to the L ₆ point, and the X ₆ axis and the Y ₆ axis point to the X axis respectively. _{The 5} -axis and the Z ₅ -axis point the same.

The joint coordinate system R ₇ -X ₇ Y ₇ Z ₇ of the rotary joint 7 is fixed at the origin R ₇ of the rotary joint 7 and coincides with the L ₆ point. Initially, the Z ₇ axis and the Y ₇ axis point to the Y ₆ axis respectively. The X ₇ axis and the Z ₆ axis point in the same direction, as opposed to the X ₆ axis.

The origin of the joint coordinate system L ₈ -X ₈ Y ₈ Z ₈ of the mobile joint ₈ is fixed on the mobile joint 8. The Z ₈ axis points from the L ₈ point to the R ₇ point, and the directions of the X ₈ and Y ₈ axes are respectively the same as The X ₇ and Y ₇ axes point the same.

The origin of the joint coordinate system _R9 - _X9Y9Z9 of the rotary joint ₉ is fixed on the rotary joint ₉ and coincides with the _L8 point. _The direction of the _Z9 axis is opposite to the direction of the _Y8 axis. Initially , the direction of the X ₉ axis is opposite to the direction of the Z ₈ axis, and the direction of the Y ₉ axis is the same as the direction of the X8 axis.

The origin of the joint coordinate system L ₁₀ -X ₁₀ Y ₁₀ Z ₁₀ of the mobile joint ₁₀ is fixed on the mobile joint 10. The Z ₁₀ axis points from the R ₉ point to the L ₁₀ point, and the X ₁₀ axis and the Y ₁₀ axis point respectively. The same point as the X ₉ and Z ₉ axes.

The length of F ₀ R ₂ is l ₁ , the length of R ₂ R ₄ is l ₂ , the length of R ₄ R ₅ is l ₃ , the length of R ₅ R ₇ is l ₄ , the length of R ₇ R ₉ is l ₅ , The length of R ₉ L ₁₀ is l ₆ , and the points L ₁ , R ₂ , L ₃ , R ₄ , R ₅ , L ₆ and R ₇ are located on the same horizontal plane.

Step 2: Obtain the DH parameters and calculate the conversion relationship from the mechanical coordinate system to the end joints of the multi-joint robotic arm.

DH parameters are shown in Table 1. In Table 1, in order to avoid confusing the rotation angles 0 and 2π of the rotary joint, a deflection angle is added to the Z-axis rotation angle θ of each rotary joint. For example, the Z-axis rotation angle of the rotary joint 2 is π+θ ₂ , where θ ₂ is the deflection angle.

Table 1. DH parameter table of the multi-joint manipulator shown in Fig. 5

A parallel platform can also be mounted on the end joint of the multi-joint manipulator shown in Figure 4. The parallel platform includes a static platform, a moving platform, and a plurality of telescopic components arranged between the static platform and the moving platform. The calculation coordinate system of the parallel platform It is completely coincident with the joint coordinate system of the end joint of the multi-joint mechanical arm, and the ultrasonic detection device is arranged on the moving platform.

The parallel platform in this embodiment can realize motion with multiple degrees of freedom. Taking the Stewart parallel platform with six degrees of freedom as an example, the Stewart parallel platform includes a static platform, a moving platform, and a plurality of telescopic and retractable platforms arranged between the static platform and the moving platform. The component can realize six degrees of freedom movement in space, namely displacement along the X axis, displacement along the Y axis, displacement along the Z axis, rotation around the X axis, rotation around the Y axis, and rotation around the Z axis. The Stewart parallel platform is supported by 6 telescopic components. Compared with the passive arm using the cantilever beam structure in series, the rigidity is higher and the structure is stable. Due to the high rigidity, the parallel structure has a higher self-weight or volume than the series structure. Carrying capacity. The error at the end of the passive arm using the cantilever beam structure in series is the accumulation and amplification of the errors of each joint, so the error is large and the precision is low. Precision surgical operation. In addition, in terms of position solution, the inverse solution of the parallel platform is very easy, and it is easy to obtain the motion posture of each telescopic component of the parallel platform according to the coordinate position.

Based on the fact that the inverse solution of the parallel platform is very easy, in the above embodiment, the displacement of the master operator can be mapped to the slave user coordinate system by establishing the slave user coordinate system on the static platform, and then the slave user coordinate system can be used to map the displacement of the master operator into the slave user coordinate system. The position information of the target position in the calculation coordinate system can be obtained by the conversion of the system and the calculation coordinate system. According to the position information, the motion posture of each telescopic assembly of the parallel platform can be easily obtained through the inverse solution. The above method greatly reduces the computational complexity, improves the control efficiency, and saves computational resources.

Solve the transformation matrix from the Stewart-calculated coordinate system to the mechanical coordinate system, that is, the transformation matrix and inverse matrix from the moving joint coordinate system L ₁₀ -X ₁₀ Y ₁₀ Z ₁₀ to the mechanical coordinate system F ₀ -X ₀ Y ₀ Z ₀ .

According to the forward kinematics of the robot, the transformation matrix between two adjacent joints is:

The transformation matrix from the mth joint (containing the m degree of freedom) to the nth joint can be expressed as:

The transformation matrix of a single robotic arm from the base to the static platform of the Stewart platform can be solved

That is, the transformation matrix from the mechanical coordinate system to the Stewart calculation coordinate system is named T _{trans_mach_st} , and the transformation matrix from the Stewart calculation coordinate system to the mechanical coordinate system is the inverse matrix

Step 3: According to the transformation matrix and the inverse matrix of the transformation matrix, coordinate transformation between the mechanical coordinate system and the Stewart calculation coordinate system is performed.

In order to simplify the conversion relationship between the probe coordinate system and the reference coordinate system, in some of the embodiments, the directions of the X, Y, and Z axes of the probe coordinate system and the directions of the X, Y, and Z axes of the joint coordinate system of the end joint are The directions are the same, and the Z axis of the probe coordinate system and the Z axis of the joint coordinate system of the end joint are collinear. When the end joint of the first serial manipulator is a parallel platform, the directions of the X, Y, and Z axes of the probe coordinate system are the same as the directions of the X, Y, and Z axes of the moving platform coordinate system, and the Z axis of the probe coordinate system is the same. The axis and the Z axis of the moving platform coordinate system are collinear.

The following describes and illustrates the probe coordinate system and the conversion relationship between the probe coordinate system and the reference coordinate system in the embodiments of the present application by using preferred embodiments.

FIG. 6 is a schematic diagram of a probe coordinate system and a Stewart calculation coordinate system of an ultrasonic detection device according to a preferred embodiment of the present application. A rotary motor is arranged between the ultrasonic detection device and the Stewart moving platform, and the rotation angle of the motor is recorded as θ _m . The end point of the ultrasonic detection device is located on the Z axis of the Stewart moving platform coordinate system. No matter what the value of θ _m is, the position of the end point of the ultrasonic detection device relative to the Stewart moving platform remains unchanged.

A probe coordinate system is established at the end of the ultrasonic detection device. The origin of the probe coordinate system coincides with the end point of the probe, and the motion of the rotating motor between the Stewart moving platform and the ultrasonic detection device is ignored. By default, the XYZ axis of the probe coordinate system is always the same as the Stewart moving platform. The XYZ axes of the coordinate system are parallel. That is, the rotation amount θ _m of the motor between the Stewart moving platform and the ultrasonic detection device is regarded as the movement of the ultrasonic detection plane relative to the probe coordinate system.

It should be noted that the establishment of the probe coordinate system is not unique, and it is not necessary to convert the rotation θ _m of the probe relative to the Stewart moving platform into the rotation of the ultrasonic detection plane relative to the probe coordinate system.

In the case of controlling the Stewart parallel platform through the master-slave mapping method, the motion of the ultrasound probe can be controlled by the master operator. The amount of movement in the horizontal plane of the master manipulator is mapped to the tandem robotic arm. Through the hybrid master-slave mapping algorithm, the position C _{coord_det_st} (three-dimensional column vector) of the end point of the ultrasonic detection device in the coordinate system calculated by Stewart, and the attitude angle of the coordinate system of the probe relative to the coordinate system calculated by Stewart can be obtained

(in

is a fixed value), the transformation matrix T _{trans_det_st} from the probe coordinate system to the Stewart calculation coordinate system can be calculated:

Among them, R _{coord_det_st} is the rotation matrix, using the attitude angle

Expressed as:

The operator can control the mechanical arm with the ultrasonic detection device installed at the end through the main operator, so that the puncture target is located on the ultrasonic detection plane (sound beam plane). The ultrasonic detection plane is a plane that passes through the end point of the ultrasonic detection device and is perpendicular to the XY plane of the probe coordinate system. At any point in the ultrasonic detection plane, the distance l _c relative to the probe coordinate system and the included angle with the Z axis can be measured. θ _c .

Through the three known quantities of l _c , θ _c , and θ _m , the coordinates Sst (Xst, Yst, Zst) of the puncture target in the probe coordinate system can be known, which is denoted as C _{coord_tar_det} :

C _{coord_tar_det} =[l _c sinθ _c cosθ _m l _c sinθ _c sinθ _m l _c cosθ _c 1] ^T .

The coordinate C _{coord_tar_mach} of the target point in the mechanical coordinate system can be obtained by coordinate transformation:

C _{coord_tar_mach} =T _{trans_mach_st} T _{trans_st_det} C _{coord_tar_det} .

In some of the embodiments, the ultrasonic positioning and puncturing system further includes a second serial robotic arm and a puncturing instrument mounted on the end of the second serial robotic arm; the computer equipment is also electrically connected to the second serial robotic arm, and the processor is running The computer program also executes the following steps: taking the second position information of the target in the reference coordinate system as the target puncture point, and controlling the puncture instrument of the second serial robotic arm to puncture the target puncture point.

The coordinate C _{coord_tar_stl} of the target point in the coordinate system calculated by Stewart of the puncture manipulator can be obtained by coordinate transformation:

C _{coord_tar_stl} =T _{trans_stl_mach} C _{coord_tar_mach} .

Among them, T _{trans_stl_mach} is the transformation matrix from the calculated coordinate system of the puncture manipulator Stewart to the calculated coordinate system.

Knowing the coordinates of the target point in the coordinate system calculated by the puncture manipulator Stewart, as well as the preset puncture path and puncture speed, the joint motion of the Stewart platform of the puncture manipulator can be calculated through the inverse kinematics of the Stewart parallel platform. Arm for precise puncture.

Through the above system with two robotic arms, the end of the first serial robotic arm is equipped with an ultrasonic detection device, and the end of the second serial robotic arm is equipped with a puncture instrument, the execution of the puncture and ablation operation is completed by the cooperation of the two robotic arms, and the first serial The arm holds the ultrasound probe, scans the surface of the human body to find the lesions, and locates the puncture target, which can achieve precise positioning of the target. The second serial robotic arm holds the ablation needle and performs puncture and ablation on the target point, which can achieve precise puncture of the target point. The whole process of positioning and puncturing is almost less dependent on human operation and is safer, and the above-mentioned target positioning and puncturing methods are compatible with existing robots with multiple manipulators and a parallel platform at the end of the manipulator, and no independent development is required. complex system.

In addition, in combination with the methods in the foregoing embodiments, the embodiments of the present application further provide a computer-readable storage medium for implementation. A computer program is stored on the computer-readable storage medium; the computer program is configured to perform the following steps when executed.

S1: The first position information of the target point is located in the ultrasonic detection plane projected by the ultrasonic detection device.

S2: Acquire the first attitude information of the ultrasonic detection plane and the second attitude information of the first serial manipulator.

S3: According to the first attitude information and the second attitude information, determine the position conversion relationship between the reference coordinate system of the ultrasonic positioning puncture system and the coordinate system where the ultrasonic detection plane is located.

S4: Determine the second position information of the target point in the reference coordinate system according to the position conversion relationship and the first position information.

In some of these embodiments, locating the first position information of the target point in the ultrasonic detection plane projected by the ultrasonic detection device includes: acquiring an ultrasonic image collected by the ultrasonic detection device, and locating the position information of the target point in the ultrasonic image; The position information of the point in the ultrasonic image and the calibration parameters of the ultrasonic detection device determine the first position information of the target point in the ultrasonic detection plane projected by the ultrasonic detection device.

In some of the embodiments, the first position information includes the distance of the target point relative to the end point, and the angle between the connecting line between the target point and the end point and the rotation axis.

In some of the embodiments, according to the first attitude information and the second attitude information, determining the position conversion relationship between the reference coordinate system of the ultrasonic positioning and puncturing system and the coordinate system where the ultrasonic detection plane is located includes: when the first serial robotic arm is in an initial state , establish the reference coordinate system of the first serial manipulator and the joint coordinate system of each joint of the first serial manipulator, and when the ultrasonic detection device is in the initial state, establish the probe coordinate system with the end point of the ultrasonic detection device as the coordinate origin; Obtain the reference coordinate system of the first serial manipulator and the DH parameters of the joint coordinate system of each joint of the first serial manipulator, and determine the reference coordinate system of the first serial manipulator to the end of the first serial manipulator according to the DH parameters The first position conversion relationship between the joint coordinate systems of the joints; obtain the position information of the coordinate origin of the probe coordinate system in the joint coordinate system of the end joint, and the attitude angle of the probe coordinate system relative to the joint coordinate system of the end joint, and According to the position information, the attitude angle and the first position conversion relationship, determine the second position conversion relationship between the reference coordinate system and the probe coordinate system; obtain the rotation angle of the ultrasonic detection plane, and determine the coordinate system where the ultrasonic detection plane is located according to the rotation angle The third position conversion relationship with the probe coordinate system; according to the second position conversion relationship and the third position conversion relationship, the position conversion relationship between the reference coordinate system of the ultrasonic positioning puncture system and the coordinate system where the ultrasonic detection plane is located is determined.

In some of these embodiments, the first serial manipulator includes a rotating joint and a moving joint, the Z axis of the joint coordinate system of the rotating joint is set along the rotating axis, the Z axis of the joint coordinate system of the moving joint is set along the moving direction, and the reference coordinate system The joint coordinate system of each joint is the same as the left-handed or right-handed system, and when the rotary joint is connected to the previous mobile joint, the origin of the joint coordinate system of the rotary joint coincides with the origin of the joint coordinate system of the previous mobile joint .

In some of these embodiments, the directions of the X, Y, and Z axes of the probe coordinate system are the same as the directions of the X, Y, and Z axes of the joint coordinate system of the end joint, and the Z axis of the probe coordinate system and the joint coordinates of the end joint The Z axis of the system is collinear.

In some of these embodiments, the computer program is further configured to execute the following steps when running: using the second position information of the target point in the reference coordinate system as the target puncture point, and controlling the puncture instrument of the second serial robotic arm to puncture the target puncture point Perform puncture.

It should be noted that, for specific examples in this embodiment, reference may be made to the examples described in the foregoing embodiments and optional implementation manners, and details are not described herein again in this embodiment.

The technical features of the above-described embodiments can be combined arbitrarily. For the sake of brevity, all possible combinations of the technical features in the above-described embodiments are not described. However, as long as there is no contradiction between the combinations of these technical features, All should be regarded as the scope described in this specification.

The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but should not be construed as a limitation on the scope of the patent application. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can also be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.

Claims

An ultrasonic positioning and puncturing system, characterized in that, the ultrasonic positioning and puncturing system comprises computer equipment, a first serial manipulator arm and an ultrasonic detection device mounted on the end of the first serial manipulator arm, and the computer equipment is respectively connected with the The first serial manipulator is electrically connected to the ultrasonic detection device, and the computer device includes a memory and a processor, where a computer program is stored in the memory, and the processor executes the following steps when running the computer program:

Locating the first position information of the target point in the ultrasonic detection plane projected by the ultrasonic detection device;

acquiring the first attitude information of the ultrasonic detection plane and the second attitude information of the first serial manipulator;

According to the first attitude information and the second attitude information, determine the position conversion relationship between the reference coordinate system of the ultrasonic positioning puncture system and the coordinate system where the ultrasonic detection plane is located;

According to the position conversion relationship and the first position information, second position information of the target point in the reference coordinate system is determined.
The ultrasonic positioning and puncturing system according to claim 1, wherein the first position information for locating the target point in the ultrasonic detection plane projected by the ultrasonic detection device comprises:

acquiring an ultrasonic image collected by the ultrasonic detection device, and locating the position information of the target point in the ultrasonic image;

According to the position information of the target point in the ultrasonic image and the calibration parameters of the ultrasonic detection device, the first position information of the target point in the ultrasonic detection plane projected by the ultrasonic detection device is determined.
The ultrasonic positioning and puncturing system according to claim 1, wherein the ultrasonic detection device is rotatably connected to the end of the first serial mechanical arm, and the ultrasonic detection plane is formed at the end of the ultrasonic detection device The end point extends along the sector from the end end point of the ultrasonic detection device; the ultrasonic detection plane rotates following the ultrasonic detection device, and the rotation axis of the ultrasonic detection plane passes through the end end point; the first attitude information includes all the The rotation angle of the ultrasonic detection plane.
The ultrasonic positioning puncture system according to claim 3, wherein the first position information includes the distance of the target point relative to the end point, and the connecting line between the target point and the end point and the The included angle of the rotation axis.
The ultrasonic positioning and puncturing system according to claim 1, wherein the first serial robotic arm includes a plurality of joints, the plurality of joints include: a rotation joint and/or a moving joint, and the second posture information includes Movement amount information for each of the joints.
The ultrasonic positioning and puncturing system according to claim 3, wherein the reference coordinate system of the ultrasonic positioning and puncturing system and the ultrasonic detection plane are determined according to the first attitude information and the second attitude information. The position conversion relationship of the coordinate system where it is located includes:

When the first serial manipulator is in the initial state, the reference coordinate system of the first serial manipulator and the joint coordinate system of each joint of the first serial manipulator are established, and when the ultrasonic detection device is in the initial state In the state, the probe coordinate system is established with the end point of the ultrasonic detection device as the coordinate origin;

Obtain the reference coordinate system of the first serial manipulator and the DH parameters of the joint coordinate system of each joint of the first serial manipulator, and determine the reference coordinate system of the first serial manipulator according to the DH parameters a first position conversion relationship to the joint coordinate system of the end joint of the first serial manipulator;

Obtain the position information of the coordinate origin of the probe coordinate system in the joint coordinate system of the terminal joint, and the attitude angle of the probe coordinate system relative to the joint coordinate system of the terminal joint, and according to the position information, attitude angle and the first position conversion relationship, to determine the second position conversion relationship between the reference coordinate system and the probe coordinate system;

acquiring the rotation angle of the ultrasonic detection plane, and determining a third position conversion relationship between the coordinate system where the ultrasonic detection plane is located and the probe coordinate system according to the rotation angle;

According to the second position conversion relationship and the third position conversion relationship, the position conversion relationship between the reference coordinate system of the ultrasonic positioning and puncturing system and the coordinate system where the ultrasonic detection plane is located is determined.
The ultrasonic positioning and puncturing system according to claim 6, wherein the first serial mechanical arm comprises a rotation joint and a moving joint, the Z axis of the joint coordinate system of the rotating joint is set along the rotation axis, and the moving joint The Z-axis of the joint coordinate system is set along the moving direction, the reference coordinate system and the joint coordinate system of each joint are both left-handed or right-handed systems, and when the rotating joint is connected to the previous moving joint, the joint coordinates of the rotating joint The origin of the system coincides with the origin of the joint coordinate system of the preceding moving joint.
The ultrasonic positioning puncture system according to claim 6, wherein the directions of the X, Y, and Z axes of the probe coordinate system are the same as the directions of the X, Y, and Z axes of the joint coordinate system of the distal joint, And the Z axis of the probe coordinate system and the Z axis of the joint coordinate system of the end joint are collinear.
The ultrasonic positioning and puncturing system according to claim 1, wherein the end joint of the first serial manipulator is a parallel motion platform, and the parallel motion platform comprises a static platform, a moving platform and a plurality of A telescopic assembly between a platform and the moving platform, and the ultrasonic detection device is arranged on the moving platform.
The ultrasonic positioning and puncturing system according to claim 1, wherein the ultrasonic positioning and puncturing system further comprises a second serial robotic arm and a puncturing instrument carried at the end of the second serial mechanical arm; the computer equipment further comprises Electrically connected with the second serial manipulator, the processor also executes the following steps when running the computer program:

The second position information of the target point in the reference coordinate system is used as the target puncture point, and the puncture instrument of the second serial robotic arm is controlled to puncture the target puncture point.
A storage medium, wherein a computer program is stored in the storage medium, wherein the computer program is configured to execute the following steps when running:

Locating the first position information of the target in the ultrasonic detection plane projected by the ultrasonic detection device;

acquiring the first attitude information of the ultrasonic detection plane and the second attitude information of the first serial manipulator;

According to the first attitude information and the second attitude information, determine the position conversion relationship between the reference coordinate system of the ultrasonic positioning puncture system and the coordinate system where the ultrasonic detection plane is located;

According to the position conversion relationship and the first position information, second position information of the target point in the reference coordinate system is determined.