CN114404047A

CN114404047A - Positioning method, system, device, computer equipment and storage medium

Info

Publication number: CN114404047A
Application number: CN202111604601.XA
Authority: CN
Inventors: 何超; 李涛; 于海英; 其他发明人请求不公开姓名
Original assignee: Suzhou Xiaowei Changxing Robot Co ltd
Current assignee: Suzhou Xiaowei Changxing Robot Co ltd
Priority date: 2021-12-24
Filing date: 2021-12-24
Publication date: 2022-04-29
Anticipated expiration: 2041-12-24
Also published as: CN114404047B; WO2023116823A1

Abstract

The application relates to a positioning method, a system, a device, a computer device and a storage medium. The method comprises the following steps: performing osteotomy surface planning according to the image information of the target part to obtain at least one osteotomy surface of the target part; determining a target osteotomy surface from the osteotomy surfaces according to the pose relationship between an osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy surface in the target part; and when the pose relation between the osteotomy guide block device and the target osteotomy surface meets the positioning condition, the mechanical arm is controlled in an automatic mode to position the osteotomy guide block device to the target osteotomy surface. The method can improve the positioning efficiency of the osteotomy surface.

Description

Positioning method, system, device, computer equipment and storage medium

Technical Field

The present application relates to the field of robotics, and in particular, to a positioning method, system, apparatus, computer device, and storage medium.

Background

With the progress of science and technology, the robot can be applied to various fields to assist people in working.

For example: in the medical field, a robot can assist a doctor in diagnosis and treatment and other related operations. Taking an orthopedic surgery as an example, in the process of the orthopedic surgery, a doctor can be assisted by the orthopedic surgery robot to perform the surgery. In the related art, after an osteotomy surface to be positioned needs to be planned by a doctor in an osteotomy process of an orthopedic surgical robot, an operator is instructed to adopt a cooperation mode to roughly place a mechanical arm to the osteotomy surface, an automatic mode is adopted to enable the mechanical arm to execute automatic fine positioning and osteotomy operation, and under the condition that a plurality of osteotomy surfaces exist, the process needs to be repeated for each osteotomy surface until all the osteotomy surfaces complete the osteotomy operation.

Above-mentioned mode of arm location osteotomy face, the locating process is numerous and diverse, and then leads to the location inefficiency.

Disclosure of Invention

In view of the above, it is necessary to provide a positioning method, a system, an apparatus, a computer device, and a storage medium capable of improving the positioning efficiency of a robot arm.

In a first aspect, the present application provides a positioning method, including:

performing osteotomy surface planning according to the image information of the target part to obtain at least one osteotomy surface of the target part;

determining a target osteotomy surface from the osteotomy surfaces according to the pose relationship between an osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy surface in the target part;

and when the pose relation between the osteotomy guide block device and the target osteotomy surface meets the positioning condition, the mechanical arm is controlled in an automatic mode to position the osteotomy guide block device to the target osteotomy surface.

In one embodiment, the performing osteotomy plane planning according to the image information of the target region to obtain at least one osteotomy plane of the target region includes:

determining feature point information of a target part from image information of the target part;

and determining at least one osteotomy surface of the target part according to the characteristic point information.

In one embodiment, the determining at least one osteotomy plane of the target site according to the feature point information includes:

determining at least one initial osteotomy surface of the target part according to the characteristic point information;

and adjusting each initial osteotomy surface according to the prosthesis device of the target part to obtain at least one osteotomy surface of the target part.

In one embodiment, the determining a target osteotomy surface from the osteotomy surfaces according to the pose relationship between the mechanical arm end-mounted osteotomy guide block device and each of the osteotomy surfaces in the target site comprises:

acquiring real-time image information of the target part through image acquisition equipment;

determining an osteotomy region for the target part according to the real-time image information and each osteotomy surface;

when the osteotomy guide block device arranged at the tail end of the mechanical arm is detected to move into the osteotomy area, determining the pose relationship between the osteotomy guide block device and each osteotomy surface;

and determining a target osteotomy surface from the osteotomy surfaces according to the pose relationship between the osteotomy guide block device and each osteotomy surface, wherein the pose relationship is used for representing the distance between the osteotomy guide block device and the osteotomy surface and the presented angular relationship.

In one embodiment, determining an osteotomy region of the target site based on the real-time image information and the respective osteotomy planes comprises:

generating a corresponding initial osteotomy region according to the target part and each osteotomy surface;

according to the real-time image information of the target part, after the target part and a three-dimensional model of the target part are subjected to spatial registration, the three-dimensional model is displayed in a display interface, the three-dimensional model comprises an initial osteotomy region and each osteotomy surface, and each osteotomy surface is located in the initial osteotomy region;

and responding to the adjustment operation of the initial osteotomy region to obtain the osteotomy region of the target part.

In one embodiment, the determining the pose relationship of the osteotomy guide block device to each osteotomy face when the osteotomy guide block device mounted on the end of the mechanical arm is detected to move into the osteotomy region comprises:

acquiring the section pose information of the section of the osteotomy guide block device arranged at the tail end of the mechanical arm under the coordinate system of the image acquisition equipment in real time;

determining whether the osteotomy guide block device moves into the osteotomy region according to the cross-section pose information of the osteotomy guide block device;

and when the osteotomy guide block device is determined to move into the osteotomy guide block area, determining the pose relationship between the osteotomy guide block device and each osteotomy surface according to the section pose information of the osteotomy guide block device and the pose information of each osteotomy surface.

In one embodiment, when it is determined that the osteotomy guide block device moves into the osteotomy guide block region, determining the pose relationship of the osteotomy guide block device to each of the osteotomy surfaces according to the cross-section pose information of the osteotomy guide block device and the pose information of each of the osteotomy surfaces includes:

when the osteotomy guide block device moves into the osteotomy guide block area, detecting whether the mechanical arm has external force control in real time;

and when no external force control exists in the mechanical arm, determining the pose relationship between the osteotomy guide block device and each osteotomy surface according to the section pose information of the osteotomy guide block device and the pose information of each osteotomy surface.

In one embodiment, the determining the pose relationship of the osteotomy guide block device to each of the osteotomy surfaces according to the cross-section pose information of the osteotomy guide block device and the pose information of each of the osteotomy surfaces includes:

aiming at any osteotomy surface, determining the angle and the distance between the osteotomy guide block device and the osteotomy surface according to the cross section pose information of the osteotomy guide block device and the pose information of the osteotomy surface;

and determining the pose relation between the osteotomy guide block device and the osteotomy surface according to the angle and the distance.

In a second aspect, the present application further provides a positioning system, the system comprising: an image acquisition device, a mechanical arm and a bone cutting guide block device arranged at the tail end of the mechanical arm, wherein,

the image acquisition device is used for planning osteotomy surfaces according to the image information of the target part to obtain at least one osteotomy surface of the target part;

the image acquisition device is further used for sending pose information of the target osteotomy surface to the mechanical arm after the target osteotomy surface is determined from the osteotomy surface according to the pose relationship between the osteotomy guide block device and each osteotomy surface in the target part;

the mechanical arm is used for positioning the osteotomy guide block device to the target osteotomy surface according to the pose information of the target osteotomy surface in an automatic mode.

In one embodiment, the image capturing apparatus is further configured to capture real-time image information of a target region, determine an osteotomy region for the target region according to the real-time image information and each of the osteotomy surfaces, and determine a target osteotomy surface from the osteotomy surfaces according to a pose relationship between the osteotomy guide block device and each of the osteotomy surfaces after determining that the osteotomy guide block device moves to the osteotomy region.

In one embodiment, the image capturing apparatus is further configured to acquire, in real time, cross-section pose information of a cross section of the osteotomy guide block device in the image capturing apparatus coordinate system, detect whether the osteotomy guide block device moves into the osteotomy region according to the cross-section pose information, and determine, when it is detected that the osteotomy guide block device moves into the osteotomy guide block region, a pose relationship between the osteotomy guide block device and each of the osteotomy surfaces according to the cross-section pose information of the osteotomy guide block device and the pose information of each of the osteotomy surfaces.

In one embodiment, the osteotomy guide block device comprises a slot, the system further comprises a marking member mounted to an end of the robotic arm;

the image acquisition equipment is further used for determining the pose information of the osteotomy guide block device in the coordinate system of the image acquisition equipment according to the coordinate transformation matrix of the osteotomy guide block device and the marking member and the pose information of the marking member in the coordinate system of the image acquisition equipment after acquiring the pose information of the marking member in the coordinate system of the image acquisition equipment;

the image acquisition equipment is further used for determining the pose information of the section of the osteotomy guide block device in the image acquisition equipment coordinate system according to the coordinate transformation matrix of the osteotomy guide block device and the clamping groove and the pose information of the osteotomy guide block device in the image acquisition equipment coordinate system.

In a third aspect, the present application further provides a positioning device, comprising:

the planning module is used for planning osteotomy surfaces according to the image information of the target part to obtain at least one osteotomy surface of the target part;

the determining module is used for determining a target osteotomy surface from the osteotomy surfaces according to the pose relationship between the osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy surface in the target part;

and the positioning module is used for controlling the mechanical arm to position the osteotomy guide block device to the target osteotomy surface in an automatic mode when the pose relation between the osteotomy guide block device and the target osteotomy surface meets the positioning condition.

In one embodiment, the planning module is further configured to:

In one embodiment, the determining module is further configured to:

In a fourth aspect, the present application further provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the steps of the above positioning method when executing the computer program.

In a fifth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above positioning method.

In a sixth aspect, the present application further provides a computer program product. The computer program product comprises a computer program which, when being executed by a processor, carries out the steps of the above positioning method.

According to the positioning method, the positioning system, the positioning device, the positioning computer equipment and the storage medium, the osteotomy surface planning can be carried out according to the image information of the target part to obtain at least one osteotomy surface of the target part, the target osteotomy surface is determined from the osteotomy surfaces according to the pose relationship between the osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy surface in the target part, and the osteotomy guide block device is positioned to the target osteotomy surface by adopting an automatic mode control mechanical arm when the pose relationship between the osteotomy guide block device and the target osteotomy surface meets the positioning condition. According to the positioning method, the positioning system, the positioning device, the computer equipment and the storage medium, the osteotomy surface aiming at the target part can be planned before the operation, the mechanical arm is controlled to select the target osteotomy surface to perform automatic positioning through visual positioning, namely, the positioning and positioning process is completely automated, the positioning process of the mechanical arm in the osteotomy process can be simplified, the positioning efficiency of the osteotomy surface is improved, the labor cost can be reduced, the time consumption of the osteotomy operation can be reduced, and the operation efficiency of the osteotomy operation can be improved.

Drawings

FIG. 1 is a schematic flow chart of a positioning method in one embodiment;

FIG. 2 is a flow diagram illustrating a positioning method in one embodiment;

FIGS. 3 a-3 c are schematic diagrams of an exemplary positioning method;

FIG. 4 is a flow diagram illustrating a positioning method in one embodiment;

FIG. 5 is a schematic illustration of a positioning method in one embodiment;

FIG. 6 is a schematic illustration of a positioning method in one embodiment;

FIG. 7 is a flow diagram illustrating a positioning method in one embodiment;

FIG. 8 is a flow diagram illustrating a positioning method in one embodiment;

FIG. 9 is a schematic illustration of a positioning method in one embodiment;

FIG. 10 is a schematic illustration of a positioning method in one embodiment;

FIGS. 11 a-11 c are schematic diagrams of an exemplary positioning method;

FIG. 12 is a flow diagram illustrating a positioning method in one embodiment;

FIG. 13 is a schematic illustration of a positioning method in one embodiment;

FIG. 14 is a schematic flow chart diagram of a positioning method in one embodiment;

FIG. 15 is a schematic illustration of a positioning method in one embodiment;

FIG. 16 is a flow diagram illustrating a positioning method in one embodiment;

FIG. 17 is a schematic illustration of a positioning method in one embodiment;

FIG. 18 is a schematic illustration of a positioning method in one embodiment;

FIG. 19 is a schematic illustration of a positioning method in one embodiment;

FIG. 20 is a schematic illustration of a positioning method in one embodiment;

FIG. 21 is a schematic illustration of a positioning method in one embodiment;

FIG. 22 is a schematic illustration of a positioning method in one embodiment;

FIG. 23 is a schematic illustration of a positioning method in one embodiment;

FIG. 24 is a schematic diagram of the positioning system in one embodiment;

FIG. 25 is a schematic view of a positioning system in one embodiment;

FIGS. 26 a-26 b are schematic views of a positioning system in one embodiment;

27 a-27 c are schematic views of a positioning system in one embodiment;

FIG. 28 is a schematic view of a positioning system in one embodiment;

FIG. 29 is a block diagram of the positioning device in one embodiment;

FIG. 30 is a diagram illustrating an internal structure of a computer device according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

In an embodiment, as shown in fig. 1, a positioning method is provided, and this embodiment is illustrated by applying the method to a robot, and it is to be understood that the method may also be applied to a terminal device controlling the robot, and may also be applied to a system including a terminal and a server, and is implemented by interaction between the terminal and the server. In this embodiment, the method includes the steps of:

and 102, planning the osteotomy surface according to the image information of the target part to obtain at least one osteotomy surface of the target part.

In the embodiment of the present application, image information of a target region may be acquired before an osteotomy procedure, and the image information may include images such as a CT (Computed Tomography) scan image, an AR (Augmented Reality) visual scan image, and the like. And may perform osteotomy face planning based on the image information of the target site to plan at least one osteotomy face of the target site. For example: at least one osteotomy face of the target site may be generated based on feature point information of the target site in the image information.

And step 104, determining a target osteotomy surface from the osteotomy surfaces according to the pose relationship between the osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy surface in the target part.

In the embodiment of the application, before the osteotomy operation, at least one osteotomy surface of the target part can be planned, in the osteotomy operation process, an assistance mode can be adopted, medical personnel control the mechanical arm to move towards the osteotomy surface of the target part, or the mechanical arm can also move towards the osteotomy surface of the target part in an automatic mode. In the moving process of the mechanical arm, the pose information of the osteotomy guide block device arranged at the tail end of the mechanical arm can be detected in real time, and then the pose relationship between the pose information and each osteotomy surface in the target part can be determined according to the pose information of the osteotomy guide block device arranged at the tail end of the mechanical arm in real time, wherein the pose relationship can comprise a distance relationship and an angle relationship formed by a cross section.

After the pose relationship between the osteotomy guide block device mounted at the tail end of the mechanical arm and each osteotomy surface in the target part is determined, a target osteotomy surface which is most matched with the pose relationship of the osteotomy guide block device can be selected from the osteotomy surfaces, and illustratively, the osteotomy surface which is closest to the osteotomy guide block device and/or has the smallest angle can be determined as the target osteotomy surface which is most matched with the pose relationship of the osteotomy guide block device.

And 106, when the pose relation between the osteotomy guide block device and the target osteotomy surface meets the positioning condition, positioning the osteotomy guide block device to the target osteotomy surface by adopting the automatic mode control mechanical arm.

In the embodiment of the application, after the target osteotomy surface is determined, the pose relationship between the osteotomy guide block device and the target osteotomy surface can be detected in real time through the image acquisition device, under the condition that the pose relationship between the osteotomy guide block device and the target osteotomy surface meets the preset positioning condition, the pose information of the target osteotomy surface can be sent to the mechanical arm, the mechanical arm is controlled in an automatic mode to perform fine positioning, and the osteotomy guide block device is automatically positioned to the target osteotomy surface according to the pose information of the target osteotomy surface. The positioning condition can include that the distance and/or the angle between the osteotomy guide block device and the target osteotomy surface is smaller than or equal to a preset distance threshold and/or an angle threshold.

For example, after the osteotomy guide block device is positioned to the target osteotomy surface, the mechanical arm may enter a holding state, and a medical worker may mount an oscillating saw or an electric drill using the osteotomy guide groove and/or guide hole on the osteotomy guide block device to perform the osteotomy and drilling operation.

According to the positioning method provided by the embodiment of the application, the osteotomy surface can be planned according to the image information of the target part to obtain at least one osteotomy surface of the target part, the target osteotomy surface is determined from the osteotomy surfaces according to the pose relationship between the osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy surface in the target part, and when the pose relationship between the osteotomy guide block device and the target osteotomy surface meets the positioning condition, the mechanical arm is controlled in an automatic mode to position the osteotomy guide block device to the target osteotomy surface. The positioning method provided by the embodiment of the application can plan the osteotomy surface aiming at the target part before the operation, and control the mechanical arm to select the target osteotomy surface to perform automatic positioning through visual positioning, namely the positioning and positioning process is completely automatic, so that the positioning process of the mechanical arm in the osteotomy process can be simplified, the positioning efficiency of the osteotomy surface is improved, the labor cost can be reduced, the time consumption of the osteotomy operation can be reduced, and the operation efficiency of the osteotomy operation can be improved.

In one embodiment, referring to fig. 2, in step 102, performing osteotomy plane planning according to the image information of the target region to obtain at least one osteotomy plane of the target region, includes:

step 202, determining feature point information of a target part from image information of the target part;

and step 204, determining at least one osteotomy surface of the target part according to the characteristic point information.

In the embodiment of the application, image information of a target part can be acquired before an osteotomy operation, characteristic point information of the target part is determined in the image information of the target part in an image recognition or manual marking mode of a medical staff, and at least one osteotomy surface of the target part is determined in the image information of the target part according to the characteristic point information of the target part. For example, feature point information for generating each osteotomy surface of the target portion may be set in advance, and after the feature point information corresponding to each osteotomy surface is identified, the corresponding osteotomy surface may be generated based on the feature point information.

For example, in a knee joint replacement operation as an example, after image information of a tibia, a femur and a knee joint is acquired through an image device (such as a CT scanning device, an AR visual scanning device), a medical worker may manually perform target area division on the image information to obtain a target area in an image. And obtaining knee joint characteristic points (including femur characteristic points including femur rotation center, knee joint center (femur force line), lateral condyle highest point, medial condyle intercondylar foveal minimum point (through condyle line (TEA)), intercondylar highest point, trochanter lowest point (Whiteside line), greater trochanter cusp, intercondylar fossa midpoint (femur anatomical axis), medial condyle distal end tangent point, lateral condyle distal end tangent point (femur distal axis), posterior condyle medial tangent point, posterior condyle lateral tangent point (posterior condyle axis), tibia characteristic points including talar midpoint, tibial plateau midpoint (tibia knee joint center) (tibia force line), tibia tubercle medial 1/3, posterior ligament cross-stop midpoint (PCL center) (tibia AP line), tibia proximal-medial tangent point, tibia proximal-lateral tangent point (tibia proximal axis) and other characteristic points) from the target area.

Illustratively, referring to fig. 3 a-3 c, with the characteristic points lateral condyle distal tangent 301, medial condyle distal tangent 302, and knee joint center 303, the femoral distal axis and femoral force line 304 may be identified, with the posterior condyle lateral tangent 305 and posterior condyle medial tangent 306, the posterior condyle axis may be identified, and a corresponding osteotomy plane may be generated based on the femoral distal axis, the femoral force line, and the posterior condyle axis. For example, the distal femoral resection and the femoral force line are perpendicular to each other.

According to the positioning method provided by the embodiment of the application, before an osteotomy operation, osteotomy surface planning can be carried out according to image information of a target position, at least one osteotomy surface of the target position is obtained, in order to control the mechanical arm to select the target osteotomy surface to carry out automatic positioning through visual positioning in the osteotomy process, namely, the positioning and positioning processes are all automatic, the positioning process of the mechanical arm in the osteotomy process can be simplified, the positioning efficiency of the osteotomy surface is improved, the labor cost can be further reduced, the time consumption of the osteotomy operation can be further reduced, and the operation efficiency of the osteotomy operation can be improved.

In one embodiment, referring to fig. 4, in step 204, determining at least one osteotomy plane of the target site according to the feature point information comprises:

step 402, determining at least one initial osteotomy surface of the target part according to the characteristic point information;

step 404, adjusting each initial osteotomy face according to the prosthetic device of the target site to obtain at least one osteotomy face of the target site.

In the embodiment of the application, initial planning can be performed based on the feature point information of the target part, at least one initial osteotomy surface of the target part is determined, and each initial osteotomy surface can be adjusted based on the selected prosthesis device of the target part, including adjusting information such as the position and the angle of each initial osteotomy surface, so as to obtain at least one osteotomy surface of the target part.

The prosthesis device may be manually selected by a medical worker, or may also be automatically selected according to the osteotomy plane or the feature point information, and the adjustment of the initial osteotomy plane may be manually adjusted by the medical worker, or may also be automatically adjusted according to the osteotomy plane information corresponding to the prosthesis device, which is not specifically limited in this embodiment of the application.

Illustratively, referring to fig. 5, knee joint image information may be acquired, and a target region may be planned based on the knee joint image information, so as to obtain the target region. And selecting the characteristic points of the target area in the knee joint image information to obtain the characteristic point information of the knee joint image information. After the prosthesis model corresponding to the knee joint is selected, at least one initial osteotomy surface of the knee joint can be generated based on the feature point information, and each initial osteotomy surface is adjusted based on the prosthesis model to obtain each osteotomy surface of the knee joint. For example, each osteotomy can be seen in fig. 6, where 601 is the position of the distal anterior femoral resection, 602 is the position of the distal anterior femoral resection, 603 is the position of the distal posterior femoral resection, 604 is the position of the distal posterior femoral resection, and 605 is the position of the distal femoral resection.

The positioning method provided by the embodiment of the application can plan the osteotomy surface according to the image information of the target part before the osteotomy operation, at least one initial osteotomy surface of the target part is obtained, the initial osteotomy surface is adjusted based on the prosthesis device, the osteotomy surface of the target part is accurately obtained, and then in the osteotomy process, the target osteotomy surface is automatically selected from the planned osteotomy surface through the visual positioning control mechanical arm to carry out automatic positioning, namely, the positioning and positioning process is completely automatic, the positioning process of the mechanical arm in the osteotomy process can be simplified, the positioning efficiency of the osteotomy surface is improved, the labor cost can be reduced, the time consumption of the osteotomy operation can be reduced, and the operation efficiency of the osteotomy operation can be improved.

In one embodiment, referring to fig. 7, in step 104, determining a target osteotomy plane from the osteotomy planes based on the pose relationship of the osteotomy guide block device mounted at the end of the robotic arm to each osteotomy plane in the target site, includes:

step 702, acquiring real-time image information of the target part through image acquisition equipment;

step 704, determining an osteotomy region of the target part according to the real-time image information and each osteotomy surface;

step 706, when it is detected that the osteotomy guide block device mounted at the tail end of the mechanical arm moves into the osteotomy area, determining the pose relationship between the osteotomy guide block device and each osteotomy surface;

and 708, determining a target osteotomy surface from the osteotomy surfaces according to the pose relationship between the osteotomy guide block device and each osteotomy surface, wherein the pose relationship is used for representing the distance between the osteotomy guide block device and the osteotomy surface and the presented angle relationship.

In the embodiment of the application, in the osteotomy operation, before the osteotomy operation is executed, the image acquisition device can acquire the real-time image information of the target part, and then the osteotomy area of the target part is generated according to each osteotomy surface and the position of the target part in the real-time image information. Illustratively, the osteotomy region is centered about a center point of the target site, and each osteotomy face is located within the osteotomy region.

After the osteotomy region is determined, the mechanical arm starts to perform osteotomy positioning to control the osteotomy guide block device to gradually approach the osteotomy region, and it should be noted that, in the osteotomy positioning process, the mechanical arm may move in a cooperation mode or an automatic mode, which is not specifically limited in this embodiment of the present application.

The method comprises the steps of detecting pose information of a osteotomy guide block device arranged at the tail end of a mechanical arm in real time, determining whether the osteotomy guide block device moves into an osteotomy area or not according to the pose information of the osteotomy guide block device, automatically determining a target osteotomy surface to be positioned from a plurality of osteotomy surfaces according to the pose relationship between the osteotomy guide block device and each osteotomy surface when the osteotomy guide block device is detected to move into the osteotomy area, and positioning the target osteotomy surface in an automatic mode. The pose relationship can be determined according to the pose information of the osteotomy guide block device and the pose information of the osteotomy surface, and the pose relationship can be used for representing the distance between the osteotomy guide block device and the osteotomy surface and the presented angle relationship.

According to the positioning method provided by the embodiment of the application, in the osteotomy process, whether the osteotomy guide block device arranged at the tail end of the mechanical arm is located in the osteotomy area or not can be determined through visual positioning, and the target osteotomy surface is automatically selected from the osteotomy area for automatic positioning according to the pose relation between the osteotomy guide block device and each osteotomy surface in the osteotomy area, namely, the coarse positioning and fine positioning processes of the mechanical arm are all automated, so that the positioning process of the mechanical arm in the osteotomy process can be simplified, the positioning efficiency of the osteotomy surface is improved, the labor cost can be reduced, the time consumption of the osteotomy operation can be reduced, and the operation efficiency of the osteotomy operation can be improved.

In one embodiment, referring to fig. 8, the determining an osteotomy region of the target site according to the real-time image information and the respective osteotomy planes in step 704 may include:

step 802, generating a corresponding initial osteotomy region according to the target part and each osteotomy surface;

804, according to the real-time image information of the target part, after spatial registration is carried out on the target part and the three-dimensional model of the target part, the three-dimensional model is displayed in a display interface, the three-dimensional model comprises an initial osteotomy area and osteotomy surfaces, and the osteotomy surfaces are all located in the initial osteotomy area;

in step 806, in response to the adjustment operation for the initial osteotomy region, an osteotomy region of the target site is obtained.

In the embodiment of the present application, a three-dimensional model corresponding to the target portion may be generated based on the feature point information in the image information of the target portion, and the exemplary three-dimensional model is shown in fig. 9. Referring to fig. 10, a target position may be selected from the image information of the target portion, a marking member (e.g., an optical target, etc.) may be disposed at each of the target positions in the target portion of the target object, real-time image information of the target portion may be acquired by an image acquisition device such as NDI, etc., and the real-time image information and the image information may be registered based on the target position to spatially register the target portion with the three-dimensional model of the target portion.

For example, the size of the osteotomy region of the target site may be preset based on prior knowledge of the target site, and then an initial osteotomy region may be generated according to the center of the target site and the preset size of the osteotomy region, and the initial osteotomy region may be displayed in the display interface. Alternatively, the initial osteotomy region may be generated based on each osteotomy surface and the center of the target site, and the generation method of the initial osteotomy region in the embodiment of the present application is not particularly limited.

In one example, still taking knee joint replacement surgery as an example, a calibration position may be selected from CT image information of a target portion, the calibration position includes a plurality of positions such as knee joints, hip joints, and ankle joints, a marking member is disposed at the plurality of positions such as knee joints, hip joints, and ankle joints on the target portion, and real-time image information is acquired by an NDI image acquisition device, so that real-time image information and image information are registered according to the marking member. After the registration is completed, an initial osteotomy region may be generated based on the target site and the osteotomy surfaces, and a three-dimensional model including the initial osteotomy region may be presented in a display interface in which the initial osteotomy region includes the osteotomy surfaces.

In the display interface, the initial osteotomy region may be a polyhedron structure displayed, for example: in various polyhedral structures such as a sphere shown in fig. 11a, a cube shown in fig. 11b, and a cylinder shown in fig. 11c, 1101 in fig. 11a is a osteotomy region, 1102 is a knee joint center point, and 1103 is an osteotomy surface. In the embodiment of the application, the polyhedral structure of the osteotomy region is not specifically limited. After the initial osteotomy region is generated, the size of the initial osteotomy region can be adjusted on the display interface to obtain the osteotomy region of the target site.

Still taking the above example as an example, a corresponding initial osteotomy region may be generated from the knee joint center. The size of the initial osteotomy region can be set according to the size of the knee joint of a patient (for example, a sphere with the center radius of the knee joint of 10-15 cm can be taken generally). The initial osteotomy region can be adjusted according to the position of each osteotomy surface to obtain the osteotomy region of the knee joint. For example: in the process of operating the mechanical arm to coarsely position to each osteotomy surface in the cooperation mode, the osteotomy guide block device is confirmed to be always located in the initial osteotomy area, and when the osteotomy guide block device is found not to be located in the initial osteotomy area, the initial osteotomy area can be adjusted to obtain the adjusted osteotomy area.

The positioning method provided by the embodiment of the application can be used for planning the osteotomy surface and the osteotomy area, and further can be used for determining whether the osteotomy guide block device installed at the tail end of the mechanical arm is positioned in the osteotomy area through visual positioning in the osteotomy process, so that the target osteotomy surface can be automatically selected from the osteotomy surface for automatic positioning, namely, the coarse positioning and fine positioning processes of the mechanical arm are all automated, the positioning process of the mechanical arm in the osteotomy process can be simplified, the positioning efficiency of the osteotomy surface is improved, the labor cost can be reduced, the time consumption of the osteotomy operation can be reduced, and the operation efficiency of the osteotomy operation can be improved.

In one embodiment, referring to fig. 12, in step 806, when it is detected that the bone block resection device mounted on the end of the mechanical arm moves into the bone resection area, determining the pose relationship of the bone block resection device to each bone resection surface may include:

step 1202, acquiring cross-section pose information of a cross section of a bone cutting guide block device arranged at the tail end of a mechanical arm in a coordinate system of image acquisition equipment in real time;

step 1204, determining whether the osteotomy guide block device moves into the osteotomy area according to the cross-section pose information of the osteotomy guide block device;

and 1206, when the osteotomy guide block device is determined to move into the osteotomy guide block area, determining the pose relationship between the osteotomy guide block device and each osteotomy surface according to the cross-section pose information of the osteotomy guide block device and the pose information of each osteotomy surface.

In the embodiment of the application, the registration of the real-time image information acquired by the image acquisition equipment and the image information of the target part is realized in the process, so that in the moving process of the mechanical arm, the pose information of the osteotomy guide block device in the coordinate system of the image acquisition equipment can be obtained in real time according to the marking member fixedly connected with the osteotomy guide block device. Further, the pose information of the osteotomy guide block device in the image acquisition device coordinate system is converted according to the coordinate conversion matrix of the osteotomy guide block device and the slot of the osteotomy guide block device, so that the pose information of the slot of the osteotomy guide block device in the image acquisition device coordinate system can be obtained, and the pose information of the slot in the image acquisition device coordinate system can be used as the section pose information (hereinafter referred to as section pose information) of the section of the osteotomy guide block device in the image acquisition device coordinate system.

After the cross-section pose information of the osteotomy guide block device is obtained, whether the osteotomy guide block device moves into the osteotomy area or not can be determined according to the cross-section pose information of the osteotomy guide block device, and when the osteotomy guide block device is determined to move into the osteotomy area, the pose relationship between the osteotomy guide block device and each osteotomy surface is determined according to the osteotomy pose information of the osteotomy guide block device and the pose information of each osteotomy surface. And determining a target osteotomy surface from each osteotomy surface according to the pose relationship between the osteotomy guide block device and each osteotomy surface, moving the target osteotomy surface to the target osteotomy surface, and starting an automatic mode to perform fine positioning under the condition that the osteotomy guide block device and the target osteotomy surface meet the positioning conditions.

Illustratively, referring to FIG. 13, the robotic arm may be controlled to move toward the osteotomy region (a cooperative mode may be employed, with the robotic arm being manually controlled to move toward the osteotomy regionAnd (f) automatically moving the mechanical arm into the osteotomy region, and detecting the cross-section pose information of the osteotomy guide block device arranged at the tail end of the mechanical arm in real time, wherein the cross-section pose information can obtain the position information X of the osteotomy guide block device in the space, and the osteotomy region is assumed to be (f)_min，f_max) Then at f_min<X<f_maxIn this case, it may be determined that the osteotomy guide block device has moved into the osteotomy region, and the target osteotomy surface may be determined from each osteotomy surface by determining a positional relationship of the osteotomy guide block device to each osteotomy surface.

In one embodiment, referring to fig. 14, when it is determined that the osteotomy guide block device moves into the osteotomy guide block area, the determining the pose relationship of the osteotomy guide block device to each osteotomy surface according to the section pose information of the osteotomy guide block device and the pose information of each osteotomy surface in step 1206 includes:

step 1402, detecting whether the mechanical arm has external force control in real time when the osteotomy guide block device moves into the osteotomy guide block area;

and 1404, when no external force control exists in the mechanical arm, determining the pose relationship between the osteotomy guide block device and each osteotomy surface according to the cross-section pose information of the osteotomy guide block device and the pose information of each osteotomy surface.

In the embodiment of the application, after the osteotomy surface and the osteotomy area are planned, the mechanical arm can intelligently select the target osteotomy surface. The mechanical arm can adopt a cooperation mode or an automatic mode in the process of selecting the target osteotomy surface, and the embodiment of the application is not particularly limited in this respect. When the osteotomy guide block device is moved into the osteotomy guide block region and no longer controlled by external force, the target osteotomy face can be determined from the osteotomy faces.

Illustratively, referring to fig. 15, in the cooperative mode, the robotic arm may be manually guided to move by a human (this process may also be implemented using automatic positioning). Whether the osteotomy guide block device is located in the osteotomy area or not can be judged in real time, and when the osteotomy guide block device is located in the osteotomy area, whether external force control exists in the mechanical arm or not is detected. When the mechanical arm is detected to have external force control, the mechanical arm can move in a flexible mode under the external force control, and whether the osteotomy guide block device is located in the osteotomy area or not is detected in real time in the moving process.

Until the mechanical arm is not controlled by external force and still located in the osteotomy area, the pose relationship between the osteotomy guide block device and each osteotomy surface can be determined, the closest target osteotomy surface can be selected from each osteotomy surface according to the pose relationship, and the mechanical arm can automatically position the target osteotomy surface in an automatic mode.

Therefore, after the osteotomy operation on the target osteotomy surface is completed, when other osteotomy surfaces are positioned, the user can give the mechanical arm an external force control, the mechanical arm continues to move after receiving the external force control, the process is repeated, automatic positioning and osteotomy operation on all the osteotomy surfaces can be realized, and after the positioning and osteotomy operation on all the osteotomy surfaces are completed, the display interface can trigger a stop osteotomy instruction to finish the osteotomy process.

The positioning method provided by the embodiment of the application can determine whether the osteotomy guide block device installed at the tail end of the mechanical arm is located in the osteotomy area through visual positioning so as to automatically select the target osteotomy surface from the osteotomy surface to perform automatic positioning, namely, the coarse positioning and fine positioning processes of the mechanical arm are completely automated, the positioning process of the mechanical arm in the osteotomy process can be simplified, the positioning efficiency of the osteotomy surface is improved, the labor cost can be reduced, the time consumption of the osteotomy operation can be reduced, and the operation efficiency of the osteotomy operation can be improved.

In one embodiment, referring to fig. 16, in step 1206, determining a pose relationship of the osteotomy guide block device to each osteotomy surface according to the cross-section pose information of the osteotomy guide block device and the pose information of each osteotomy surface may include:

step 1602, aiming at any osteotomy surface, determining an angle and a distance between the osteotomy guide block device and the osteotomy surface according to the cross-section pose information of the osteotomy guide block device and the pose information of the osteotomy surface;

and 1604, determining the pose relation between the osteotomy guide block device and the osteotomy surface according to the angle and the distance.

According to the embodiment of the application, the cross section pose information of the osteotomy guide block device can be obtained in real time, and whether the osteotomy guide block device is in the osteotomy area or not can be judged according to the cross section pose information. If the osteotomy guide block device is in the osteotomy region and the robotic arm is not controlled by an external force, an included angle α between a cross-section of the osteotomy guide block device and each osteotomy face (illustratively, each osteotomy face is numbered 1,2,3,4,5,6, respectively, assuming there are 6 osteotomy faces) is calculated_i(exemplary, α)_iTypically in the range of 0 to 45 °, i being an integer greater than 0 and less than 7). Referring to FIG. 17, it is assumed that a direction vector of the distal femoral resection surface 1701 in the image capturing device coordinate system is available in the image capturing device

And the direction vector of the osteotomy guide block apparatus 1702 in the image capturing device coordinate system

Then pass through

Can calculate the angle alpha between the osteotomy guide block device and the ith osteotomy surface_iAnd the distance d between the osteotomy guide block device 1702 and the distal osteotomy surface of the femur can be obtained_i. Further, the function f (alpha) can be judged through the pose relation_i,d_i)＝kα_i+(1-k)d_iDetermining the pose relationship f between the osteotomy guide block device 1702 and the distal femoral osteotomy surface_iWhere k is a weighting factor of angle and distance, and k ∈ (0, 1). By analogy, the pose relationship between the osteotomy guide block device 1702 and each osteotomy surface can be determined, and the pose relationship f can be selected from each osteotomy surface_iThe smallest osteotomy face, which is determined to be closest to the osteotomy guide block device, may be determined as the target osteotomy guide block.

In one example, referring to fig. 18, after the target osteotomy surface is determined, an angle between the target osteotomy surface and the osteotomy guide block device may be displayed in a display interface, and when the angle between the osteotomy guide block device and the target osteotomy surface is smaller than an angle threshold, the user may be prompted by voice, text, AR, or the like. Still taking the example shown in fig. 17 as an example, when the distance and the included angle between the distal end section 1701 of the osteotomy guide block device 1702 and the distal end section 1701 of the femur are the minimum, that is, the distal end section 1701 of the femur is the target osteotomy plane (assuming that the mechanical arm is not controlled by external force at this time), the included angle between the distal end section 1701 of the femur and the section of the osteotomy guide block device 1702 is shown in the display interface. Assuming that the osteotomy guide block device is continuously moved to enable the osteotomy guide block device to be attached to the distal femur anterior resection section until the distance and the included angle between the osteotomy guide block device and the distal femur anterior resection section are minimum, determining that the distal femur anterior resection section is a target osteotomy surface (assuming that the mechanical arm is not controlled by external force at the moment), and displaying the included angle between the distal femur anterior resection section and the section of the osteotomy guide block device in a display interface.

According to the positioning method provided by the embodiment of the application, the pose relation between the osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy surface can be determined through visual positioning, and the target osteotomy surface is selected to be automatically positioned, namely, the coarse positioning and fine positioning processes of the mechanical arm are completely automated, so that the positioning process of the mechanical arm in the osteotomy process can be simplified, the positioning efficiency of the osteotomy surface is improved, the labor cost can be reduced, the time consumption of the osteotomy operation can be reduced, and the operation efficiency of the osteotomy operation can be improved.

In one embodiment, referring to fig. 19, the pose information of the osteotomy guide block device may be detected in real time by the image capturing apparatus, and it is determined whether the osteotomy guide block device is located within the osteotomy region according to the pose information. When the osteotomy guide block is not positioned in the osteotomy area, the mechanical arm can be manually controlled to move (or the mechanical arm can be controlled to move in an automatic positioning mode) until the osteotomy guide block device is detected to be positioned in the osteotomy area, and corresponding prompt information can be displayed in a display interface. At this time, if the robot arm is manually controlled to move, the robot arm can be controlled by a person, automatically positioned by the robot arm, or controlled to move in a compliant mode.

Under the condition that the mechanical arm is not controlled by external force, the pose relation between the osteotomy guide block device and each osteotomy surface can be judged through the image acquisition equipment, a target osteotomy surface is selected from each osteotomy surface according to the pose relation, and the position relation of an included angle between the osteotomy guide block device and the target osteotomy surface is displayed in the display interface. When the included angle between the osteotomy guide block device and the target osteotomy surface is smaller than the included angle threshold value, corresponding prompt information can be displayed in the display interface. At the moment, the mechanical arm can be automatically positioned in an automatic mode, and corresponding prompt information is displayed in a display interface after the target osteotomy surface is positioned.

After the osteotomy guide block device reaches a target osteotomy surface, if fine adjustment is needed, a terminal can be used for sending a fine adjustment instruction, the mechanical arm keeps the current posture unchanged, the mechanical arm can be controlled by a human hand or can be automatically subjected to fine adjustment in the direction of the target osteotomy surface and the normal direction of the target osteotomy surface, when the osteotomy guide block device moves to an osteotomy area in the moving process, bone edge information of a target part can be obtained through image acquisition equipment, the mechanical arm has a collision detection function in the moving process, and human legs or other objects can be prevented from being collided in the process of operating the mechanical arm. When the mechanical arm detects the collision, the mechanical arm stops the motion in the current direction.

In one example, when a human hand drags the mechanical arm to move in the osteotomy region, a compliance positioning mode can be adopted, and a model structure corresponding to the compliance positioning mode can be shown in fig. 20. Fig. 20 shows a compliant positioning mode, where a control torque is output from the controller 2006 according to a difference between the position and speed command 2002 and the position and speed fed back by the actuator encoder 2004, and a total torque is obtained by adding the feedforward gravity compensation 2008 and the friction compensation, and the total torque is sent to each control joint of the robot arm body 2010 to implement the compliant positioning mode.

The structure of the controller 2006 is shown in fig. 21. The difference between the position and speed command 2002 and the speed fed back by the actuator encoder 2004 is multiplied by a coefficient Ki through an integration element, and the difference between the position and speed command 2002 and the speed fed back by the actuator encoder 2004 is directly multiplied by a gain coefficient Kp, and then the two are summed to be output as the torque of the controller 2006.

Alternatively, the structure of the controller 2006 described above may be as shown in fig. 22. Fig. 22 shows that the friction and gravity of each joint of the robot arm body 2010 have been compensated by feedforward compensation, the whole robot arm is in a zero-force state, and when an external force is applied, the robot arm body follows the external force.

When the actuator arm performs automatic positioning in the automatic mode, the controller 2006 is configured as shown in fig. 23, in which the position and speed command 2002 and the position and speed fed back by the actuator encoder 2004 are differentiated in fig. 23, and a PID controller with a speed filter is used to generate a control torque required for the entire positioning process as an output of the controller 2006.

In one embodiment, referring to fig. 24, there is provided a positioning system, comprising: an image acquisition device 2402, a mechanical arm 2404 and a bone cutting guide block device 2406 arranged at the tail end of the mechanical arm 2404, wherein,

the image acquisition device 2402 is used for planning osteotomy surfaces according to the image information of the target part to obtain at least one osteotomy surface of the target part;

the image acquisition device 2402 is further used for sending pose information of the target osteotomy surface to the mechanical arm 2404 after the target osteotomy surface is determined from the osteotomy surface according to the pose relationship between the osteotomy guide block device 2406 and each osteotomy surface in the target part;

the mechanical arm 2404 is used for positioning the osteotomy guide block device to the target osteotomy surface according to the pose information of the target osteotomy surface in the automatic positioning mode.

In this embodiment, the image acquisition device 2402 may be an NDI navigation device. Before the osteotomy, the image acquisition device 2402 may perform osteotomy plane planning through the acquired image information of the target site to obtain at least one osteotomy plane of the target site, and the specific planning process of the osteotomy plane may refer to the related description of the foregoing embodiment, which is not described herein again.

After the osteotomy surface is planned, the image acquisition device 2402 can acquire real-time image information of the target part during the osteotomy operation, realize registration with the image information for planning the osteotomy surface based on a marking member (an optical target and the like) arranged at a marking position of the target part, so as to acquire pose information of the osteotomy guide block device 2406 arranged at the tail end of the mechanical arm 2404 in real time after registration, determine the pose relationship between the osteotomy guide block device 2406 and each osteotomy surface in the target part according to the pose information of the osteotomy guide block device 2406, further determine the target osteotomy surface from the osteotomy surface according to the pose relationship between the osteotomy guide block device 2406 and each osteotomy surface in the target part, and send the pose information of the target osteotomy surface to the mechanical arm 2404.

The mechanical arm 2404 can be automatically positioned in an automatic mode, the target osteotomy surface is positioned according to the pose information of the target osteotomy surface, and after the positioning of the target osteotomy surface is completed, the holding mode is entered, so that a doctor can use a swing saw or an electric drill to perform osteotomy and drilling operations through the osteotomy guide slot and the guide hole of the osteotomy guide block device, and after the osteotomy and drilling operations are completed, the doctor can mount a prosthesis and perform other surgical operations.

According to the positioning system provided by the embodiment of the application, the image acquisition device is used for planning the osteotomy surface according to the image information of the target part to obtain at least one osteotomy surface of the target part, and the image acquisition device is further used for sending the pose information of the target osteotomy surface to the mechanical arm after the target osteotomy surface is determined from the osteotomy surface according to the pose relationship between the osteotomy guide block device and each osteotomy surface in the target part in the osteotomy operation process. The mechanical arm is used for positioning the osteotomy guide block device to the target osteotomy surface according to the pose information of the target osteotomy surface in an automatic mode. The positioning system that this application embodiment provided can plan the osteotomy face to the target site before the operation to through vision positioning control mechanical arm selection target osteotomy face carry out automatic positioning, also the pendulum position and the whole automation of positioning process, can simplify the location flow of osteotomy in-process mechanical arm, improve the location efficiency of osteotomy face, and then can reduce the cost of labor and reduce the consuming time of osteotomy operation, can improve the operation efficiency of osteotomy operation.

In one embodiment, the image capturing device 2402 is further configured to capture real-time image information of the target site, determine an osteotomy region for the target site based on the real-time image information and each osteotomy surface, and determine the target osteotomy surface from the osteotomy surfaces based on a positional relationship of the osteotomy guide block device and each osteotomy surface when the osteotomy guide block device 2406 is determined to move into the osteotomy region.

In this embodiment, the image acquisition device 2402 determines the osteotomy region for the target portion according to the real-time image information and each osteotomy surface, and when it is determined that the osteotomy guide block device 2406 moves into the osteotomy region, the specific process of determining the target osteotomy surface from the osteotomy surface according to the pose relationship between the osteotomy guide block device and each osteotomy surface may refer to the description of the foregoing embodiment, which is not described herein again in this embodiment.

In one embodiment, the image capturing device 2402 is further configured to obtain, in real time, cross-sectional pose information of the cross-section of the osteotomy guide block device 2406 in the image capturing device coordinate system, detect whether the osteotomy guide block device 2406 moves into the osteotomy region according to the cross-sectional pose information, and determine the pose relationship between the osteotomy guide block device 2406 and each osteotomy surface according to the cross-sectional pose information of the osteotomy guide block device 2406 and the pose information of each osteotomy surface when the osteotomy guide block device 2406 is detected to move into the osteotomy guide block region.

In this embodiment, the above operation process of the image acquisition device 2402 may specifically refer to the related description of the foregoing embodiment, and this embodiment is not described herein again.

In one embodiment, the osteotomy guide block device 2406 includes a slot, the system further includes a marker member 2408 mounted to the end of the robotic arm;

the image acquisition equipment 2402 is further configured to determine the pose information of the osteotomy guide block device 2406 in the image acquisition equipment coordinate system according to the coordinate transformation matrix of the osteotomy guide block device 2406 and the marking member 2408 and the pose information of the marking member 2408 in the image acquisition equipment coordinate system after acquiring the pose information of the marking member 2408 in the image acquisition equipment coordinate system;

the image acquisition device 2402 is further configured to determine pose information of the cross section of the osteotomy guide block device 2406 in the image acquisition device coordinate system according to the coordinate transformation matrix of the osteotomy guide block device 2406 and the card slot and the pose information of the osteotomy guide block device 2406 in the image acquisition device coordinate system.

In the subject embodiment, the osteotomy guide block device 2406 includes a slot, as shown with reference to fig. 25. The osteotomy guide block device 2406 may have a plurality of slots, or may be a single slot, where the multi-slot osteotomy guide block device 2406 may be shown in fig. 26a, and the single-slot osteotomy guide block device 2406 may be shown in fig. 26 b. The number of the slots of the osteotomy guide block device 2406 is not particularly limited in the embodiment of the present application. The osteotomy guide block device 2406 may be an osteotomy guide block without a degree of freedom, or a multi-degree-of-freedom osteotomy guide block as shown in fig. 27a to 27c, and the degree of freedom of the osteotomy guide block device in the embodiment of the present application is not particularly limited.

Referring to fig. 24, the osteotomy guide block device 2406 may be an adjustable device with multiple degrees of freedom, or may be a guide block device with only one osteotomy slot or multiple osteotomy slots.

In the robot arm, the coordinate conversion relationship between the osteotomy guide device 2406 and the marker member 2408 is fixed, and the coordinate conversion relationship between the osteotomy guide device 2406 and the marker member 2408 may be acquired in advance

The pose information of the marking member 2408 in the image capturing apparatus 2402 can be obtained by the image capturing apparatus 2402 as

The pose information of the osteotomy guide block device 2406 in the image acquisition device 2402 can be obtained through conversion

Comprises the following steps:

reference toFIG. 25 shows the coordinate system transformation relationship of the slot 2502 in the coordinate system of the osteotomy guide block device 2406 obtained by measurement and calibration

The cross-section pose information of the osteotomy guide block device 2406 can be obtained in real time along with the movement of the mechanical arm 2404

Comprises the following steps:

when the image acquisition equipment is used for section planning, the section pose information of each osteotomy surface can be obtained

Knowledge of cross-sectional pose information

From the position information of the osteotomy face

And posture information of osteotomy face

Is composed of, i.e.

Similarly, the cross-sectional pose information of the osteotomy guide block device 2406

Also from the position information of the osteotomy guide block device 2406

And attitude information

The composition, that is,

then can be based on

Whether the osteotomy guide block device 2406 is within the osteotomy region is determined within the boundary of the osteotomy region, and when the osteotomy guide block device 2406 is determined to be within the osteotomy region, the position information of the osteotomy face may be based on

And cross-sectional position information of the osteotomy guide block device 2406

Determining the distance d between the osteotomy guide block device 2406 and the osteotomy face_iThe specific process can refer to the following formula (one).

By analogy, the section attitude information of the osteotomy guide block device 2406 is determined

Posture information of osteotomy surface

And the direction vector of the known osteotomy guide block device 2406 in the osteotomy guide block device coordinate system is

The direction vector of the osteotomy surface in the coordinate system of the target part is

Then the included angle alpha between the section attitude information of the osteotomy guide block device 2406 and the attitude information of the osteotomy surface can be calculated_iThe specific process can refer to the following formula (two).

For indicating the amount of orientation

The die of (a) is used,

for indicating the amount of orientation

The die of (1).

Obtaining the included angle alpha between the attitude information of the section of the osteotomy guide block device 2406 and the attitude information of the osteotomy surface_iAnd a distance d_iThen, the included angle alpha between the section attitude information of the osteotomy guide block device 2406 and the attitude information of the osteotomy surface can be converted according to the pose relationship and_iand distance, determining the pose relationship between the osteotomy guide block device 2406 and the osteotomy surface, and the embodiment of the present application does not specifically limit the pose relationship conversion function, and exemplarily can refer to the formula (iii).

f(α_i,d_i)＝k*α_i+(1-k)*d_iFormula (III)

The positioning system provided by the embodiment of the application can obtain the characteristic point information of the target part skeleton through the image acquisition device, and further sets the osteotomy area and the corresponding osteotomy surface according to the characteristic point information of the target part skeleton. Whether the osteotomy guide block device reaches the osteotomy area is determined by identifying the osteotomy guide block device and the position information of the edge of the osteotomy area. When the osteotomy guide block device is determined to be located in the osteotomy area, the distance and the angle between the osteotomy guide block device and each osteotomy surface are determined, the target osteotomy surface is selected according to the distance and the angle between the osteotomy guide block device and each osteotomy surface, the mechanical arm is operated in a flexible mode to enable the osteotomy guide block device to slowly approach the target osteotomy surface, when the set included angle between the target osteotomy surface and the osteotomy guide block device reaches the threshold value of the included angle, the mechanical arm automatically positions the target osteotomy surface in an automatic mode, and after the target osteotomy surface is positioned and corresponding osteotomy operation is completed, external force can be applied to the osteotomy guide block device to repeat the process, so that the positioning of other osteotomy surfaces is completed.

According to the invention, the corresponding osteotomy area and the corresponding osteotomy surface in the operation process are obtained through the vision system, the target osteotomy surface is automatically selected by detecting the pose relation between the osteotomy guide block device and the osteotomy surface, and the accurate positioning of the osteotomy guide block device is realized by adopting a mode of automatically controlling the mechanical arm to automatically position to the target osteotomy surface under the condition that the osteotomy guide block device and the target osteotomy surface meet the positioning condition, so that the positioning process is simplified, the use interaction is optimized, and the time consumed by the operation can be reduced.

In order to better understand the application embodiments, the embodiments are described below by specific examples.

Referring to fig. 28, a knee joint replacement operation is exemplified. Fig. 28 includes surgical trolley 2801, robotic arm 2802, tool target 2803, resection guide 2804, pendulum saw 2805, NDI navigation device 2806, auxiliary display 2807, main display 2808, navigation trolley 2809, keyboard 2810, femoral target 2811, femur 2812, tibia target 2813, tibia 2814, and base target 2815.

The surgical and

navigation trolleys

2801, 2809 may be placed in the appropriate position alongside the hospital bed and the femoral target 2811, tibial target 2812, base target 2815, sterile bag, osteotomy guide block device 2804, tool target 2803, etc. installed. The doctor guides the patient bone CT scanning model into a computer to plan the preoperative osteotomy surface, for example: planning the planar coordinates of the osteotomy face and selecting the appropriate model prosthesis and adjusting the osteotomy face, the computer includes a main display 8, a keyboard 10 and controls located in a navigation trolley 9.

The doctor uses the target pen to click the characteristic points of the femur and the tibia of the patient, the NDI navigation equipment 2806 takes the base target 2815 as a reference, the positions of the bone characteristic points of the patient are recorded according to the click of the target pen, the positions of the bone characteristic points are sent to the computer, then the computer obtains the actual positions of the femur and the tibia through a characteristic matching algorithm and corresponds to the positions of the CT images of the femur and the tibia, and then the navigation system associates the actual positions of the femur and the tibia with the corresponding targets installed on the femur and the tibia, so that the actual positions of the bone can be tracked in real time by the femur target and the tibia target.

The NDI navigation apparatus 2806 determines a target osteotomy surface according to the pose information of the osteotomy guide block device 2804 and the pose information of the osteotomy surface, and sends the coordinates of the osteotomy plane of the target osteotomy surface planned before the operation to the mechanical arm 2802, the mechanical arm 2802 automatically positions the target osteotomy surface through a tool target (mounted on the mechanical arm or the osteotomy guide block device) and moves to a predetermined position, the mechanical arm enters a holding state, a doctor can perform osteotomy and drilling operations through an osteotomy guide slot and a guide hole of the osteotomy guide block module by using a swing saw or an electric drill, and after the osteotomy and drilling operations are completed, the doctor can mount a prosthesis and perform other surgical operations.

The positioning method and the positioning system provided by the embodiment of the application can simplify the osteotomy operation process, the mechanical arm can be directly assisted by medical personnel, the automatic positioning of the osteotomy surface can be completed, the whole process can be realized without extra operator to perform back-and-forth cutting, the selection of the cutting surface can be automatically determined by the operator or the medical personnel or automatically selected by the mechanical arm, the osteotomy guide block device of the mechanical arm is placed to the osteotomy area, the mechanical arm can select the target osteotomy surface for automatic positioning, the positioning method and the positioning system provided by the embodiment of the application are beneficial to the use of the medical personnel, the interaction is better, and the operation time can be shortened.

It should be understood that, although the steps in the flowcharts related to the embodiments as described above are sequentially displayed as indicated by arrows, the steps are not necessarily performed sequentially as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the flowcharts related to the embodiments described above may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the execution order of the steps or stages is not necessarily sequential, but may be rotated or alternated with other steps or at least a part of the steps or stages in other steps.

Based on the same inventive concept, the embodiment of the present application further provides a positioning apparatus for implementing the above-mentioned positioning method. The solution of the problem provided by the device is similar to the solution described in the above method, so the specific limitations in one or more embodiments of the positioning device provided below can refer to the limitations on the positioning method in the above, and are not described herein again.

In one embodiment, as shown in fig. 29, there is provided a positioning device including: planning module 2902, determination module 2904, and positioning module 2906, wherein:

a planning module 2902, configured to perform osteotomy plane planning according to the image information of the target portion, to obtain at least one osteotomy plane of the target portion;

a determining module 2904, configured to determine a target osteotomy surface from the osteotomy surfaces according to the pose relationship between the osteotomy guide block device mounted at the end of the mechanical arm and each of the osteotomy surfaces in the target portion;

a positioning module 2906, configured to, when the pose relationship between the osteotomy guide block device and the target osteotomy surface satisfies a positioning condition, control the mechanical arm to position the osteotomy guide block device to the target osteotomy surface in an automatic mode.

The positioning device can plan the osteotomy surface according to the image information of the target part to obtain at least one osteotomy surface of the target part, determines the target osteotomy surface from the osteotomy surfaces according to the pose relationship between the osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy surface in the target part, and controls the mechanical arm to position the osteotomy guide block device to the target osteotomy surface in an automatic mode when the pose relationship between the osteotomy guide block device and the target osteotomy surface meets the positioning condition. The positioning device provided by the embodiment of the application can plan the osteotomy face aiming at the target position before the operation, and control the mechanical arm to select the target osteotomy face to carry out automatic positioning through visual positioning, namely the whole automation of the positioning and positioning process, can simplify the positioning process of the mechanical arm in the osteotomy process, improve the positioning efficiency of the osteotomy face, and then can reduce the labor cost and reduce the time consumption of the osteotomy operation, and can improve the operation efficiency of the osteotomy operation.

In one embodiment, the planning module 2902 is further configured to:

In one embodiment, the determining module 2904 is further configured to:

determining an osteotomy region of the target part according to the real-time image information and each osteotomy surface;

In one embodiment, the determining module 2904 is further configured to:

The modules in the positioning device can be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 30. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a positioning method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

Those skilled in the art will appreciate that the architecture shown in fig. 30 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is further provided, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method embodiments when executing the computer program.

In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.

In an embodiment, a computer program product is provided, comprising a computer program which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.

It should be noted that, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high-density embedded nonvolatile Memory, resistive Random Access Memory (ReRAM), Magnetic Random Access Memory (MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (PCM), graphene Memory, and the like. Volatile Memory can include Random Access Memory (RAM), external cache Memory, and the like. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others. The databases referred to in various embodiments provided herein may include at least one of relational and non-relational databases. The non-relational database may include, but is not limited to, a block chain based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing based data processing logic devices, etc., without limitation.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A method of positioning, the method comprising:

2. The method of claim 1, wherein the osteotomy face planning from the image information of the target site to obtain at least one osteotomy face of the target site comprises:

3. The method of claim 2, wherein determining at least one osteotomy plane of the target site from the feature point information comprises:

4. The method according to any one of claims 1 to 3, wherein the determining a target osteotomy face from the osteotomy faces according to the positional relationship of the robotic arm tip mounted osteotomy guide block device to each of the osteotomy faces in the target site comprises:

5. The method of claim 4, wherein determining an osteotomy region of the target site from the real-time image information and each of the osteotomy surfaces comprises:

6. The method of claim 4, wherein determining the positional relationship of the osteotomy guide block device to each of the osteotomy faces upon detecting movement of the mechanical arm tip mounted osteotomy guide block device into the osteotomy region comprises:

7. The method of claim 6, wherein determining the pose relationship of the osteotomy guide block device to each of the osteotomy surfaces based on the cross-sectional pose information of the osteotomy guide block device and the pose information of each of the osteotomy surfaces when it is determined that the osteotomy guide block device is moved into the osteotomy guide block region comprises:

8. The method according to claim 6 or 7, wherein determining the pose relationship of the osteotomy guide block device to each of the osteotomy surfaces according to the cross-section pose information of the osteotomy guide block device and the pose information of each of the osteotomy surfaces comprises:

9. A positioning system, characterized in that the system comprises: an image acquisition device, a mechanical arm and a bone cutting guide block device arranged at the tail end of the mechanical arm, wherein,

10. The system of claim 9, wherein the image capturing device is further configured to capture real-time image information of the target site, determine an osteotomy region for the target site based on the real-time image information and each of the osteotomy surfaces, and determine a target osteotomy surface from the osteotomy surfaces based on a positional relationship between the osteotomy guide block device and each of the osteotomy surfaces when the osteotomy guide block device is determined to move into the osteotomy region.

11. The system of claim 10, wherein the image capturing device is further configured to obtain cross-sectional pose information of the cross-section of the osteotomy guide block device in the image capturing device coordinate system in real time, detect whether the osteotomy guide block device moves into the osteotomy region according to the cross-sectional pose information, and determine the pose relationship between the osteotomy guide block device and each of the osteotomy surfaces according to the cross-sectional pose information of the osteotomy guide block device and the pose information of each of the osteotomy surfaces when the osteotomy guide block device is detected to move into the osteotomy guide block region.

12. The system of claim 11, wherein the osteotomy guide block device comprises a slot, the system further comprising a marking member mounted to a distal end of the robotic arm;

13. A positioning device, the device comprising:

the determination module is used for determining a target osteotomy surface from the osteotomy surfaces according to the pose relationship between the osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy surface in the target part;

14. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 8.

15. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 8.