WO2022188651A1

WO2022188651A1 - Surgical system

Info

Publication number: WO2022188651A1
Application number: PCT/CN2022/078302
Authority: WO
Inventors: 常兆华; 何超; 陈浩; 常新朝; 何洪军; 周佳音
Original assignee: 上海微创医疗机器人（集团）股份有限公司
Priority date: 2021-03-12
Filing date: 2022-02-28
Publication date: 2022-09-15
Also published as: CN113057734A

Abstract

The present disclosure provides a surgical system, comprising a control device and a surgical apparatus. The control device comprises a lesion recognition module, an automatic planning module, and a control module. The lesion recognition module is configured to obtain real-time lesion information according to an intra-operative real-time medical image and a pre-operative medical image. The automatic planning module is configured to plan a surgical path according to the real-time lesion information, so as to obtain a target surgical path. The control module is configured to control, according to a surgical operation parameter and the obtained target surgical path, the surgical apparatus to perform a surgery. The present invention has the advantages of being accurate, reliable, safe, and efficient, can avoid numerous risks caused by the fact that a surgical process completely relies on a doctor's clinical experience, and can also effectively reduce the workload of a doctor and improve the working efficiency of a doctor.

Description

a surgical system

technical field

The present invention relates to the field of medical technology, in particular to a surgical system.

Background technique

Cryoablation mainly uses low-temperature instruments to control the process of cooling, freezing, and rewarming the lesion tissue, thereby causing irreversible damage or even necrosis of cells. For example, the killing mechanisms of cryoablation on tumors are: cell dehydration and shrinkage; intracellular ice crystal formation and mechanical damage to ice crystals; cellular electrolyte toxicity concentration and pH change; cell membrane lipoprotein composition degeneration; immune effects, etc. Cryoablation surgery is not only less traumatic, but also has the advantages of anesthesia, less postoperative complications, and prevention of tumor spread. It is well received by doctors and patients.

At present, the identification of lesions in cryotherapy or puncture surgery is mainly based on preoperative Magnetic Resonance (MR) and CT images to determine the location and volume of lesions. However, there is a single image, incomplete lesion display, and imaging time and operation time. , the current lesions and other issues cannot be fully displayed. Doctors rely on preoperative images to determine the lesion, and then plan the puncture path based on clinical experience, determine freezing parameters, and perform cryoablation. Since the whole process cannot be monitored, the surgical process is completely dependent on the clinical experience of the doctor, and there are many risks.

It should be noted that the information disclosed in this Background of the Invention section is only intended to deepen the understanding of the general background of the present invention, and should not be construed as an acknowledgement or implied in any form that the information constitutes already known to those skilled in the art current technology.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a surgical system, which can solve the problem that in the prior art, doctors rely on preoperative images to determine the lesions, plan the surgical path and determine the surgical parameters according to clinical experience, and then perform the surgery, so that the entire surgical process cannot be monitored in real time, and The entire surgical procedure is completely dependent on the clinical experience of doctors, and there are many risks.

In order to solve the above technical problems, the present invention provides a surgical system, including a control device and a surgical device, wherein the surgical device is connected in communication with the control device;

The control device includes a lesion identification module, an automatic planning module and a control module connected in communication;

Wherein, the lesion identification module is used to acquire real-time lesion information according to the intraoperative real-time medical image and the preoperative medical image,

The automatic planning module is used for planning a surgical path according to the real-time lesion information, so as to obtain a target surgical path, and

The control module is configured to control the surgical device to perform surgery according to the surgical operation parameters and the acquired target surgical path. Optionally, the surgical system further includes a first image acquisition device, the first image acquisition device is connected in communication with the control device, and is used for acquiring intraoperative real-time medical images.

Optionally, the lesion identification module includes an image acquisition unit, an image registration unit and a lesion identification unit that are connected in communication;

Wherein, the image acquisition unit is used to acquire preoperative medical images and intraoperative real-time medical images,

the image registration unit for registering the preoperative medical image and the intraoperative real-time medical image to obtain a real-time registered image, and

The lesion identification unit is configured to acquire real-time lesion information according to the real-time registration image.

Optionally, if the preoperative medical image is a CT image or an MRI image, and the intraoperative real-time medical image is a CT image or an MRI image, the image registration unit obtains a real-time registration image, including:

performing three-dimensional modeling on the preoperative medical image to obtain a preoperative three-dimensional medical image;

performing three-dimensional modeling on the intraoperative real-time medical image to obtain the intraoperative real-time three-dimensional medical image;

The preoperative three-dimensional medical image is registered to the intraoperative real-time three-dimensional medical image to obtain a real-time registered image.

Optionally, if the preoperative medical image is a CT image or an MRI image, and the intraoperative real-time medical image is an ultrasound image, the image registration unit obtains a real-time registration image, including:

registering the preoperative three-dimensional medical image to the intraoperative real-time medical image to obtain a first real-time registered image;

The intraoperative real-time medical image is registered to the pre-operative three-dimensional medical image to obtain a second real-time registered image.

Optionally, the surgical system further includes a second image acquisition device that is communicatively connected to the control device, and the second image acquisition device is used to acquire intraoperative real-time patient skin images;

The image acquisition unit is also used to acquire intraoperative real-time patient skin images;

The image registration unit acquires real-time registration images, including:

performing three-dimensional modeling on the intraoperative real-time patient skin image to obtain intraoperative real-time human body model images;

performing registration and fusion on the preoperative three-dimensional medical image and the intraoperative real-time three-dimensional medical image to obtain a first real-time fusion image;

The first real-time fusion image is registered to the intraoperative real-time human model image to obtain a real-time registered image.

Optionally, the lesion identification unit obtains real-time lesion information, including:

A pre-trained deep neural network model is used to identify lesions on the registration image, so as to obtain real-time lesion information.

Segment the preoperative three-dimensional medical image by using a pre-trained deep neural network model to obtain segmented images;

fusing the segmented image with the first real-time registration image to obtain a second real-time fused image;

fusing the segmented image with the second real-time registration image to obtain a third real-time fused image;

Fusing the second real-time fusion image with the third real-time fusion image to obtain a fourth real-time fusion image;

According to the fourth real-time fusion image, real-time lesion information is acquired.

Optionally, the real-time lesion information includes real-time lesion location information. Further optionally, the real-time lesion information further includes one or more of real-time lesion volume information, real-time lesion shape information, and real-time key organ tissue information.

Optionally, the automatic planning module obtains the target surgical path, including:

According to the real-time lesion information, obtain the location information of at least one lesion target point and multiple puncture points;

According to preset conditions, each of the puncture points is evaluated to obtain the target puncture point;

Connect the corresponding lesion target point and the target puncture point to obtain the target puncture path.

Optionally, according to the real-time lesion information, obtaining the location information of at least one lesion target point and multiple puncture points, including:

Obtain the spatial mapping relationship between the image coordinate system and the surgical equipment coordinate system;

According to the spatial mapping relationship and the real-time lesion information, obtain the real-time position information of the lesion in the coordinate system of the surgical equipment;

According to the real-time position information of the lesion in the coordinate system of the surgical device, the position information of at least one target point of the lesion and the multiple puncture points in the coordinate system of the surgical device is acquired.

Optionally, evaluating each of the puncture points to obtain a target puncture point according to preset conditions, including:

Step A, score each described puncture point according to a preset scoring criterion, and use the puncture point with the highest score as the target puncture point;

Step B, judging whether the target puncture point can cover all lesions;

If not, then perform step C;

In step C, each non-target puncture point is scored according to a preset scoring criterion, and the non-target puncture point with the highest score is used as the target puncture point;

Step D, judging whether all the target puncture points can cover all the lesions together;

If not, repeat steps C and D until all the target puncture points can cover the lesion together.

Optionally, scoring each of the puncture points according to a preset scoring criterion, and using the puncture point with the highest score as the target puncture point, including:

i Score each of the puncture points respectively according to a plurality of preset scoring criteria, so as to obtain various scores of each of the puncture points;

ii Calculate the comprehensive score of each described puncture point according to the corresponding weights of the preset scoring criteria;

iii Take the puncture point with the highest comprehensive score as the target puncture point;

The described scoring is performed on each non-target puncture point according to the preset scoring criteria, and the non-target puncture point with the highest score is used as the target puncture point, including:

i Score each non-target puncture point respectively according to a plurality of pre-set scoring criteria, so as to obtain various scores of each of the non-target puncture points;

ii Calculate the comprehensive score of each of the non-target puncture points according to the weights corresponding to the preset scoring criteria;

iii Take the non-target puncture point with the highest comprehensive score as the target puncture point.

Optionally, the control device further includes a functional safety module that is communicatively connected to the lesion identification module, and the functional safety module is configured to perform real-time monitoring of the surgical equipment according to the registration image output by the image registration unit. The movement trajectory is monitored.

Optionally, the functional safety module is further configured to acquire safety operation boundary information according to the real-time lesion information, and determine whether the real-time motion trajectory of the surgical device exceeds the safety operation boundary area according to the safety operation boundary information.

Optionally, the automatic planning module is further configured to plan surgical operation parameters according to the real-time lesion information. For example, the automatic planning module may be used to plan the surgical operation parameters by modifying the preoperatively determined surgical operation parameters in real time according to the real-time lesion information.

Optionally, the surgical equipment includes a driving unit and a surgical instrument, the surgical instrument is mounted on the driving unit, and the control module is configured to control the driving according to surgical operation parameters and the acquired target surgical path. The unit drives the surgical instrument to perform surgery.

Optionally, the driving unit is a robotic arm, and a fixator for fixing the surgical instrument is installed at the end of the robotic arm.

Optionally, the surgical instrument is an instrument for performing a puncture operation;

The automatic planning module is configured to plan the puncture path according to the real-time lesion information, so as to obtain the target puncture path;

The control module is configured to control the surgical device to perform a puncture operation according to the surgical operation parameters and the acquired target puncture path.

Optionally, the surgical instrument is a cryoablation needle, the surgical device further includes a refrigeration device, and the control module is configured to control the refrigeration device to provide a cold source to the cryoablation needle according to the surgical operation parameter.

Optionally, the surgical operation parameters include freezing time, freezing cycle times and freezing dose.

Optionally, the surgical system further includes a human-computer interaction module communicatively connected to the control device, and the human-computer interaction module is used for data display and interaction.

Optionally, the control device further includes a data storage module, and the data storage module is used for data storage and management.

Optionally, the first image acquisition device is an ultrasound instrument, and the surgical system further includes a bracket, the bracket includes a base and a first fixing device and a second fixing device mounted on the base, the The first fixing device is used for fixing the ultrasonic probe of the ultrasonic instrument, the second fixing device is used for fixing the headgear of the ultrasonic instrument, and the first fixing device can be close to and away from the second fixing device.

Compared with the prior art, the surgical system provided by the present invention has the following advantages:

(1) The surgical system provided by the present invention performs the planning of the surgical path according to the acquired real-time lesion information, which can ensure the real-time performance of the planned target surgical path, which is more conducive to eliminating the lesions more accurately in the subsequent surgical execution stage. Compared with the operation system in the prior art, the present invention can not only ensure the accuracy of the operation, avoid many risks caused by the operation process completely relying on the clinical experience of the doctor, but also can effectively reduce the workload of the doctor and improve the work efficiency of the doctor. , which enables doctors to devote more energy to disease analysis and optimization of treatment plans.

(2) The lesion identification module in the present invention obtains a real-time registered image by acquiring a pre-operative medical image and an intra-operative real-time medical image, and registering the pre-operative medical image and the intra-operative real-time medical image, thereby obtaining a real-time registered image. High-resolution intraoperative real-time images can be obtained. The lesion identification module in the present invention also obtains real-time lesion information according to the real-time registration image, which can not only effectively improve the accuracy of lesion identification, reduce the workload of doctors, and lay a solid foundation for the subsequent operation path planning stage and operation execution stage It can also ensure that the acquired lesion information is synchronized with the operation time, which can solve the problem that the current lesion cannot be fully displayed in the prior art because the imaging time is earlier than the operation time, which is more conducive to the elimination of the lesion.

(3) The control device in the present invention further includes a functional safety module, which can monitor the real-time motion trajectory of the surgical equipment during the operation according to the real-time registration image. Due to the high-definition and accurate real-time registration images, fast and accurate monitoring of surgical equipment can be achieved. Compared with the prior art, the present invention has real-time monitoring of surgical equipment, which can avoid the deviation of the actual surgical path; at the same time, the functional safety module can obtain the safety operation boundary information according to the real-time lesion information, which ensures the safety boundary. Therefore, the safety performance of the surgical system during the operation is greatly improved.

(4) The surgical system provided by the present invention further includes a human-computer interaction module. By setting the human-computer interaction module, the entire surgical process can be displayed in real time. This not only enables the entire surgical procedure to be carried out under the supervision of the doctor, but also enables the doctor to observe the intraoperative real-time images to obtain more surgical information and further reduce the surgical risk.

Description of drawings

1 is a schematic diagram of an application scenario of a surgical system provided by an embodiment of the present invention;

FIG. 2 is a schematic block diagram of a surgical system according to an embodiment of the present invention;

3 is a schematic structural diagram of a stent provided by an embodiment of the present invention;

4 is a schematic flowchart of an image registration provided by the first embodiment of the present invention;

5 is a schematic flowchart of an image registration provided by a second embodiment of the present invention;

6 is a schematic flowchart of an image registration provided by a third embodiment of the present invention;

FIG. 7 is a schematic flowchart of identifying lesions using a deep neural network model according to an embodiment of the present invention;

FIG. 8 is a schematic flowchart of identifying lesions using a deep neural network model according to another embodiment of the present invention;

9 is a schematic diagram of a training process of a deep neural network model provided by an embodiment of the present invention;

10 is a schematic flowchart of planning a puncture path according to an embodiment of the present invention;

11 is a schematic flowchart of obtaining a target puncture point according to an embodiment of the present invention;

12 is a schematic block diagram of a surgical system according to another embodiment of the present invention;

13 is a schematic diagram of a workflow of a functional safety module provided by an embodiment of the present invention;

14 is a schematic diagram of a partial structure of a surgical device provided by an embodiment of the present invention;

15 is a schematic diagram of a partial structure of a driving unit provided by an embodiment of the present invention;

16 is a schematic structural diagram of a stent provided by another embodiment of the present invention;

17 is a schematic structural diagram of a refrigeration device according to an embodiment of the present invention;

FIG. 18 is a schematic diagram of a partial structure of a cryoablation needle according to an embodiment of the present invention;

FIG. 19 is a schematic flowchart of a cryoablation process according to an embodiment of the present invention.

Among them, the reference numerals are as follows:

first image acquisition device-100; control device-200; surgical equipment-300; lesion identification module-210; image acquisition unit-211; image registration unit-212; lesion identification unit-213; automatic planning module-220; Module-230; Second Image Acquisition Device-400; Drive Unit-310; Surgical Instrument-320; Fixture-311; Functional Safety Module-240; Human-Machine Interaction Module-600; Base-510; First Fixture Device-520; Second Fixture-530; Ultrasonic Probe-110; Head Cover-120; Scale-321; Thermal Coating-322; Display Screen-323; Cold unit-331; Heat exchange unit-332; Refrigerated gas source-333; Valve-334.

Detailed ways

The surgical system proposed by the present invention will be described in further detail below with reference to FIGS. 1 to 19 and the specific embodiments. The advantages and features of the present invention will become more apparent from the following description. It should be noted that, the drawings in the present invention are in a very simplified form and all use inaccurate scales, and are only used to facilitate and clearly assist the purpose of explaining the embodiments of the present invention. For the purpose, features and advantages of the present invention to be more clearly understood, please refer to the accompanying drawings. It should be noted that the structures, proportions, sizes, etc. shown in the drawings in this specification are only used to cooperate with the contents disclosed in the specification, so as to be understood and read by those who are familiar with the technology, and are not used to limit the implementation of the present invention. condition. Therefore, any modification of the structure, the change of the proportional relationship or the adjustment of the size should still fall within the technical content disclosed in the present invention under the same or similar situation as the effect that the present invention can produce and the purpose that can be achieved. within the range that can be covered. The specific design features of the invention disclosed herein, including, for example, the specific dimensions, orientations, locations, and profiles will be determined in part by the specific intended application and use environment. In the embodiment described below, the same reference numerals are used in common between different drawings to denote the same parts or parts having the same function, and repeated description thereof may be omitted. In this specification, like numerals and letters are used to refer to like items, so once an item is defined in one figure, it need not be discussed further in subsequent figures. Additionally, if a method described herein includes a series of steps, the order of the steps presented herein is not necessarily the only order in which the steps may be performed, and some of the steps described may be omitted and/or some not described herein Other steps can be added to the method.

Furthermore, it should be noted that herein, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these entities or operations. There is no such actual relationship or sequence between operations. Moreover, the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device that includes a list of elements includes not only those elements, but also includes not explicitly listed or other elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

The main purpose of the present invention is to provide a surgical system for automatic surgery, which has the advantages of accuracy, reliability, safety, high efficiency, etc., which can solve the problem in the prior art that doctors rely on preoperative images to judge lesions, and determine the lesions according to clinical experience. Planning the surgical path and determining the surgical parameters makes the surgical process completely depend on the doctor's clinical experience, and there are many risks.

To achieve the above object, the present invention provides a surgical system. Please refer to FIG. 1 and FIG. 2 , wherein FIG. 1 schematically shows a schematic diagram of an application scenario of a surgical system provided by an embodiment of the present invention; FIG. 2 schematically shows an application scenario of the surgical system provided by an embodiment of the present invention. Schematic diagram of the block structure. As shown in FIGS. 1 and 2 , the surgical system includes a control device 200 and a surgical device 300 , and the surgical device 300 is connected in communication with the control device 200 . Optionally, the surgical system further includes a first image acquisition device 100 . The first image acquisition device 100 is connected in communication with the control device 200 for acquiring intraoperative real-time medical images. Specifically, the first image acquisition device 100 may be an ultrasound machine, a CT device or an MR device. Intraoperative real-time ultrasound images can be acquired by the ultrasound apparatus. Intraoperative real-time CT images can be acquired by CT equipment. Intraoperative real-time MRI images can be acquired by MR equipment. Of course, as can be understood by those skilled in the art, the first image acquisition device may also be other imaging equipment capable of acquiring medical images except for ultrasound equipment, CT equipment, and MR equipment, which is not limited in the present invention.

Please refer to FIG. 3 , which schematically shows a structural diagram of a bracket provided by an implementation of the present invention. As shown in FIG. 3 , when the first image acquisition device 100 is an ultrasound instrument, the surgical system further includes a bracket 500 , and the bracket 500 includes a base 510 and a first fixing device mounted on the base 510 device 520 and second fixture 520. The first fixing device 520 is used for fixing the ultrasonic probe 110 of the ultrasonic instrument, the second fixing device 530 is used for fixing the headgear 120 of the ultrasonic instrument, and the first fixing device 520 can be close to and away from the second Fixture 530. Therefore, by arranging the bracket 500 , the placement of the ultrasonic probe 110 and the headgear 120 can be more convenient. The headgear 120 is used to wrap the ultrasonic probe 110 when the ultrasonic probe 110 is used, so as to effectively prevent the cross infection of germs and further improve the safety performance during the operation. When the ultrasound probe 110 needs to be used to collect images, the first fixing device 520 can be moved toward the position of the second fixing device 530 to insert the ultrasound probe 110 into the headgear 120, and then the cover The ultrasound probe 110 provided with the headgear 120 is inserted into the patient's body to collect intraoperative real-time medical images.

Specifically, a slide rail may be provided on the base 510, and the first fixing device 520 is connected with a driving device (eg, a motor). Under the driving of the driving device, the first fixing device 520 can slide along the sliding rail to approach and move away from the second fixing device 530 . Of course, as can be understood by those skilled in the art, in other embodiments, the first fixing device 520 can also be approached and separated from the second fixing device 530 by other means in the prior art.

Please continue to refer to FIG. 1 and FIG. 2 . As shown in FIG. 1 and FIG. 2 , the control device 200 includes a lesion identification module 210 , an automatic planning module 220 and a control module 230 that are communicatively connected. The lesion identification module 210 is used for acquiring real-time lesion information according to the intraoperative real-time medical image and the preoperative medical image. The automatic planning module 220 is configured to plan an operation path according to the real-time lesion information, so as to obtain a target operation path. The control module 230 is configured to control the surgical device 300 to perform surgery according to the surgical operation parameters and the acquired target surgical path. Among them, the surgical operation parameters can only be determined by the doctor based on clinical experience before the operation. Alternatively, the surgical operation parameters are only automatically planned by the automatic planning module, or alternatively, the surgical operation parameters are determined by the doctor based on clinical experience before the operation, and during the operation, the automatic planning module determines the operation parameters before the operation according to the real-time lesion information. The operating parameters are modified (planned) in real time to obtain surgical operating parameters. Therefore, the surgical system provided by the present invention can automatically acquire real-time lesion information through the lesion identification module 210 according to the intraoperative real-time medical image and the preoperative medical image; Automatically plan the surgical path, such as the planning of the puncture surgical path, the planning of the cutting surgical path, etc.; the control module 230 can automatically control the surgical device 300 to perform surgery according to the planned path (ie, the target path), so as to achieve The purpose of eliminating lesions. It can be seen that the surgical system provided by the present invention performs the planning of the surgical path according to the acquired real-time lesion information, which can ensure the real-time performance of the planned target surgical path, which is more conducive to more accurate removal of lesions in the subsequent surgical execution stage. Compared with the operation system in the prior art, the present invention can not only ensure the accuracy of the operation, avoid many risks caused by the operation process completely relying on the clinical experience of the doctor, but also can effectively reduce the workload of the doctor and improve the work efficiency of the doctor. , which enables doctors to devote more energy to disease analysis and optimization of treatment plans.

Preferably, the real-time lesion information may include real-time lesion location information, real-time lesion volume information, real-time lesion shape information, real-time key organ tissue information, and the like. It should be noted that, as understood by those skilled in the art, in some embodiments, the real-time lesion information may only include real-time lesion location information; in some embodiments, the real-time lesion information may only include real-time lesion information location information and real-time lesion volume information; in some embodiments, the real-time lesion information may only include real-time lesion location information and real-time lesion shape information; in some embodiments, the real-time lesion information may include real-time lesion location information, Real-time lesion volume information, real-time lesion shape information; in some embodiments, the real-time lesion information may only include real-time lesion location information and real-time key organ tissue information. In addition, it should be noted that the real-time key organ tissue information includes real-time information on the organ where the lesion is located and the surrounding organs at risk.

Further, as shown in FIG. 1 and FIG. 2 , the lesion identification module 210 includes an image acquisition unit 211 , an image registration unit 212 and a lesion identification unit 213 . The image acquisition unit 211 is used for acquiring preoperative medical images and intraoperative real-time medical images. The image registration unit 212 is configured to register the preoperative medical image and the intraoperative real-time medical image to obtain a real-time registered image. The lesion identification unit 213 is configured to acquire real-time lesion information according to the real-time registration image. Therefore, the lesion identification module 210 in the present invention obtains the real-time registered image by acquiring the pre-operative medical image and the intra-operative real-time medical image, and registering the pre-operative medical image and the intra-operative real-time medical image, Thus, high-definition intraoperative real-time images can be obtained. The lesion identification module 210 in the present invention also acquires real-time lesion information according to the real-time registration image, which can not only effectively improve the accuracy of lesion identification, reduce the workload of doctors, and lay the foundation for the subsequent operation path planning stage and operation execution stage A good foundation can also ensure that the acquired real-time lesion information is synchronized with the operation time, thereby solving the problem that the current lesion cannot be fully displayed in the prior art because the imaging time is earlier than the operation time, which is more conducive to the detection of the lesion. eliminate.

Please continue to refer to FIG. 4 , which schematically shows a flow chart of the image registration provided by the first embodiment of the present invention. As shown in FIG. 4 , when the preoperative medical image is a CT image or an MRI image, and the intraoperative real-time medical image is a CT image or an MRI image, the image registration unit 212 specifically performs the following process for the operation The pre-medical image and the intraoperative real-time medical image are registered to obtain the real-time registered image:

Image registration refers to seeking one or a series of spatial transformations for an image to make it spatially match the corresponding points on another image. For medical images, this matching means that the same anatomical point on the human body has the same spatial location on the two matched medical images. There are many registration methods for medical images, which can be classified according to different classification methods. The methods based on the internal features of the image include: the method based on the boundary, the method based on the similarity of the voxel; the methods based on the external features of the image include: the frame method, the external labeling method and so on. If divided according to the linearity and nonlinearity of the registration geometric transformation, the registration method can be divided into linear registration transformation and nonlinear registration transformation. Linear registration transformation includes rigid transformation, affine transformation and projection transformation; nonlinear registration transformation is what we usually call elastic registration transformation. Specifically, in this embodiment, an elastic registration method can be adopted, that is, an appropriate elastic transformation model is constructed, and based on the elastic transformation model, the preoperative 3D medical image is registered to the intraoperative real-time 3D medical image , to obtain an intraoperative real-time 3D medical image after registration, that is, a registered image. Elastic registration is more locally adaptable. Therefore, more accurate registration can be achieved by using the elastic registration method. It should be noted that, as can be understood by those skilled in the art, in other embodiments, other image registration methods other than the elastic registration method may also be used to register the preoperative three-dimensional medical image to the desired location. Intraoperative real-time three-dimensional medical images, such as rigid transformation registration methods, affine transformation registration methods, projection transformation registration methods, etc., are not limited in the present invention.

Since the preoperative CT image or MRI image acquired in this embodiment is a high-definition image, when the preoperative 3D medical image is registered to the intraoperative real-time 3D medical image, the post-registration accuracy can be effectively improved. The clarity of intraoperative real-time 3D medical images lays a good foundation for subsequent lesion identification and improves the accuracy of lesion identification. In addition, in this embodiment, three-dimensional modeling is performed on the preoperative medical image and the intraoperative real-time medical image, and then the preoperative three-dimensional medical image is registered to the intraoperative real-time three-dimensional medical image, so that a three-dimensional registered image can be obtained. , so that the real-time lesion information can be more comprehensively displayed through the three-dimensional registration image.

Further, the real-time registration image is an image obtained by fusing the registered preoperative 3D medical image with the intraoperative real-time 3D medical image. Image fusion is to use a specific algorithm to combine two or more images into a new image. Image fusion can make the fused image have a more comprehensive and clearer description of the scene because it can utilize the correlation in space and time and the complementarity in information of two (or more) images. Thus, the present invention obtains a real-time registered image by fusing the registered pre-operative three-dimensional medical image with the intra-operative real-time three-dimensional medical image. This enables the real-time registration image to fuse the information in the pre-operative 3D medical image and the intra-operative real-time 3D medical image, so that the lesions in the real-time registration image can be displayed more comprehensively, thereby further improving subsequent lesion identification. 's accuracy.

Please continue to refer to FIG. 5 , which schematically shows a flow chart of the image registration provided by the second embodiment of the present invention. As shown in FIG. 5 , in this embodiment, the preoperative medical image is a CT image or an MRI image, the intraoperative real-time medical image is an ultrasound image, and the image registration unit 212 specifically performs the following process for the The preoperative medical image and the intraoperative real-time medical image are registered to obtain the real-time registered image:

Thus, by registering the preoperative three-dimensional medical image to the intraoperative real-time medical image, a local two-dimensional real-time image, that is, the first real-time registered image, can be obtained; by registering the intraoperative real-time medical image with By aligning to the preoperative three-dimensional medical image, a global three-dimensional real-time image, that is, the second real-time registration image, can be obtained. Thus, more information about the lesion can be obtained through the first real-time registration image and the second real-time registration image.

Further, in order to improve the registration efficiency, registering the intraoperative real-time medical image to the preoperative three-dimensional medical image to obtain a second real-time registration image includes:

The intraoperative real-time three-dimensional medical image is registered to the pre-operative three-dimensional medical image to obtain a second real-time registered image.

Therefore, the intraoperative real-time 3D medical image is obtained by first performing 3D modeling on the intraoperative real-time medical image, and then the intraoperative real-time 3D medical image is registered to the preoperative 3D medical image to obtain the intraoperative real-time 3D medical image. The second real-time registration image can greatly improve the registration efficiency. Specifically, the ultrasonic probe may be continuously scanned to obtain ultrasonic images of different scanning layers (ie, intraoperative real-time medical images), and then three-dimensional modeling of the ultrasonic images of different scanning layers may be performed to obtain intraoperative real-time 3D medical images. image. It should be noted that, as can be understood by those skilled in the art, in other embodiments, the obtained ultrasound images of different scanning layers can also be registered layer by layer into the preoperative three-dimensional medical image , to obtain the second real-time registration image.

Please continue to refer to FIG. 2 , as shown in FIG. 2 , the surgical system further includes a second image acquisition device 400 communicatively connected to the control device, and the second image acquisition device 400 is used for real-time acquisition of intraoperative real-time patient skin image.

Please continue to refer to FIG. 6 , which schematically shows a flow chart of the image registration provided by the third embodiment of the present invention. As shown in FIG. 6 , the image registration unit 212 specifically performs the following process for the image registration. The pre-medical image and the intraoperative real-time medical image are registered to obtain the real-time registered image:

Therefore, in this embodiment, the intraoperative real-time skin image of the patient is collected by the second image acquisition device 400, and the intraoperative real-time skin image of the patient is 3D modeled by the image registration unit 212, so that the intraoperative real-time skin image can be obtained. a real-time human model image, and then register the first real-time fusion image obtained by registering and fusing the preoperative three-dimensional medical image and the intraoperative real-time three-dimensional medical image to the intraoperative real-time human model image, Thereby, a three-dimensional real-time registration image containing the outline of the human body can be obtained. In the process of identifying the lesions on the real-time registration images through the deep neural network model described below, the actual position of the lesions in the human body can be directly output, which is more conducive to the planning of subsequent surgical paths and the execution of related operations.

Specifically, in this embodiment, the preoperative medical image may be a CT image or an MRI image, and the intraoperative real-time medical image may be a CT image, an MRI image or an ultrasound image. The second image acquisition device 400 may specifically be an optical monitor. At this time, the second image acquisition device 400 may acquire an intraoperative real-time skin image of the patient based on an optical tracking method. Of course, as can be understood by those skilled in the art, the second image acquisition device 400 may also be a binocular camera. At this time, the second image acquisition device 400 may acquire intraoperative real-time patient skin images based on the principle of binocular vision measurement.

Further, the real-time registration image is an image obtained by fusing the registered intraoperative real-time human model image and the first real-time fusion image. Thus, the obtained real-time registration image is simultaneously fused with the information in the preoperative 3D medical image, the intraoperative real-time 3D medical image, and the intraoperative 3D human model image. As a result, the lesions in the real-time registration image can be displayed more comprehensively, and the accuracy of subsequent lesion identification is further improved, thereby laying a good foundation for the subsequent automatic operation path planning and automatic operation stage, and further improving the cost of The automatic surgery effect of the surgical system provided by the invention.

In an exemplary embodiment, the lesion identification unit 213 uses a pre-trained deep neural network model to identify lesions on the real-time registration image, so as to obtain real-time lesion information.

Wherein, the real-time registration image may be a real-time registration image obtained by adopting the registration method provided in the first embodiment or the third embodiment. Therefore, the present invention uses the pre-trained deep neural network model to identify the lesions in the real-time registration image, which not only can automatically identify the lesions, but also lays a good foundation for the subsequent automatic operation path planning and automatic operation stage. , further improving the automatic operation effect of the surgical system provided by the present invention; it can also effectively reduce the workload of the doctor, improve the work efficiency of the doctor, and enable the doctor to devote more energy to the analysis of the patient's condition and the optimization of the treatment plan. In addition, by using a pre-trained deep neural network model to identify the real-time registration image, the accuracy and efficiency of lesion identification can be effectively improved. Specifically, different colors and shades may be used to display the identified lesions, the organs where the lesions are located, and the surrounding organs at risk in the real-time registration image. Therefore, this arrangement can make it more convenient for the doctor to observe the location of the lesion.

Please continue to refer to FIG. 7 , which schematically shows a flowchart of a lesion identification using a deep neural network model according to an embodiment of the present invention. As shown in FIG. 7 , in this embodiment, a neural network model can be used to identify lesions through the following process:

Step 1. Input the real-time registration image;

Step 2, carrying out modular division according to the features of the real-time registration image to obtain a plurality of image modules;

Step 3: Identifying the feature of each of the image modules, and reconstructing the image based on the identified feature to obtain a corresponding reconstructed image;

Step 4: Obtain real-time lesion information and accuracy rate information through deep learning based on the reconstructed image;

Step 5, judging whether the accuracy rate is greater than a preset threshold;

If so, output the real-time lesion information;

If not, return to step 2, and re-identify the lesion until the accuracy rate is greater than the preset threshold.

Wherein, in this embodiment, the real-time registration image may be a real-time registration image obtained by using the registration method provided in the first embodiment or the third embodiment.

Please continue to refer to FIG. 8 , which schematically shows a flow chart of identifying lesions using a deep neural network model provided by another embodiment of the present invention. As shown in FIG. 8 , in this embodiment, the lesion identification unit 213 acquires real-time lesion information through the following process:

Therefore, by using a pre-trained deep neural network model to segment the acquired preoperative three-dimensional medical image, a segmented image including the lesion, the organ where the lesion is located, and the surrounding organs at risk can be acquired. Specifically, the lesions, the organs where the lesions are located, and the surrounding organs at risk can be displayed in different colors and shades, and then the segmented images are respectively fused with the first real-time registration image and the second real-time registration image to obtain The second real-time fusion image and the third real-time fusion image, and finally the second real-time fusion image and the third real-time fusion image are fused, so as to output the real-time coordinates of the lesion, the organ where the lesion is located, and the surrounding organs at risk. The fourth real-time fusion image.

Please continue to refer to FIG. 9 , which schematically shows a training flow diagram of a deep neural network model provided by an embodiment of the present invention. As shown in Figure 9, the deep neural network model can be obtained through the following process:

Get training samples;

Perform preprocessing on the training sample, such as region segmentation, lesion and key organ labeling;

Based on the idea of clustering, construct a 3D segmentation deep neural network loss function, and design a 3D deep neural network structure;

The preprocessed training samples are input into the three-dimensional deep neural network structure for training to obtain a deep neural network model.

Wherein, the training samples come from existing case images. About how to construct a 3D segmentation deep neural network loss function based on the idea of clustering, and design a 3D deep neural network structure, and how to train the 3D deep neural network structure through the preprocessed training samples to obtain a deep neural network structure. For the network model, reference may be made to the prior art, so the present invention will not describe it again.

Please continue to refer to 10, which schematically shows a flow chart of planning a puncture path provided by an embodiment of the present invention. As shown in FIG. 10 , the automatic planning module 220 specifically performs the planning of the puncture surgical path through the following process:

Thus, the automatic planning module 220 can obtain the needle insertion angle and needle insertion depth of the puncture instrument according to the target puncture path, and the control module 230 can automatically control the surgical device 300 to reach the lesion according to the target puncture path position, to automatically perform puncture surgery, so as to achieve the purpose of automatically eliminating the lesion.

Further, according to the real-time lesion information, obtain the location information of at least one lesion target point and multiple puncture points, including:

According to the position information of the lesion in the coordinate system of the surgical device, the position information of at least one target point of the lesion and the multiple puncture points in the coordinate system of the surgical device is acquired.

Specifically, the image coordinate system is the coordinate system of the real-time registration image or the fourth real-time fusion image including the real-time lesion information. For the acquisition of the spatial mapping relationship between coordinate systems, reference may be made to the existing method, which will not be repeated in the present invention. Therefore, the present invention establishes the spatial mapping relationship between the image coordinate system and the coordinate system of the surgical device 300 , and according to the spatial mapping relationship, converts the location information of the lesion in the image coordinate system to its position in the surgical device 300 coordinate system. Then, according to the location information of the lesion in the coordinate system of the surgical device 300, obtain the location information of at least one target point of the lesion and a plurality of puncture points in the coordinate system of the surgical device 300, so as to ensure the final The obtained target puncture path is directly associated with the surgical device 300, so that the control module 230 can automatically control the surgical device 300 to perform a puncture operation according to the target puncture path.

The lesion target refers to the end point of the planned puncture path, and the position of the acquired lesion target point is the end point of the planned puncture path. The puncture point refers to the starting point of the planned puncturing path, and the acquired position of the puncturing point is the starting point position of the planned puncturing path. In actual operation, the number and position of the lesion target points can be set according to the specific conditions of the position and number of the lesion. For example, when there are multiple lesions, there are also multiple target points of the lesions, that is, one lesion corresponds to at least one target point of the lesion. Multiple targets can also be set for the same lesion according to actual conditions such as the volume and type of the lesion. When acquiring the puncture point according to the real-time lesion information, a plurality of puncture points that evenly cover the lesion may be set according to information such as the position and volume of the lesion, or a doctor may set a plurality of puncture points based on experience. As an embodiment of the present invention, multiple puncture points may also be selected according to pre-obtained body surface data of the patient. Then, according to the preset conditions, each of the puncture points is evaluated to obtain the target puncture point, and finally the corresponding target puncture point and the target puncture point are connected to obtain the target puncture path.

Please continue to refer to FIG. 11 , which schematically shows a flow chart of acquiring a target puncture point provided by an embodiment of the present invention. As shown in FIG. 11 , the evaluation of each of the puncture points to obtain the target puncture point according to the preset conditions specifically includes the following process:

Step A, each described puncture point is scored according to the preset scoring criterion, and the puncture point with the highest score is used as the (first) target puncture point;

Step B, judging whether the (first) target puncture point can cover all lesions;

If not, then perform step C;

Step C, score each non-target puncture point according to the preset scoring criterion, and use the non-target puncture point with the highest score as the (second) target puncture point;

Step D, judging whether all the target puncture points (ie the first and second target puncture points) can jointly cover all the lesions;

If not, steps C and D are repeated until all the target puncture points (ie, the first and second target puncture points) can cover the lesion together.

Specifically, if the judgment result in step B is that the target puncture point can cover all the lesions, the selection of the target puncture point is ended, and the corresponding target puncture point and the target puncture point are directly connected to obtain the target puncture path . It should be noted that the judging whether the target puncture point can cover all the lesions refers to whether the surgical instrument used for puncturing can reach all the lesions through the target puncture point without touching the surrounding organs at risk. Similarly, judging whether all the target puncture points can cover all the lesions together refers to whether the surgical instrument used for puncturing can reach all the target puncture points without touching the surrounding organs at risk. All lesions.

Further, scoring each of the puncture points according to the preset scoring criteria, and taking the puncture point with the highest score as the target puncture point, including:

Each of the puncture points is scored according to a plurality of preset scoring criteria, so as to obtain various scores of each of the puncture points;

Calculate the comprehensive score of each of the puncture points according to the weights corresponding to the preset scoring criteria;

Take the puncture point with the highest comprehensive score as the target puncture point;

Each non-target puncture point is scored respectively according to a plurality of pre-set scoring criteria, to obtain each score of each described non-target puncture point;

Calculate the comprehensive score of each of the non-target puncture points according to the weights corresponding to the preset scoring criteria;

The non-target puncture point with the highest score is used as the target puncture point.

Specifically, the scoring criteria include: the puncture distance (ie, the distance between the puncture point and the lesion target), whether the puncture path touches the surrounding organs at risk, the number of lesion targets that can be reached by the puncture point, and the like. During the specific operation, the multiple puncture points and the lesion target points are connected one by one to obtain multiple puncture paths. For example, when the number of puncture points is N and the number of lesion targets is M, N×M puncture paths can be obtained, that is, each puncture point corresponds to M puncture paths. Then, according to the scoring criterion of the puncturing distance of the puncturing path, the first scoring is performed on the puncturing point corresponding to each puncturing path. The more the number of puncture paths with the shortest puncture distance among the M puncture paths corresponding to the puncture point, the higher the score of the first item of the puncture point. Then, according to the scoring criterion of whether the puncture path is in contact with the surrounding organ at risk, a second score is performed on the puncture point corresponding to each puncture path. Among the M puncture paths corresponding to the puncture point, the more the number of puncture paths that do not touch the surrounding organ at risk, the higher the second item score of the puncture point. Then, according to the scoring criteria of the number of target points that can be reached by the puncture point, the third item is scored for each puncture point. The more the number of target points that can be reached by the puncture point, the higher the score of the third item corresponding to the puncture point. Finally, the comprehensive score of each of the puncture points is calculated according to the weights corresponding to each scoring criterion, and the puncture point with the highest comprehensive score is used as the (first) target puncture point. When the number of lesions is multiple, the target puncture point obtained for the first time may not cover all the lesions. Therefore, it is also necessary to reselect the target puncture point for the lesions that are not covered by the (first) target puncture point obtained for the first time (ie, the non-covered lesions). During the specific operation, firstly connect the remaining puncture points except the target puncture point obtained for the first time, that is, the non-target puncture point and the lesion target point corresponding to the uncovered lesion one by one, to obtain multiple puncture points path. Then, according to various scoring criteria, each of the non-target puncture points is scored, and a comprehensive score of each of the non-target puncture points is obtained. The non-target puncture point with the highest score was taken as the (second) target puncture point. Then, it is determined whether the (first) target puncture point obtained for the first time and the (second) target puncture point obtained this time can jointly cover all the lesions. If not, then connect the remaining non-target puncture points except all the target puncture points (the first and second target puncture points) with the lesion target points corresponding to the remaining lesions that cannot be covered one by one, for multiple piercing paths. Then, according to various scoring criteria, each of the non-target puncture points is scored, and a comprehensive score of each of the non-target puncture points is obtained. The non-target puncture point with the highest score was taken as the (third) target puncture point. Then, it is judged whether all the target puncture points (ie, all the target puncture points obtained in the first, second and this time) can jointly cover all the lesions. If not, repeat the above steps to select a new target puncture point until all the lesions can be covered.

In order to further improve the surgical effect and surgical efficiency of the surgical system provided by the present invention, in an exemplary embodiment, the automatic planning module 220 is further configured to plan surgical operation parameters according to the real-time lesion information. For example, for cryoablation surgery, information such as the volume and shape of the lesion can be obtained according to the real-time lesion information, and the cryoablation volume, freezing time, and number of freezing cycles can be automatically set according to the obtained information such as the volume and shape of the lesion. , freezing dose and other parameters. Therefore, by automatically planning surgical operation parameters according to the obtained real-time lesion information, the workload of doctors can be further reduced and the operation efficiency can be improved. In addition, compared with the prior art in which the surgical operation parameters are determined according to the doctor's experience, the present invention can further reduce the surgical risk. It should be noted that, as can be understood by those skilled in the art, in other embodiments, the surgical operation parameters may also be parameters manually determined by a doctor according to the acquired real-time lesion information.

Please continue to refer to FIG. 12 , which schematically shows a block diagram of a surgical system provided by another embodiment of the present invention. As shown in FIG. 12 , in this embodiment, the surgical system further includes a human-computer interaction module 600 that is communicatively connected to the control device 200 . The human-computer interaction module 600 is used to display and interact with data. Wherein, the human-computer interaction module 600 may include a display device and interaction software. Thus, the doctor can view the planned surgical path through the display device, and can adjust the surgical path in real time according to the actual situation. In addition, the display device can stereoscopically display the surgical process in real time, and the interactive software can receive the doctor's control information in real time, such as suspending or controlling the surgical process. In addition, the human-computer interaction module 600 can also realize the recording and display of surgery-related information. It can be seen that, by setting the human-computer interaction module 600, the entire operation process can be completely monitored by the doctor, and the doctor has complete control, that is, the operation process can be confirmed, interrupted, and modified at any time. In addition, by setting the human-computer interaction module 600, the doctor can also observe the intraoperative real-time image, so that the doctor can obtain more surgical information and further reduce the surgical risk.

Further, as shown in FIG. 2 and FIG. 12 , the control device 200 further includes a data storage module 250 . The data storage module 250 is used for data storage and management. Thus, by setting the data storage module 250, image data, patient data, surgery-related data can be stored and data management functions can be provided.

Furthermore, as shown in FIG. 2 and FIG. 12 , the control device 200 further includes a functional safety module 240 communicatively connected to the lesion identification module 210 . The functional safety module 240 is configured to monitor the real-time motion trajectory of the surgical device 300 according to the real-time registration image output by the image registration unit 212 . When the real-time motion trajectory of the surgical device 300 deviates too much from the planned path, the functional safety module 240 outputs alarm information. Specifically, a deviation threshold may be set. When it is detected that the deviation between the real-time motion trajectory and the planned path is greater than the threshold, the functional safety module 240 outputs alarm information. Therefore, during the operation of the surgical device 300 , the functional safety module 240 can monitor the real-time motion trajectory of the surgical device 300 to avoid deviation from the actual surgical path, thereby ensuring that the surgical device 300 can be In the event of an accident, the machine can be stopped in time to prevent injury to the patient and ensure the safety performance of the surgical device 300 during the operation.

Further, the functional safety module 240 also obtains the real-time lesion information through the lesion identification unit 213 to generate safe operation boundary information. For example, according to key organ tissue information, etc., set the area range that does not damage other tissues (ie, the safe operation boundary area), and determine whether the real-time motion trajectory of the surgical device 300 exceeds the safe operation boundary according to the obtained safe operation boundary information. area. When the real-time motion trajectory of the surgical device 300 touches or exceeds the safe operation boundary area, the functional safety module 240 outputs alarm information.

Therefore, the control module 230 can control the surgical device 300 to perform surgery according to the surgical operation parameters, the acquired target surgical path and the safe operation boundary information, and the functional safety module 240 can control the operation of the surgical device 300. The real-time motion trajectory of the surgical device 300 is monitored, so that when the functional safety module 240 detects that the surgical device 300 encounters the operation boundary, the system will automatically stop the operation and make an alarm to further improve the safety during the operation. performance. For example, for puncture surgery, when the puncture needle touches the operating boundary, the system will automatically stop the operation and automatically withdraw the needle to provide safety.

Please continue to refer to FIG. 13 , which schematically shows a work flow diagram of a functional safety module provided by an embodiment of the present invention. As shown in FIG. 13 , intraoperative real-time medical images can be acquired by the first image acquisition device 100; intraoperative real-time medical images acquired by the first image acquisition device 100 in real time can be acquired by the image acquisition unit 211; The image registration unit 212 can perform real-time registration of preoperative medical images and intraoperative real-time medical images to obtain real-time registered images; the human-computer interaction module 600 can display the real-time registered images , so that the movement trajectory (such as the puncture trajectory) of the surgical device 300 can be displayed in real time; the functional safety module 240 can judge in real time whether the movement trajectory (such as the puncture trajectory) of the surgical device 300 deviates from the target surgical route, And whether the movement trajectory (eg, the puncture trajectory) of the surgical device 300 exceeds the safe operation boundary. If the judgment result is that the movement trajectory (eg, the puncture trajectory) of the surgical device 300 deviates too much from the target surgical path, or the movement trajectory (eg, the puncture trajectory) of the surgical device 300 exceeds the safe operation boundary, the system will automatically stop , and output alarm information. The alarm information can be output through modes such as sound, light, and the display of alarm information on a human-computer interaction interface.

Please continue to refer to FIG. 14 , which schematically shows a partial structural diagram of a surgical device 300 provided by an embodiment of the present invention. As shown in FIG. 14 , the surgical device 300 includes a driving unit 310 and a surgical instrument 320 , the surgical instrument 320 is mounted on the driving unit 310 , and the control module 230 is configured to operate according to the surgical operation parameters and the acquired data. The target surgical path controls the driving unit 310 to drive the surgical instrument 320 to perform surgery.

Specifically, please refer to FIG. 15 , which schematically shows a partial structure diagram of a driving unit provided by an embodiment of the present invention. As shown in FIG. 14 and FIG. 15 , the driving unit 310 may be a robotic arm, and a fixator 311 for fixing the surgical instrument 320 is installed at the end of the robotic arm. For example, when the surgical instrument 320 is a puncture instrument, the holder 311 is a puncture device, and the puncture device can hold the puncture instrument. Therefore, the surgical instrument 320 can be driven by the robotic arm to perform surgery, so as to further reduce the surgical risk and improve the safety performance during the surgical procedure. It should be noted that, as can be understood by those skilled in the art, in other embodiments, the driving unit 310 may also be other automated devices other than a robotic arm, which is not limited in the present invention.

Please continue to refer to FIG. 16 , which schematically shows a schematic structural diagram of a stent provided by another embodiment of the present invention. As shown in FIG. 16 , in this embodiment, the bracket 500 is provided with a first fixing device 520 for fixing the ultrasonic probe 110 of the ultrasonic instrument and a second fixing device 520 for fixing the headgear 120 of the ultrasonic instrument. In addition to the fixing device 530, there is also a fixing device 311 for fixing the surgical instrument 320, such as a puncture device for fixing the puncturing instrument. In specific use, the ultrasonic probe 110 and the surgical instrument 320 can be fixed on the bracket 500 together, and then the bracket 500 can be installed on the drive unit 310, such as a robotic arm. Therefore, the ultrasonic probe 110 and the surgical instrument 320 can be controlled by the driving unit 310 at the same time, so that the operation can be more convenient.

Further, the surgical instrument 320 may be an instrument for performing a puncture operation, such as a biopsy needle for performing a biopsy puncture operation or a cryoablation needle for performing an ablation operation, and the like. Of course, as can be understood by those skilled in the art, the surgical instrument 320 may also be other instruments except for performing puncture surgery, which is not limited in the present invention.

When the surgical instrument 320 is an instrument for performing a puncture operation, that is, when the surgical device 300 is used for performing a puncture operation, the automatic planning module 220 is configured to plan the puncture path according to the real-time lesion information, to obtain a target puncture path; the control module 230 is configured to control the surgical device 300 to perform a puncture operation according to surgical operation parameters and the acquired target puncture path. Regarding how the automatic planning module 220 performs the planning of the puncture path according to the real-time lesion information, reference may be made to the relevant content in the surgical path planning above, so it will not be repeated here.

When the surgical instrument 320 is a cryoablation needle, as shown in FIGS. 2 and 12 , the surgical instrument 300 further includes a cooling device 330 . The control module 230 is configured to control the refrigeration device 330 to provide a cold source to the cryoablation needle according to the surgical operation parameters, so that the cryoablation needle can follow the required surgical operation parameters, such as freezing time, freezing time, and freezing time. The number of cycles and the amount of cryoablation are used to perform cryoablation. Thus, the control module 230 controls the refrigeration device 330 to provide a cold source to the cryoablation needle according to the surgical operation parameters, so that the purpose of autonomously eliminating the lesion can be achieved.

Please refer to FIG. 17 , which schematically shows a schematic structural diagram of a refrigeration device provided by an embodiment of the present invention. As shown in FIG. 17 , the refrigeration device 330 includes a pre-cooling device 331 , a heat exchange device 332 , a refrigerated gas source 333 and the like. The pre-cooling device 331 and the heat exchange device 332 are both connected to the refrigerated gas source 333 . Therefore, the temperature and flow rate of the gas flowing into the cryoablation needle can be better controlled by the pre-cooling device 331 and the heat exchange device 332 . Wherein, a valve 334 is provided on the pipeline connecting the heat exchange device 332 and the refrigerated gas source 333 . Thus, through the valve 334, the flow of gas through the heat exchange device 332 can be controlled. Regarding the working principles of the precooling device 331 and the heat exchanging device 332, reference may be made to the prior art, which will not be repeated in the present invention.

Please continue to refer to FIG. 18 , which schematically shows a partial structure diagram of a cryoablation needle provided by an embodiment of the present invention. As shown in FIG. 18 , the cryoablation needle is provided with a scale 321 , a thermal insulation coating 322 , a temperature sensor (not shown in the figure) and a display screen 323 . Specifically, the scale 321 is provided on the outer surface of the cryoablation needle. Therefore, by setting the scale 321, the needle insertion depth of the cryoablation needle can be precisely adjusted. The thermal barrier coating 322 is provided on the inner surface of the cryoablation needle except for the tip portion (the end near the patient). Therefore, by providing the thermal insulation coating 322, the cryoablation needle can effectively prevent normal tissues other than the lesion from being frostbitten during the treatment process, so as to improve the safety during the operation. The temperature sensor is arranged at the tip portion of the cryoablation needle. Thereby, the current temperature of the tip of the cryoablation needle can be monitored in real time by the temperature sensor. The display screen 323 is provided at the needle holding end of the cryoablation needle (the end close to the operator (ie, the doctor)). Therefore, the results measured by the temperature sensor can be displayed in real time through the display screen 323, so that the doctor can monitor the current temperature of the tip of the cryoablation needle in real time, so that the cryoablation effect can be further improved.

Please continue to refer to FIG. 19 , which schematically shows a flow chart of a cryoablation process provided by an embodiment of the present invention. As shown in Figure 19, the cryoablation process includes the following processes:

The cryoablation needle is inserted into the lesion according to the target puncture path;

When the freezing mode is activated, the ice ball generated at the tip of the needle gradually increases;

With continuous cryoablation, the ice ball enlarges and covers the lesion;

Turn on the rewarming mode, and the ice hockey will melt;

Determine whether the number of freezing cycles has been reached;

If so, withdraw the cryoablation needle to complete the operation;

If not, restart freeze mode.

To sum up, compared with the prior art, the surgical system provided by the present invention has the following advantages:

(1) The surgical system provided by the present invention can automatically acquire real-time lesion information according to intraoperative real-time medical images and preoperative medical images through the lesion identification module; Carry out the planning of the surgical path; the control module can automatically control the surgical equipment to perform surgery according to the planned real-time surgical path (ie, the target path), so as to achieve the purpose of eliminating lesions. It can be seen that the surgical system provided by the present invention performs the planning of the surgical path according to the acquired real-time lesion information, thereby ensuring the real-time performance of the planned target surgical path, which is more conducive to accurately eliminating the lesions in the subsequent surgical execution stage. Compared with the operation system in the prior art, the present invention can not only ensure the accuracy of the operation, avoid many risks brought about by the operation process completely relying on the clinical experience of the doctor, but also can effectively reduce the workload of the doctor, improve the work efficiency of the doctor, and make the operation more efficient. Doctors can devote more energy to disease analysis and optimization of treatment plans.

(2) The lesion identification module in the present invention obtains a real-time registered image by acquiring a pre-operative medical image and an intra-operative real-time medical image, and registering the pre-operative medical image and the intra-operative real-time medical image, thereby obtaining a real-time registered image. High-resolution intraoperative real-time images can be obtained. The lesion identification module in the present invention also obtains real-time lesion information according to the real-time registration image, which can not only effectively improve the accuracy of lesion identification, reduce the workload of doctors, but also lay a good foundation for the subsequent surgical path planning stage and surgical execution stage. Basically, it can also ensure that the acquired lesion information is synchronized with the operation time, thereby solving the problem that the current lesion cannot be fully displayed in the prior art because the imaging time is earlier than the operation time, which is more conducive to the elimination of the lesion.

(4) The surgical system provided by the present invention further includes a human-computer interaction module. By setting the human-computer interaction module, the entire surgical process can be displayed in real time. This not only enables the entire surgical procedure to be carried out under the supervision of the doctor, but also enables the doctor to observe the intraoperative real-time images to obtain more surgical information, further reducing the surgical risk.

It should be noted that the devices and methods disclosed in the embodiments herein can also be implemented in other manners. The device embodiments described above are merely illustrative. For example, the flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments herein. In this regard, each block in the flowchart or block diagrams may represent a module, program segment, or portion of code, which comprises one or more configurable functions for implementing the specified logical function(s) Execute the instruction. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or actions. implementation, or may be implemented in a combination of special purpose hardware and computer instructions.

In addition, the functional modules in the various embodiments herein can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

Further, in the description of this specification, reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples" and the like means descriptions that are described in conjunction with the embodiment or example. Particular features, structures, materials, or characteristics are included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different implementations or examples described in this specification and the features of the different implementations or examples without conflicting each other.

To sum up, the above embodiments describe in detail the lesion identification method, the surgical path planning method, the storage medium and the surgical system proposed by the present invention. Of course, the above description is only a description of the preferred embodiments of the present invention, and does not limit the scope of the present invention. Any changes and modifications made by persons of ordinary skill in the field of the present invention according to the above disclosure shall fall within the protection scope of the present invention. It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit and scope of the invention. Provided that these modifications and variations fall within the scope of the present invention and its technical equivalents, the present invention is also intended to include these modifications and variations.

Claims

A surgical system, characterized in that it includes a control device and a surgical device, wherein the surgical device is connected in communication with the control device;

The control device includes a lesion identification module, an automatic planning module and a control module connected in communication;

Wherein, the lesion identification module is used to acquire real-time lesion information according to the intraoperative real-time medical image and the preoperative medical image,

The automatic planning module is used for planning a surgical path according to the real-time lesion information, so as to obtain a target surgical path, and

The control module is used for controlling the surgical device to perform surgery according to the surgical operation parameters and the acquired target surgical path.
The surgical system according to claim 1, further comprising a first image acquisition device, wherein the first image acquisition device is connected in communication with the control device, and is used for acquiring intraoperative real-time medical images.
The surgical system according to claim 1, wherein the lesion identification module comprises an image acquisition unit, an image registration unit and a lesion identification unit that are communicatively connected;

Wherein, the image acquisition unit is used to acquire preoperative medical images and intraoperative real-time medical images,

the image registration unit for registering the preoperative medical image and the intraoperative real-time medical image to obtain a real-time registered image, and

The lesion identification unit is configured to acquire real-time lesion information according to the real-time registration image.
The surgical system according to claim 3, wherein if the preoperative medical image is a CT image or an MRI image, and the intraoperative real-time medical image is a CT image or an MRI image, the image registration unit obtains the Register images in real time, including:

performing three-dimensional modeling on the preoperative medical image to obtain a preoperative three-dimensional medical image;

performing three-dimensional modeling on the intraoperative real-time medical image to obtain the intraoperative real-time three-dimensional medical image;

The preoperative three-dimensional medical image is registered to the intraoperative real-time three-dimensional medical image to obtain a real-time registered image.
The surgical system according to claim 3, wherein if the preoperative medical image is a CT image or an MRI image, and the intraoperative real-time medical image is an ultrasound image, the image registration unit obtains the real-time registration images, including:

performing three-dimensional modeling on the preoperative medical image to obtain a preoperative three-dimensional medical image;

registering the preoperative three-dimensional medical image to the intraoperative real-time medical image to obtain a first real-time registered image;

The intraoperative real-time medical image is registered to the pre-operative three-dimensional medical image to obtain a second real-time registered image.
The surgical system according to claim 3, wherein the surgical system further comprises a second image acquisition device communicatively connected to the control device, the second image acquisition device is used to acquire intraoperative real-time patient skin images ;

The image acquisition unit is also used to acquire intraoperative real-time patient skin images;

The image registration unit acquires real-time registration images, including:

performing three-dimensional modeling on the preoperative medical image to obtain a preoperative three-dimensional medical image;

performing three-dimensional modeling on the intraoperative real-time medical image to obtain the intraoperative real-time three-dimensional medical image;

performing three-dimensional modeling on the intraoperative real-time patient skin image to obtain intraoperative real-time human body model images;

performing registration and fusion on the preoperative three-dimensional medical image and the intraoperative real-time three-dimensional medical image to obtain a first real-time fusion image;

The first real-time fusion image is registered to the intraoperative real-time human model image to obtain a real-time registered image.
The surgical system according to claim 4 or 6, wherein the lesion identification unit acquires real-time lesion information, comprising:

A pre-trained deep neural network model is used to identify lesions on the real-time registration images to obtain real-time lesion information.
The surgical system according to claim 5, wherein the lesion identification unit obtains real-time lesion information, comprising:

Segment the preoperative three-dimensional medical image by using a pre-trained deep neural network model to obtain segmented images;

fusing the segmented image with the first real-time registration image to obtain a second real-time fused image;

fusing the segmented image with the second real-time registration image to obtain a third real-time fused image;

Fusing the second real-time fusion image with the third real-time fusion image to obtain a fourth real-time fusion image;

According to the fourth real-time fusion image, real-time lesion information is acquired.
The surgical system according to claim 1, wherein the real-time lesion information includes real-time lesion location information.
The surgical system according to claim 9, wherein the real-time lesion information further comprises one or more of real-time lesion volume information, real-time lesion shape information and real-time key organ tissue information.
The surgical system according to claim 1, wherein the automatic planning module obtains the target surgical path, comprising:

According to the real-time lesion information, obtain the location information of at least one lesion target point and multiple puncture points;

According to preset conditions, each of the puncture points is evaluated to obtain the target puncture point;

Connect the corresponding lesion target point and the target puncture point to obtain the target puncture path.
The surgical system according to claim 11, wherein, according to the real-time lesion information, acquiring the location information of at least one target lesion and multiple puncture points comprises:

Obtain the spatial mapping relationship between the image coordinate system and the surgical equipment coordinate system;

According to the spatial mapping relationship and the real-time lesion information, obtain the real-time position information of the lesion in the coordinate system of the surgical equipment;

According to the real-time position information of the lesion in the coordinate system of the surgical device, the position information of at least one target point of the lesion and the multiple puncture points in the coordinate system of the surgical device is acquired.
The surgical system according to claim 11, wherein the evaluating each of the puncture points according to preset conditions to obtain a target puncture point comprises:

Step A, score each described puncture point according to a preset scoring criterion, and use the puncture point with the highest score as the target puncture point;

Step B, judging whether the target puncture point can cover all lesions;

If not, then perform step C;

In step C, each non-target puncture point is scored according to a preset scoring criterion, and the non-target puncture point with the highest score is used as the target puncture point;

Step D, judging whether all the target puncture points can cover all the lesions together;

If not, repeat steps C and D until all the target puncture points can cover the lesion together.
The surgical system according to claim 13, wherein the scoring of each of the puncture points according to a preset scoring criterion, and taking the puncture point with the highest score as the target puncture point, comprising:

i Score each of the puncture points respectively according to a plurality of preset scoring criteria, so as to obtain various scores of each of the puncture points;

ii Calculate the comprehensive score of each described puncture point according to the corresponding weights of the preset scoring criteria;

iii Take the puncture point with the highest comprehensive score as the target puncture point;

The described scoring is performed on each non-target puncture point according to the preset scoring criteria, and the non-target puncture point with the highest score is used as the target puncture point, including:

i Score each non-target puncture point respectively according to a plurality of pre-set scoring criteria, so as to obtain various scores of each of the non-target puncture points;

ii Calculate the comprehensive score of each of the non-target puncture points according to the weights corresponding to the preset scoring criteria;

iii Take the non-target puncture point with the highest comprehensive score as the target puncture point.
The surgical system according to claim 3, wherein the control device further comprises a functional safety module communicatively connected to the lesion identification module, and the functional safety module is used for real-time output according to the image registration unit. The images are registered to monitor the real-time motion trajectory of the surgical equipment.
The surgical system according to claim 15, wherein the functional safety module is further configured to obtain safe operation boundary information according to the real-time lesion information, and determine the operation safety of the surgical equipment according to the safe operation boundary information. Whether the real-time motion trajectory exceeds the safe operation boundary area.
The surgical system according to claim 1, wherein the automatic planning module is further configured to plan surgical operation parameters according to the real-time lesion information.
The surgical system according to claim 1, wherein the surgical equipment comprises a driving unit and a surgical instrument, the surgical instrument is mounted on the driving unit, and the control module is configured to operate according to the surgical operation parameters and the acquired The target surgical path controls the drive unit to drive the surgical instrument to perform surgery.
The surgical system according to claim 18, wherein the driving unit is a robotic arm, and a distal end of the robotic arm is provided with a fixator for fixing the surgical instrument.
The surgical system according to claim 19, wherein the surgical instrument is an instrument for performing a puncture operation;

The automatic planning module is configured to plan the puncture path according to the real-time lesion information, so as to obtain the target puncture path;

The control module is configured to control the surgical device to perform a puncture operation according to the surgical operation parameters and the acquired target puncture path.
The surgical system according to claim 20, wherein the surgical instrument is a cryoablation needle, the surgical instrument further comprises a refrigeration device, and the control module is configured to control the refrigeration device to move to the cryoablation device according to the surgical operation parameter. The cryoablation needle provides a source of cold.
The surgical system according to claim 21, wherein the surgical operation parameters include freezing time, freezing cycle number and freezing dose.
The surgical system according to claim 1, characterized in that, the surgical system further comprises a human-computer interaction module communicatively connected with the control device, and the human-computer interaction module is used for data display and interaction.
The surgical system according to claim 1, wherein the control device further comprises a data storage module, and the data storage module is used for data storage and management.
The surgical system according to claim 2, wherein the first image acquisition device is an ultrasonic instrument, the surgical system further comprises a bracket, the bracket comprises a base and a first image mounted on the base A fixing device and a second fixing device, the first fixing device is used to fix the ultrasonic probe of the ultrasonic instrument, the second fixing device is used to fix the headgear of the ultrasonic instrument, and the first fixing device can be close to and away from the second fixture.