WO2024077889A1

WO2024077889A1 - Operation prompting method and apparatus for flexible instrument, device, and storage medium

Info

Publication number: WO2024077889A1
Application number: PCT/CN2023/085904
Authority: WO
Inventors: 陈浩; 刘宏斌
Original assignee: 中国科学院自动化研究所
Priority date: 2022-10-13
Filing date: 2023-04-03
Publication date: 2024-04-18
Also published as: CN115906556A

Abstract

Embodiments of the present application provide an operation prompting method and apparatus for a flexible instrument, a device, and a storage medium. The method comprises: constructing a finite element model of a flexible instrument, wherein the finite element model of the flexible instrument comprises a plurality of finite element nodes; determining the displacement of the flexible instrument on the basis of the principle of minimum potential energy and a target constraint condition, wherein the target constraint condition is used for representing the constraint of a positional relationship between the finite element nodes in the finite element model of the flexible instrument and contact target patches in a three-dimensional model of the natural orifice of the human body, and the displacement of the flexible instrument comprises the displacement of each finite element node; determining a stress result of the flexible instrument according to the displacement of the flexible instrument; and outputting operation indication information of the flexible instrument according to the determined displacement of the flexible instrument and stress result of the flexible instrument, wherein the operation indication information of the flexible instrument is used for indicating a mode of performing surgery on the natural orifice of the human body by means of the flexible instrument. The method of the embodiments of the present application can effectively improve the efficiency and safety of surgery.

Description

Method, device, equipment and storage medium for operation prompting of flexible instrument

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Chinese patent application No. 202211255852.6, filed on October 13, 2022, and entitled “Operation prompt method, device, equipment and storage medium for flexible instruments”, which is incorporated herein by reference in its entirety.

Technical Field

The present application relates to the technical field of medical devices, and in particular to an operation prompt method, device, equipment and storage medium for a flexible device.

Background technique

During minimally invasive surgery, the spatial position information of medical devices in the patient's body plays a vital role in the accurate operation of medical devices. However, many medical devices will inevitably change their shape after entering the body, so how to accurately determine the force and deformation of flexible devices and provide accurate information for surgery is of great significance.

In the related art, when determining the force and deformation of a flexible instrument, it is necessary to rely on artificially given contact force between the flexible instrument and the natural cavity of the human body or rely on a collision detection method to determine the contact position of the flexible instrument and the natural cavity of the human body, and the contact point between the flexible instrument and the natural cavity of the human body does not change during the iteration process, resulting in large errors in the determined force and displacement of the flexible instrument, increasing the surgical risk for the patient.

Summary of the invention

In view of the problems in the prior art, the embodiments of the present application provide an operation prompt method, device, equipment and storage medium for a flexible device.

Specifically, the embodiments of the present application provide the following technical solutions:

In a first aspect, an embodiment of the present application provides an operation prompt method for a flexible device, comprising:

Establishing a finite element model of a flexible device; the finite element model of the flexible device includes a plurality of finite element nodes;

Based on the minimum potential energy principle and target constraint conditions, the displacement of the flexible instrument is determined; the target constraint conditions are used to represent the constraints on the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the contact point target patches in the three-dimensional model of the natural cavity of the human body; the displacement of the flexible instrument includes the displacement of each finite element node;

Determine the force result of the flexible device according to the displacement of the flexible device;

According to the determined displacement of the flexible instrument and the force result of the flexible instrument, operation instruction information of the flexible instrument is output; the operation instruction information of the flexible instrument is used to indicate the way of performing surgery on the natural cavity of the human body through the flexible instrument.

Furthermore, the determination of the displacement of the flexible device based on the minimum potential energy principle and the target constraint condition includes:

The displacement of the flexible device is determined when the potential energy change of the flexible device is minimal and the positional relationship between the flexible device and the natural cavity of the human body meets the target constraint condition; the target constraint condition is that the finite element nodes in the finite element model of the flexible device and the contact point target surfaces in the three-dimensional model of the natural cavity of the human body are not penetrable.

The displacement of the flexible device is determined using the following formula:

or
_Axi ＝ _bi

Said represents the minimum change in potential energy of the flexible device; _Δxi represents the displacement of the flexible device from time i-1 to time i; xi _-1 is the displacement of the finite element node at time i-1; K( _xi-1 ) represents the stiffness matrix of the flexible device under the displacement _xi-1 ; F represents the target external force on the flexible device, and the target external force does not include the contact force between the flexible device and the natural cavity of the human body; or Represents the target constraint; represents the normal vector of the contact point patch between the flexible instrument and the natural cavity of the human body; the _xi represents the finite element node displacement at time i; the _Si represents the generalized coordinate position of the flexible instrument in the undeformed state; the _X0 represents an arbitrary position on the contact point patch of the three-dimensional model of the natural cavity of the human body; the _Axi = _bi represents the first constraint condition; the A represents the projection matrix; the _bi represents the target position of the flexible instrument; and the _xi represents the finite element node displacement at time i.

Furthermore, determining the force result of the flexible device according to the displacement of the flexible device includes:

The force result of the flexible instrument is determined according to the displacement of the flexible instrument and the stiffness matrix of the flexible instrument.

Furthermore, the method further comprises:

The stiffness matrix of the flexible device is determined based on a rotation matrix between a local coordinate system of a unit and a global coordinate system in a finite element model of the flexible device.

Further, after determining the displacement of the flexible device, the method further includes:

In the case where the natural cavity of the human body is a narrow bifurcated cavity, based on the determined displacement of the flexible device, determining the positional relationship between the finite element nodes in the finite element model of the flexible device and each surface patch in the three-dimensional model of the natural cavity of the human body;

In the case where there is penetration between a finite element node in the finite element model of the flexible device and at least one of the plurality of facets in the three-dimensional model of the natural cavity of the human body, the target constraint condition is updated.

In a second aspect, the embodiment of the present application further provides an operation prompt device for a flexible device, comprising:

An establishment module is used to establish a finite element model of a flexible device; the finite element model of the flexible device includes a plurality of finite element nodes;

The first determination module is used to determine the displacement of the flexible instrument based on the minimum potential energy principle and the target constraint condition; the target constraint condition is used to represent the constraint of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the contact point target surface patch in the three-dimensional model of the natural cavity of the human body; the displacement of the flexible instrument includes the displacement of each finite element node;

A second determination module is used to determine the force result of the flexible device according to the displacement of the flexible device;

The prompt module is used to output operation instruction information of the flexible instrument according to the determined displacement of the flexible instrument and the force result of the flexible instrument; the operation instruction information of the flexible instrument is used to indicate the method of performing surgery on the natural cavity of the human body through the flexible instrument.

In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the operation prompt method for the flexible device as described in the first aspect is implemented.

In a fourth aspect, an embodiment of the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the operation prompt method of the flexible instrument as described in the first aspect is implemented.

In a fifth aspect, an embodiment of the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements the operation prompt method of the flexible instrument as described in the first aspect.

The operation prompt method, device, equipment and storage medium of the flexible instrument provided in the embodiments of the present application are based on the constraints of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the contact point target patches in the three-dimensional model of the natural cavity of the human body and the principle of minimum potential energy, and determine that the displacement corresponding to the minimum potential energy change of the flexible instrument is the displacement of the flexible instrument, so that the determined displacement of the flexible instrument meets the highly restricted environment of the human cavity, and the determined displacement of the flexible instrument is more accurate; then, based on the determined displacement of the flexible instrument and the operation instruction information of the flexible instrument, the surgical personnel can accurately operate the flexible instrument, thereby improving the efficiency and safety of the operation.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

FIG1 is a schematic flow chart of an operation prompt method for a flexible device provided in an embodiment of the present application;

2a-2b are schematic diagrams of the flexible device provided in the embodiments of the present application in contact with the natural cavity of the human body;

FIG3 is a schematic flow chart of another operation prompt method of a flexible device provided in an embodiment of the present application;

FIG4 is a schematic diagram of the structure of an operation prompt device for a flexible device provided in an embodiment of the present application;

FIG5 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

The method of the embodiment of the present application can be applied to surgical scenarios based on medical devices to implement operation prompts for flexible devices and improve surgical efficiency and safety.

In the related art, when determining the force and displacement of a flexible instrument, it is necessary to rely on artificially given contact force between the flexible instrument and the natural cavity of the human body or rely on a collision detection method to determine the contact position of the flexible instrument and the natural cavity of the human body, and the contact point between the flexible instrument and the natural cavity of the human body does not change during the iteration process, resulting in large errors in the determined force and displacement of the flexible instrument, increasing the surgical risk for the patient.

The operation prompt method of the flexible instrument in the embodiment of the present application is based on the constraints of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the contact point target patches in the three-dimensional model of the natural cavity of the human body and the principle of minimum potential energy, and determines that the displacement corresponding to the minimum potential energy change of the flexible instrument is the displacement of the flexible instrument, so that the determined displacement of the flexible instrument meets the highly restricted environment of the human cavity, and the determined displacement of the flexible instrument is more accurate; then, based on the determined displacement of the flexible instrument and the operation instruction information of the flexible instrument, the surgical personnel can accurately operate the flexible instrument, thereby improving the efficiency and safety of the operation.

The technical solution of the present application is described in detail with reference to specific embodiments in conjunction with Figures 1 to 5. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

FIG1 is a flow chart of an embodiment of an operation prompt method for a flexible device provided in an embodiment of the present application. As shown in FIG1 , the method provided in this embodiment includes:

Step 101, establishing a finite element model of a flexible device; the finite element model of the flexible device includes a plurality of finite element nodes;

Specifically, during minimally invasive surgery, the spatial information of medical devices in the patient's body plays a vital role in the accurate operation of medical devices. However, after entering the body, the flexible device will inevitably change its shape, so it is necessary to accurately determine the force and displacement of the flexible device to provide doctors with accurate information to improve the efficiency and safety of the operation.

In the embodiment of the present application, a finite element model of a flexible device is first established. Optionally, the finite element model of a flexible device is a model of a flexible device established using a finite element analysis method, and is a group of unit combinations that are connected only at nodes, transmit forces only at nodes, and are constrained only at nodes. The finite element model of a flexible device includes multiple finite element units and finite element nodes. By establishing a finite element model of a flexible device, the continuous geometric mechanism corresponding to the flexible device can be discretized into a finite number of flexible device finite element units, and a finite number of flexible device finite element nodes can be set in each flexible device finite element unit, so that the continuum corresponding to the flexible device is regarded as a group of flexible device finite element units that are connected only at the finite element nodes of the flexible device.

After establishing the finite element model of the flexible device, when determining the displacement and force of the flexible device, the flexible device can be regarded as being composed of many flexible device finite element units and flexible device finite element nodes that are interconnected. After determining the displacement and force of each subunit of the flexible device, the displacement and force of the flexible device can be determined. The deformation of the flexible device can be simulated by a large number of flexible device finite element units and finite element nodes, thereby reducing the difficulty of determining the displacement and force results of the flexible device.

Step 102: Determine the displacement of the flexible device based on the minimum potential energy principle and target constraint conditions; the target constraint conditions are used to represent the constraints on the positional relationship between the finite element nodes in the finite element model of the flexible device and the contact point target patches in the three-dimensional model of the natural cavity of the human body; the displacement of the flexible device includes the displacement of each finite element node;

Specifically, after the finite element model of the flexible device is established, the displacement of the flexible device can be determined based on the minimum potential energy principle and the target constraint condition; optionally, the potential energy of the flexible device at a certain time i-1 is expressed as follows:

Where U is the total potential energy, _xi-1 is the generalized coordinate column vector of the node displacement at time i-1, and F is the known total external force; the total external force F does not include the interaction force between the flexible device and the natural cavity of the human body; K( _xi-1 ) is the stiffness matrix of the flexible device, and the potential energy of the flexible device at the next time i is as follows:

Where _Δxi is the displacement of the flexible device from time i-1 to time i, _xi = _xi-1 + _Δxi , and C is a constant term independent of _Δxi . The unknown potential energy change ΔU related to _Δxi is defined as follows:

According to the minimum potential energy principle, using the incremental method, it is known that U _i-1 needs to find the minimum value of U _i , and only ΔU needs to be minimized, and the displacement corresponding to the minimum potential energy change of the flexible instrument is the displacement _Δxi of the flexible instrument; optionally, the displacement _Δxi of the flexible instrument includes the displacement of each finite element node, that is, the displacement _Δxi of the flexible instrument is composed and determined by the multi-dimensional finite element node displacement. In addition, when determining the displacement of the flexible instrument, it is necessary to consider that in the highly restricted environment of the human cavity, the displacement _Δxi of the flexible instrument is subject to environmental constraints. That is, the embodiment of the present application is based on the constraints of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the contact point target patch in the three-dimensional model of the natural cavity of the human body and the principle of minimum potential energy, and determines that the displacement corresponding to the minimum potential energy change of the flexible instrument is the displacement of the flexible instrument, so that the determined displacement of the flexible instrument meets the highly restricted environment of the human cavity, and also makes the determined displacement of the flexible instrument more accurate.

Step 103, determining the force result of the flexible device according to the displacement of the flexible device;

Specifically, after determining the displacement of the flexible device based on the principle of minimum potential energy and the constraints of the positional relationship between the finite element nodes in the finite element model of the flexible device and the contact point target patch in the three-dimensional model of the natural cavity of the human body, the force result of the flexible device can be determined based on the determined displacement of the flexible device. Optionally, the force result of the flexible device can be determined according to Hooke's theorem. Optionally, the determined force result of the flexible device includes the force result of each finite element node, that is, the determined force result of the flexible device is composed and determined based on the multi-dimensional finite element node force result.

Step 104: outputting operation instruction information of the flexible instrument according to the determined displacement of the flexible instrument and the force result of the flexible instrument; the operation instruction information of the flexible instrument is used to indicate the method of performing surgery on the natural cavity of the human body through the flexible instrument.

Specifically, after determining the displacement of the flexible instrument and the force results of the flexible instrument based on the minimum potential energy principle and the constraint conditions of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the contact point target patch in the three-dimensional model of the natural cavity of the human body, the operation instruction information of the flexible instrument can be output based on the determined displacement and force results of the flexible instrument, wherein the instruction information is used to indicate how to operate the flexible instrument to perform surgery on the natural cavity of the human body in the next step. The doctor can also accurately operate the flexible instrument based on the operation instruction information of the flexible instrument, thereby improving the efficiency and safety of the operation. Optionally, after determining the displacement and force results of the flexible instrument, the displacement and force results of the flexible instrument can also be displayed on the screen for the doctor's reference, so as to accurately determine the next surgical action and improve the efficiency and safety of the operation.

Existing technical solutions usually directly give the force results of the flexible instrument or rely on the collision detection algorithm to determine the contact position between the flexible instrument and the natural cavity of the human body, and in some technical solutions, the contact point between the flexible instrument and the natural cavity of the human body does not change during the iteration process, resulting in errors in the displacement and force results of the determined flexible instrument, insufficient accuracy and real-time performance, and errors in the indication information, affecting the efficiency and safety of the operation. In the embodiment of the present application, by combining the determination of the displacement and force results of the flexible instrument with the constraint conditions of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the contact point target patch in the three-dimensional model of the natural cavity of the human body, it does not rely on the contact force between the flexible instrument and the natural cavity of the human body given by humans, nor does it rely on the collision detection algorithm to determine the contact position. Therefore, the method of the embodiment of the present application will neither introduce errors of different collision detection methods, nor assume that the contact point position between the flexible instrument and the natural cavity of the human body remains unchanged during the calculation of two time frames, thereby making the displacement and force results of the flexible instrument determined in the embodiment of the present application more accurate.

On the other hand, in the related art, the speed of determining the displacement and force results of the flexible instrument will be significantly reduced when the number of contact points between the flexible instrument and the natural cavity of the human body increases, and the number of contact points greatly affects the real-time and stability of determining the displacement and force results of the flexible instrument. In the process of determining the displacement and force of the flexible instrument in the embodiment of the present application, it does not rely on the artificially given contact force between the flexible instrument and the natural cavity of the human body, nor does it rely on the collision detection algorithm to determine the contact position of the flexible instrument and the natural cavity of the human body, so that the method of the embodiment of the present application can calculate the displacement and force of the flexible instrument in real time under different contact conditions between the flexible instrument and the natural cavity of the human body, especially when there are many contact points between the flexible instrument and the natural cavity of the human body, thereby improving the stability and practicality of determining the displacement and force results of the flexible instrument, and avoiding the problem that the efficiency of determining the displacement of the flexible instrument in the related art will decrease with the increase of the collision points between the flexible instrument and the natural cavity of the human body.

On the third aspect, in the related art, when the relative movement speed of the flexible instrument and the natural cavity of the human body is relatively large during the contact process (relative to the patch scale of the natural cavity model of the human body), the accuracy of the displacement and force of the flexible instrument determined by simulation is seriously insufficient. The embodiment of the present application does not limit the relative movement speed of the contact between the flexible instrument and the natural cavity of the human body, and will not limit the relative movement speed between the two objects of the flexible instrument and the natural cavity of the human body. It can well simulate the physical movement process of an object with a relatively large relative movement speed, so that the displacement and force results of the flexible instrument determined are more accurate.

The method of the above embodiment is based on the constraints of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the contact point target patches in the three-dimensional model of the natural cavity of the human body and the principle of minimum potential energy, and determines that the displacement corresponding to the minimum potential energy change of the flexible instrument is the displacement of the flexible instrument, so that the determined displacement of the flexible instrument meets the highly restricted environment of the human cavity, and the determined displacement of the flexible instrument is more accurate; then, the surgical personnel can accurately operate the flexible instrument based on the determined displacement of the flexible instrument and the operation instruction information of the flexible instrument, thereby improving the efficiency and safety of the operation.

In one embodiment, based on the minimum potential energy principle and target constraint conditions, determining the displacement of the flexible device includes:

The displacement of the flexible device is determined when the potential energy change of the flexible device is minimal and the positional relationship between the flexible device and the natural cavity of the human body meets the target constraint condition; the target constraint condition is that the finite element nodes in the finite element model of the flexible device and the contact point target patches in the three-dimensional model of the natural cavity of the human body are not penetrable.

Specifically, since the height of the human body cavity is limited, when determining the displacement of the flexible instrument based on the principle of minimum potential energy, it is necessary to make the displacement of the flexible instrument meet the height of the natural cavity of the human body, that is, when the finite element nodes in the finite element model of the flexible instrument and the contact point target surface in the three-dimensional model of the natural cavity of the human body are impenetrable, the displacement of the flexible instrument is determined, which means that the displacement of the flexible instrument is determined within the natural cavity of the human body, and the displacement of the flexible instrument cannot be determined outside the natural cavity of the human body, which makes the displacement of the flexible instrument consistent with the actual surgical situation, so that the displacement of the flexible instrument is more accurate. Optionally, the contact point target surface is the surface closest to the three-dimensional model of the natural cavity of the human body that the finite element node of the flexible instrument contacts.

The method of the above embodiment, when determining the displacement of the flexible instrument based on the principle of minimum potential energy, uses the impenetrable constraint condition between the finite element nodes in the finite element model of the flexible instrument and the contact point target patches in the three-dimensional model of the natural cavity of the human body, so that the determined displacement of the flexible instrument meets the height of the natural cavity of the human body, that is, the determined displacement of the flexible instrument is the displacement within the natural cavity of the human body, and the displacement of the flexible instrument cannot be determined outside the natural cavity of the human body, which makes the determined displacement of the flexible instrument more accurate and more in line with the actual surgical situation, so that based on the determined displacement of the flexible instrument, the next surgical plan can be determined more accurately, thereby improving the efficiency and safety of the operation.

represents the minimum change in potential energy of the flexible device; _Δxi represents the displacement of the flexible device from time i-1 to time i; the xi _-1 is the displacement of the finite element node at time i-1; the K( _xi-1 ) represents the stiffness matrix of the flexible device under the displacement _xi-1 ; F represents the target external force on the flexible device, and the target external force does not include the contact force between the flexible device and the natural cavity of the human body; or represents the target constraint; represents the surface normal vector of the contact point between the flexible instrument and the natural cavity of the human body; _xi represents the finite element node displacement at time i; _Si represents the generalized coordinate position of the flexible instrument in the undeformed state; _X0 represents any position on the plane corresponding to the three-dimensional model of the natural cavity of the human body; _Axi = _bi represents the first constraint condition; A represents the projection matrix; _bi represents the known or measurable target position of the flexible instrument; _xi represents the finite element node displacement at time i.

Specifically, the embodiment of the present application is based on the constraints of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the contact point target patches in the three-dimensional model of the natural cavity of the human body and the principle of minimum potential energy, and determines that the displacement corresponding to the minimum potential energy change of the flexible instrument is the displacement of the flexible instrument, so that the determined displacement of the flexible instrument meets the highly restricted environment of the human cavity, and the determined displacement of the flexible instrument is more accurate, so that the operation prompt information of the flexible instrument can be accurately provided to the surgical personnel, thereby improving the efficiency and safety of the operation.

Optionally, compared to the static balance method, one of the advantages of determining the displacement of the flexible instrument based on the minimum potential energy principle in the embodiment of the present application is the simple expression of the environmental constraints between the flexible instrument and the natural cavity of the human body, which improves the accuracy and efficiency of the determined displacement of the flexible instrument, thereby accurately providing the operating prompt information of the flexible instrument to the surgical personnel, thereby improving the efficiency and safety of the operation. Optionally, the displacement of the flexible instrument is determined using the following formula:

or
_Axi ＝ _bi

represents the minimum change in potential energy of the flexible device; _Δxi represents the displacement of the flexible device from time i-1 to time i; the xi _-1 is the displacement of the finite element node at time i-1; the K( _xi-1 ) represents the stiffness matrix of the flexible device under the displacement _xi-1 ; F represents the target external force on the flexible device, and the target external force does not include the contact force between the flexible device and the natural cavity of the human body; or represents the target constraint; represents the surface normal vector of the contact point between the flexible device and the natural cavity of the human body; _xi represents the finite element node displacement at time i; _Si represents the generalized coordinate position of the flexible device in the undeformed state; _X0 represents any position on the plane corresponding to the three-dimensional model of the natural cavity of the human body; _Axi = _bi represents the first constraint condition; A represents the projection matrix; _bi represents the target position of the flexible device; _xi represents the finite element node displacement at time i. Among them, _Axi = _bi represents mapping the known actual position of the flexible device to the finite element model of the flexible device; It is used to determine the displacement of the flexible device, that is, to determine the displacement _Δxi of the flexible device by determining the displacement corresponding to the minimum potential energy change of the flexible device. The displacement _Δxi of the flexible device is composed and determined by multi-dimensional finite element node displacements.

Optionally, the three-dimensional model of the natural cavity of the human body reconstructed from the three-dimensional CT before surgery is composed of multiple facets in the virtual environment. The inequality constraint of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the contact point target facets in the three-dimensional model of the natural cavity of the human body simplifies the collision between the flexible instrument and the natural cavity of the human body into a collision between a point and a surface. The normal vector of the contact point facet of the flexible instrument and the natural cavity of the human body is For any point on the plane, position X ₀ , we have or The direction of the inequality of the positional relationship between the finite element nodes in the finite element model of the flexible device and the contact point target patch in the three-dimensional model of the natural cavity of the human body depends on the initial value, where S _i is the generalized coordinate position of the flexible device in the undeformed state. Optionally, or The finite element nodes in the finite element model of the flexible device and the contact point target patches in the three-dimensional model of the human body's natural cavity are not penetrated. The inequality of the positional relationship between the finite element nodes in the finite element model of the flexible device and the contact point target patches in the three-dimensional model of the human body's natural cavity is added as a constraint to the minimum potential energy principle for determining the displacement of the flexible device. Then, the equation representing the known position of the flexible device obtained by the flexible robot through sensors or other algorithms is introduced, approximately _Axi = _bi . That is, the displacement of the flexible device can be accurately determined by the following formula, so that the determined displacement of the flexible device satisfies the constraint condition of the height of the human body's natural cavity, and the determined displacement of the flexible device is more accurate:

or
_Axi ＝ _bi

For example, as shown in Figures 2a and 2b, Figure 2a shows the contact point and force diagram between the finite element model of the flexible instrument and the natural cavity model of the human body between two frames in the embodiment of the present application, and Figure 2b shows the contact point and force diagram between the finite element model of the flexible instrument and the natural cavity model of the human body between two frames in the prior art, where n ₁ to n ₆ are finite element nodes of the flexible instrument, X is the contact point on the facet of the natural cavity model of the human body, and F is the contact force. As can be seen from Figures 2a and 2b, the contact point between the flexible instrument and the natural cavity of the human body between two frames in the embodiment of the present application can be any position within the domain composed of multiple facets in the natural cavity model of the human body, while the prior art assumes that the contact point between the flexible instrument and the natural cavity of the human body between every two frames remains unchanged, and only the force direction and magnitude at the contact point change, thereby causing estimation errors. In the embodiment of the present application, the displacement of the flexible instrument is determined based on the finite element model of the flexible instrument and the principle of minimum potential energy. Compared with the prior art, the advantage is that the assumption that the contact point between the flexible instrument and the natural cavity of the human body is fixed is removed in the embodiment of the present application. The contact point between the flexible instrument and the natural cavity of the human body can change between two frames, while the prior art assumes that the contact point between the flexible instrument and the natural cavity of the human body remains unchanged in two adjacent frames. In practical applications, such as in the process of passing through the natural cavity, the flexible instrument is inserted into the natural cavity at a fast speed, and the contact point between the flexible instrument and the natural cavity of the human body moves a large distance within one frame, and the position of the contact point changes greatly, which will bring a large error in the prior art. Since the prior art assumes that the contact point between the flexible instrument and the natural cavity of the human body is fixed, when determining the displacement of the flexible instrument in the prior art, there is a large restriction on the relative movement speed between the flexible instrument and the natural cavity of the human body. When the relative movement speed is too fast, mold penetration and instability will occur. The embodiment of the present application establishes an inequality constraint on the positional relationship between the finite element nodes in the finite element model of the flexible device and the contact point target patches in the three-dimensional model of the human body's natural cavity. This only constrains the finite element nodes in the finite element model of the flexible device and the contact point target patches in the three-dimensional model of the human body's natural cavity to be impenetrable, while the contact position and force position between the flexible device and the human body's natural cavity can be at any position within the domain composed of multiple patches of the human body's natural cavity model. This makes the displacement and force results of the flexible device determined in Figure 2a more accurate than the displacement and force results of the flexible device determined in the prior art in Figure 2b.

In the embodiment of the present application, the displacement of the flexible device is determined based on the constraints of the positional relationship between the finite element nodes in the finite element model of the flexible device and the contact point target patch in the three-dimensional model of the natural cavity of the human body and the principle of minimum potential energy. It does not rely on the artificially given contact force of the flexible device, nor does it rely on the collision detection method to determine the contact position between the flexible device and the natural cavity of the human body. Therefore, the method of the embodiment of the present application will neither introduce errors from different collision detection methods, nor assume that the contact point position between the flexible device and the natural cavity of the human body remains unchanged between two frames, and will not limit the relative movement speed of the flexible device and the natural cavity of the human body when in contact (relative to the patch scale of the natural cavity model of the human body), which greatly improves the accuracy of the determined displacement and force results of the flexible device. That is, the embodiment of the present application ensures the non-inter-invasion between the finite element model of the flexible device and the natural cavity of the human body through the inequality constraint of the positional relationship between the finite element nodes in the finite element model of the flexible device and the contact point target patches in the three-dimensional model of the natural cavity of the human body. It does not rely on the collision detection method, that is, there is no need to assume the contact points and collision positions between the flexible device and the natural cavity of the human body in advance. The contact and collision positions of the flexible device and the natural cavity of the human body can be directly determined by the method of the embodiment of the present application, and the contact and collision positions of the flexible device and the natural cavity of the human body are changed between different frames, so that the determined displacement and force results of the flexible device are more accurate.

In one embodiment, determining the force result of the flexible device according to the displacement of the flexible device includes:

The force result of the flexible device is determined according to the displacement of the flexible device and the stiffness matrix of the flexible device.

Specifically, after establishing a finite element model of the flexible device and determining the displacement _Δxi of the flexible device based on the minimum potential energy principle and the target constraint condition, the force result of the flexible device can be determined according to the determined displacement _Δxi of the flexible device. Optionally, the force result of the flexible device can be determined according to the following formula:
_fi =fi _-1 +K(xi _-1 ) _Δxi

Wherein, _fi represents the force result of the flexible instrument. Optionally, the force result of the flexible instrument is composed and determined based on the multi-dimensional finite element node force result. The force result of the flexible instrument includes the external force or the gravity of the flexible instrument when it contacts the natural cavity tissue of the human body; fi _-1 represents the force result of the flexible instrument at moment i-1, _Δxi represents the displacement of the flexible instrument, and K( _xi-1 ) represents the stiffness matrix of the flexible instrument.

In one embodiment, the stiffness matrix of the flexible device is determined based on a rotation matrix between a local coordinate system of an element in a finite element model of the flexible device and a global coordinate system.

Specifically, the stiffness matrix of the flexible device is a semi-positive definite band symmetric matrix, which is related to the physical properties of the flexible body and the deformation size under large deformation conditions. The efficiency of determining the displacement and force results of the flexible device depends on the dimension and processing efficiency of the flexible device stiffness matrix K( _xi-1 ). Optionally, the stiffness matrix of the flexible device is:

Where _Te is the rotation matrix between the unit coordinate system and the global coordinate system, which is related to the displacement x of the flexible device. The value of _Te needs to be recalculated in each frame calculation. is the unit stiffness, and a new stiffness matrix is recalculated for each calculation under large deformation conditions. ∑ is a stiffness matrix assembly method, which assembles unit stiffness matrices into an overall stiffness matrix. Optionally, in an embodiment of the present application, the displacement of the flexible instrument is determined based on the minimum potential energy principle by determining the corresponding displacement under the condition of the minimum potential energy change of the flexible instrument, and the stiffness matrix of the finite element model of the flexible instrument is re-determined between each frame. By combining the minimum potential energy change of the flexible instrument with the stiffness matrix, the deformation and force of the flexible instrument under large deformation conditions can be accurately simulated, and the simulation of large deformation flexible instruments can be realized, which can provide accurate prompt information of the displacement and force results of the flexible instrument for surgery, thereby improving the efficiency and safety of the surgery.

The method of the above embodiment determines the displacement of the flexible instrument based on the principle of minimum potential energy by determining the corresponding displacement under the condition of minimum potential energy change of the flexible instrument, and redetermines the stiffness matrix of the finite element model of the flexible instrument between each frame, thereby being able to accurately simulate the deformation and force of the flexible instrument under large deformation conditions, and can provide doctors with accurate displacement and force results of the flexible instrument, thereby improving the efficiency and safety of the operation.

In one embodiment, after determining the displacement of the flexible device, the method further includes:

Specifically, in the case where the natural cavity of the human body is a narrow and bifurcated cavity, due to the narrowness and bifurcation of the natural cavity of the human body, there is no penetration between the finite element nodes in the finite element model of the flexible instrument and the contact point target facets in the three-dimensional model of the natural cavity of the human body. However, there may still be penetration between the finite element nodes in the finite element model of the flexible instrument and other facets in the three-dimensional model of the natural cavity of the human body. Therefore, when it is determined that there is penetration between the finite element nodes in the finite element model of the flexible instrument and at least one of the multiple facets in the three-dimensional model of the natural cavity of the human body, the position constraint conditions between the flexible instrument and the natural cavity of the human body need to be updated; that is, according to the calculated each time The node penetration surface situation determines the inequality constraints of the positional relationship between the finite element nodes in the finite element model of the new flexible instrument and the facets in the three-dimensional model of the human body's natural cavity, until the finite element nodes of the flexible instrument converge with all the facets of the human body's natural cavity model when there is no more penetration, so that the determined displacement of the flexible instrument meets the actual situation of the operation, and the determined displacement and force results of the flexible instrument are more accurate, which can provide doctors with accurate flexible instrument displacement, force results and flexible instrument operation prompt information, which can ensure that the flexible instrument can pass through the narrow cavity of the human body better, is suitable for natural cavity surgeries such as bronchial surgeries, and improves the efficiency and safety of the operation.

Exemplarily, the operation prompt method of the flexible instrument as shown in Figure 3 first establishes a finite element model of the flexible instrument, and then based on the principle of minimum potential energy and the constraints of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the contact point target patches in the three-dimensional model of the natural cavity of the human body, the displacement of the flexible instrument can be determined; after determining the displacement of the flexible instrument, the force result of the flexible instrument can be determined based on the stiffness matrix of the flexible instrument, and the determined displacement and force result of the flexible instrument can be displayed, and the operation instruction information of the flexible instrument can be output. Optionally, in the case where the natural cavity of the human body is a narrow and bifurcated cavity, if it is determined that there is penetration between the finite element nodes in the finite element model of the flexible instrument and at least one of the multiple facets in the three-dimensional model of the natural cavity of the human body, the inequality constraints on the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the facets in the three-dimensional model of the natural cavity of the human body are updated until they converge when there is no more penetration between the finite element nodes of the flexible instrument and all the facets in the three-dimensional model of the natural cavity of the human body; by updating the constraints on the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the facets in the three-dimensional model of the natural cavity of the human body, the displacement and force results of the flexible instrument determined according to the constraints on the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the facets in the three-dimensional model of the natural cavity of the human body after the update can ensure that the flexible instrument can pass through the narrow cavity better, and is suitable for simulating natural cavity surgery in narrow spaces such as bronchial tubes. That is, the inequality constraint condition for the positional relationship between the finite element nodes in the finite element model of the updated flexible instrument and the facets in the three-dimensional model of the human body's natural cavity in the black arrow dotted line in Figure 3 is not a necessary step, but an optimization step in determining the displacement and force results of the flexible instrument in a narrow and bifurcated natural cavity of the human body. As a result, the method in the embodiment of the present application can more accurately determine the displacement and force results of the flexible instrument in a narrow and bifurcated natural cavity of the human body and the flexible instrument operation prompt information, which can ensure that the flexible instrument can pass through the narrow cavity of the human body better and improve the efficiency and safety of the operation.

The method of the above embodiment updates the constraint conditions of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the facets in the three-dimensional model of the natural cavity of the human body, thereby more accurately determining the displacement and force results of the flexible instrument in the narrow and bifurcated natural cavity of the human body and the flexible instrument operation prompt information, thereby ensuring that the flexible instrument can better pass through the narrow cavity of the human body and improving the efficiency and safety of the operation.

The operation prompt device of the flexible instrument provided in the present application is described below. The operation prompt device of the flexible instrument described below and the operation prompt method of the flexible instrument described above can be referred to each other.

FIG4 is a schematic diagram of the structure of the operation prompt device of the flexible device provided in the present application. The operation prompt device of the flexible device provided in this embodiment includes:

Establishing module 710, for establishing a finite element model of the flexible device; the finite element model of the flexible device includes a plurality of finite element nodes;

The first determination module 720 is used to determine the displacement of the flexible device based on the minimum potential energy principle and the target constraint condition; the target constraint condition is used to represent the constraint of the positional relationship between the finite element nodes in the finite element model of the flexible device and the contact point target surface patch in the three-dimensional model of the natural cavity of the human body; the displacement of the flexible device includes the displacement of each finite element node;

A second determination module 730 is used to determine the force result of the flexible device according to the displacement of the flexible device;

The prompt module 740 is used to output the operation instruction information of the flexible instrument according to the determined displacement of the flexible instrument and the force result of the flexible instrument; the operation instruction information of the flexible instrument is used to indicate the method of performing surgery on the natural cavity of the human body through the flexible instrument.

Optionally, the first determination module 720 is specifically used to determine the displacement of the flexible device when the potential energy change of the flexible device is minimized and the positional relationship between the flexible device and the natural cavity of the human body meets the target constraint condition; the target constraint condition is that the finite element nodes in the finite element model of the flexible device and the contact point target patches in the three-dimensional model of the natural cavity of the human body are not penetrable.

Optionally, the first determination module 720 is specifically configured to determine the displacement of the flexible device using the following formula:

or
_Axi ＝ _bi

represents the minimum change in potential energy of the flexible device; _Δxi represents the displacement of the flexible device from time i-1 to time i; K( _xi-1 ) represents the stiffness matrix of the flexible device; _xi-1 is the displacement of the finite element node at time i-1; F represents the target external force on the flexible device, and the target external force does not include the contact force between the flexible device and the natural cavity of the human body; or represents the target constraint; represents the surface normal vector of the contact point between the flexible instrument and the natural cavity of the human body; _xi represents the finite element node displacement at time i; _Si represents the generalized coordinate position of the flexible instrument in the undeformed state; _X0 represents any position on the plane corresponding to the three-dimensional model of the natural cavity of the human body; _Axi = _bi represents the first constraint condition; A represents the projection matrix; _bi represents the target position of the flexible instrument; _xi represents the finite element node displacement at time i.

Optionally, the second determination module 730 is specifically used to determine the force result of the flexible device according to the displacement of the flexible device, including:

Optionally, the second determination module 730 is further used to determine the stiffness matrix of the flexible device based on a rotation matrix between a unit local coordinate system and a global coordinate system in a finite element model of the flexible device.

Optionally, the second determination module 730 is further used to: determine the positional relationship between the finite element nodes in the finite element model of the flexible device and each surface patch in the three-dimensional model of the natural cavity of the human body based on the determined displacement of the flexible device when the natural cavity of the human body is a narrow bifurcated cavity;

The device of the embodiment of the present application is used to execute the method in any of the aforementioned method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

FIG5 illustrates a schematic diagram of the physical structure of an electronic device, which may include: a processor 810, a communications interface 820, a memory 830 and a communications bus 840, wherein the processor 810, the communications interface 820 and the memory 830 communicate with each other through the communications bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the operation prompt method of the flexible instrument, which method includes: establishing a finite element model of the flexible instrument; the finite element model of the flexible instrument includes a plurality of finite element nodes; based on the minimum potential energy principle and the target constraint condition, determining the displacement of the flexible instrument; the target constraint condition is used to represent the constraint of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the contact point target patch in the three-dimensional model of the natural cavity of the human body; the displacement of the flexible instrument includes the displacement of each finite element node; according to the displacement of the flexible instrument, determining the force result of the flexible instrument; according to the determined displacement of the flexible instrument and the force result of the flexible instrument, outputting the operation instruction information of the flexible instrument; The operation instruction information of the flexible instrument is used to indicate the method of performing surgery on the natural cavity of the human body through the flexible instrument.

In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

On the other hand, the present application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the operation prompt method of the flexible instrument provided by the above-mentioned methods, and the method includes: establishing a finite element model of the flexible instrument; the finite element model of the flexible instrument includes multiple finite element nodes; based on the minimum potential energy principle and target constraint conditions, determining the displacement of the flexible instrument; the target constraint conditions are used to represent the constraints on the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the contact point target patches in the three-dimensional model of the natural cavity of the human body; the displacement of the flexible instrument includes the displacement of each finite element node; according to the displacement of the flexible instrument, the force result of the flexible instrument is determined; according to the determined displacement of the flexible instrument and the force result of the flexible instrument, the operation instruction information of the flexible instrument is output; the operation instruction information of the flexible instrument is used to indicate the method of performing surgery on the natural cavity of the human body through the flexible instrument.

On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned flexible instrument operation prompt method, the method comprising: establishing a finite element model of the flexible instrument; the finite element model of the flexible instrument comprises a plurality of finite element nodes; based on the minimum potential energy principle and target constraint conditions, determining the displacement of the flexible instrument; the target constraint conditions are used to represent the constraints on the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the contact point target patches in the three-dimensional model of the natural cavity of the human body; the displacement of the flexible instrument comprises the displacement of each finite element node; based on the displacement of the flexible instrument, determining the force result of the flexible instrument; based on the determined displacement of the flexible instrument and the force result of the flexible instrument, outputting the operation instruction information of the flexible instrument; the operation instruction information of the flexible instrument is used to indicate the method of performing surgery on the natural cavity of the human body through the flexible instrument.

The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technologies therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

An operation prompt method for a flexible device, comprising:

Establishing a finite element model of a flexible device; the finite element model of the flexible device includes a plurality of finite element nodes;

Based on the minimum potential energy principle and target constraint conditions, the displacement of the flexible instrument is determined; the target constraint conditions are used to represent the constraints on the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the contact point target patches in the three-dimensional model of the natural cavity of the human body; the displacement of the flexible instrument includes the displacement of each finite element node;

Determine the force result of the flexible device according to the displacement of the flexible device;

According to the determined displacement of the flexible instrument and the force result of the flexible instrument, operation instruction information of the flexible instrument is output; the operation instruction information of the flexible instrument is used to indicate the way of performing surgery on the natural cavity of the human body through the flexible instrument.
The method for prompting an operation of a flexible device according to claim 1, wherein the step of determining the displacement of the flexible device based on the minimum potential energy principle and the target constraint condition comprises:

The displacement of the flexible device is determined when the potential energy change of the flexible device is minimal and the positional relationship between the flexible device and the natural cavity of the human body meets the target constraint condition; the target constraint condition is that the finite element nodes in the finite element model of the flexible device and the contact point target surfaces in the three-dimensional model of the natural cavity of the human body are not penetrable.
The operation prompt method of the flexible instrument according to claim 1 or 2, wherein the determining the displacement of the flexible instrument based on the minimum potential energy principle and the target constraint condition comprises:

The displacement of the flexible device is determined using the following formula:

Axi ＝ bi

Said represents the minimum change in potential energy of the flexible device; Δxi represents the displacement of the flexible device from time i-1 to time i; K( xi-1 ) represents the stiffness matrix of the flexible device; xi -1 is the displacement of the finite element node at time i-1; F represents the target external force on the flexible device, and the target external force does not include the contact force between the flexible device and the natural cavity of the human body; Represents the target constraint; represents the surface normal vector of the contact point between the flexible instrument and the natural cavity of the human body; the xi represents the finite element node displacement of the flexible instrument at time i; the Si represents the generalized coordinate position of the flexible instrument in the undeformed state; the X0 represents an arbitrary position on the plane corresponding to the three-dimensional model of the natural cavity of the human body; the Axi = bi represents the first constraint condition; the A represents the projection matrix; the bi represents the target position of the flexible instrument; the xi represents the finite element node displacement of the flexible instrument at time i.
The operation prompt method of the flexible instrument according to claim 3, wherein the step of determining the force result of the flexible instrument according to the displacement of the flexible instrument comprises:

The force result of the flexible instrument is determined according to the displacement of the flexible instrument and the stiffness matrix of the flexible instrument.
The method for prompting an operation of a flexible device according to claim 4, wherein the method further comprises:

The stiffness matrix of the flexible device is determined based on a rotation matrix between a local coordinate system of a unit and a global coordinate system in a finite element model of the flexible device.
The method for prompting operation of a flexible device according to claim 3, wherein after determining the displacement of the flexible device, the method further comprises:

In the case where the natural cavity of the human body is a narrow bifurcated cavity, based on the determined displacement of the flexible device, determining the positional relationship between the finite element nodes in the finite element model of the flexible device and each surface patch in the three-dimensional model of the natural cavity of the human body;

In the case where there is penetration between a finite element node in the finite element model of the flexible device and at least one of the plurality of facets in the three-dimensional model of the natural cavity of the human body, the target constraint condition is updated.
An operation prompting device for a flexible instrument, comprising:

An establishment module is used to establish a finite element model of a flexible device; the finite element model of the flexible device includes a plurality of finite element nodes;

The first determination module is used to determine the displacement of the flexible instrument based on the minimum potential energy principle and the target constraint condition; the target constraint condition is used to represent the constraint of the positional relationship between the finite element nodes in the finite element model of the flexible instrument and the contact point target surface patch in the three-dimensional model of the natural cavity of the human body; the displacement of the flexible instrument includes the displacement of each finite element node;

A second determination module is used to determine the force result of the flexible device according to the displacement of the flexible device;

The prompt module is used to output operation instruction information of the flexible instrument according to the determined displacement of the flexible instrument and the force result of the flexible instrument; the operation instruction information of the flexible instrument is used to indicate the method of performing surgery on the natural cavity of the human body through the flexible instrument.
An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method for prompting an operation of a flexible device as claimed in any one of claims 1 to 6 are implemented.
A non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the operation prompt method for a flexible device as described in any one of claims 1 to 6.
A computer program product stores executable instructions thereon, wherein when the instructions are executed by a processor, the processor implements the steps of the operation prompt method for a flexible device as claimed in any one of claims 1 to 6.