CN111026269B - Haptic feedback method, device and equipment for biological tissue structure based on force feedback - Google Patents
Haptic feedback method, device and equipment for biological tissue structure based on force feedback Download PDFInfo
- Publication number
- CN111026269B CN111026269B CN201911224558.7A CN201911224558A CN111026269B CN 111026269 B CN111026269 B CN 111026269B CN 201911224558 A CN201911224558 A CN 201911224558A CN 111026269 B CN111026269 B CN 111026269B
- Authority
- CN
- China
- Prior art keywords
- force
- feedback
- tissue structure
- force feedback
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B23/00—Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes
- G09B23/28—Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes for medicine
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Business, Economics & Management (AREA)
- Algebra (AREA)
- Computational Mathematics (AREA)
- Medical Informatics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Health & Medical Sciences (AREA)
- Pure & Applied Mathematics (AREA)
- Medicinal Chemistry (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- General Health & Medical Sciences (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Chemical & Material Sciences (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- User Interface Of Digital Computer (AREA)
- Instructional Devices (AREA)
Abstract
The application relates to a tactile feedback method, a device and equipment of a biological tissue structure based on force feedback. Wherein the method comprises the following steps: acquiring a corresponding real-time position of a stress point of force feedback equipment in a preset tissue structure information map; the tissue structure information map contains area information corresponding to each tissue structure of the living being; determining a current organization structure corresponding to the real-time position; based on preset tissue structure parameter information, calculating feedback force born by a stress point of force feedback equipment on a current tissue structure; the feedback force is transmitted to the force feedback device to drive the force feedback device to make a haptic force feedback response. By the arrangement, in the teaching training of the virtual puncture operation, a trainee can feel real and fine touch feedback force in interaction, the training level of the puncture operation can be greatly improved, and the problem of real operation feeling training in the virtual puncture operation training is solved.
Description
Technical Field
The application relates to the field of medical teaching simulation training, in particular to a biological tissue structure tactile feedback method, device and equipment based on force feedback.
Background
Puncture is one of the common clinical surgical procedures and important methods of administration and treatment, playing a decisive role in diagnosis, treatment and rescue of critical cases.
In traditional operation training, a practicing doctor often exercises biological targets such as an artificial model, a cadaver or an animal after demonstration by a doctor with abundant experience. However, artificial models are not realistic enough, cadaver resources are limited and expensive, and living tissues and structures of animals are greatly different from human bodies. Therefore, the force of the operation cannot be well mastered by the beginner, and the teacher has difficulty in accurately transmitting the experience of the beginner to the student.
On the basis, a virtual operation training scheme also exists, namely, basic operation skills of doctors are helped to be trained by constructing three-dimensional virtual scenes of operations, three-dimensional virtual patients, simulating basic operation processes and the like. However, the virtual operation training has weak interactive feeling and lacks direct feedback on touch sense, is more suitable for the training of operation flow and has poor feeling culture on actual operation.
That is, the existing virtual puncture surgery training method is difficult to cultivate the actual manipulation feeling.
Disclosure of Invention
The application provides a tactile feedback method, a device and equipment of a biological tissue structure based on force feedback, which are used for solving the problem that the conventional virtual puncture operation training method is difficult to cultivate the actual operation feeling.
The above object of the present application is achieved by the following technical solutions:
in a first aspect, an embodiment of the present application provides a haptic feedback method for a biological tissue structure based on force feedback, including:
acquiring a corresponding real-time position of a stress point of force feedback equipment in a preset tissue structure information map; the tissue structure information map comprises area information corresponding to each tissue structure of the living being;
determining a current organization structure corresponding to the real-time position;
based on preset tissue structure parameter information, calculating feedback force born by a stress point of force feedback equipment on a current tissue structure;
transmitting the feedback force to the force feedback device to drive the force feedback device to make a haptic force feedback response.
Optionally, the obtaining the corresponding real-time position of the stress point of the force feedback device in the preset tissue structure information map includes:
acquiring the position of the force feedback device in a 3D coordinate system of a working space;
mapping the working space and tissue structure information of the force feedback equipment to coordinates, so that the position under the 3D coordinate system is corresponding to the position under the 2D coordinate system;
and acquiring the real-time position of the stress point of the force feedback equipment in the 2D coordinate system where the tissue structure information map is positioned through a ray collision detection technology.
Optionally, the process for creating the organization structure information map includes:
performing section radiography on a part to be punctured of a living body to obtain a biological tissue structure section view;
and dividing the biological tissue structure section according to the tissue structure to form a corresponding region information diagram.
Optionally, the method further comprises:
each tissue structure is represented by a different color, and the region range of each color represents the region range of the corresponding tissue structure.
Optionally, the calculating, based on the preset tissue structure parameter information, the feedback force received by the stress point of the force feedback device at the current tissue structure includes:
acquiring the operation force of a user on the force feedback equipment;
calculating the product of the displacement of the stress point of the force feedback equipment in the current tissue structure and a preset resistance coefficient to obtain the resistance of the force feedback equipment;
and calculating the difference between the operation force and the resistance force to obtain the feedback force.
Optionally, the acquiring the operation force of the user on the force feedback device includes:
collecting real-time data of force feedback equipment according to a preset first time interval;
decoding the real-time data into data which can be identified during calculation;
sequentially judging whether the decoded real-time data contains noise data or not;
the noise data is discarded and the operating force is determined with the remaining data.
Optionally, the transmitting the feedback force to the force feedback device to drive the force feedback device to make a haptic force feedback response, including:
and encoding the feedback force into data which can be identified by the force feedback device, transmitting the data to the force feedback device, and enabling the force feedback device to make a tactile force feedback response.
Optionally, the method further comprises:
if any operation data of the force feedback equipment is not obtained within a preset second time interval, controlling the force feedback equipment to enter an energy-saving standby state;
and if the temperature of the force feedback equipment is detected to exceed the preset temperature threshold, controlling the force feedback equipment to reduce the performance or closing the force feedback equipment.
In a second aspect, embodiments of the present application further provide a haptic feedback device for a force feedback-based biological tissue structure, including:
the acquisition module is used for acquiring the corresponding real-time position of the stress point of the force feedback equipment in the preset tissue structure information map; the tissue structure information map comprises area information corresponding to each tissue structure of the living being;
the determining module is used for determining the current organization structure corresponding to the real-time position;
the calculation module is used for calculating the feedback force of the stress point of the force feedback equipment on the current tissue structure based on the preset tissue structure parameter information;
and the transmission module is used for transmitting the feedback force to the force feedback equipment so as to drive the force feedback equipment to make a tactile force feedback response.
In a third aspect, embodiments of the present application also provide a haptic feedback device for force feedback-based biological tissue structures, comprising:
a memory and a processor coupled to the memory;
the memory is used for storing a program, and the program is at least used for executing the biological tissue structure tactile feedback method based on force feedback;
the processor is used for calling and executing the program stored in the memory.
The technical scheme provided by the embodiment of the application can comprise the following beneficial effects:
according to the technical scheme provided by the embodiment of the application, the operation force data of the user on the force feedback equipment is obtained, the feedback force is obtained by carrying out parameter calculation according to the preset biological tissue structure parameter information and is transmitted to the force feedback equipment for response, so that the puncture operation of the user on the real organ can be simulated. By the arrangement, in the teaching and training of the virtual puncture operation, a trainee can feel real and fine touch feedback force in interaction. The trainee interacts with the virtual organ model to complete a complete operation training interaction operation on the virtual organ model, and the training level of the puncture operation can be greatly improved as the operation of the real organ, so that the problem of actual operation feeling culture in the virtual puncture operation training is solved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
Fig. 1 is a schematic flow chart of a haptic feedback method of a biological tissue structure based on force feedback according to an embodiment of the present application;
fig. 2 is a schematic diagram of mapping a 3D coordinate to a 2D coordinate according to an embodiment of the present application;
FIG. 3 is a schematic diagram of the stress situation of a tissue structure in a puncture operation according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a deformation principle of a force feedback model based on a mass-spring model according to an embodiment of the present application;
FIG. 5 is a graph of feedback force before a tissue structure is pierced, according to an embodiment of the present application;
FIG. 6 is a graph of feedback force after a tissue structure is pierced, according to an embodiment of the present application;
FIG. 7 is a graph of feedback force after interpolation according to an embodiment of the present application;
FIG. 8 is a schematic structural diagram of a haptic feedback device with a force-feedback-based biological tissue structure according to an embodiment of the present application;
fig. 9 is a schematic structural diagram of a haptic feedback device with a force feedback-based biological tissue structure according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of apparatus and methods consistent with aspects of the application as detailed in the accompanying claims.
Currently, in the training of puncture surgery, the existing methods mainly include the following two methods:
artificial model scheme: the proposal simulates the structural characteristics of human body mainly by adopting an artificial model similar to the operation requirement. The artificial model has stronger tactile feedback due to the adoption of deformable materials such as silica gel and the like, but the model can be aged and damaged along with repeated puncture or cutting use, and the repeated utilization rate is low. In addition, the scheme can not train the whole operation flow, lacks statistics and evaluation of operation, can not depart from guidance and supervision of teachers, and does not meet the requirements of medical teaching.
Virtual surgical scheme: the virtual surgical system can help train basic surgical skills of doctors by constructing three-dimensional virtual scenes of surgery, three-dimensional virtual patients, simulating basic surgical procedures and the like. Different from the traditional training mode, the virtual operation system can be repeatedly practiced without worrying about the problems of specimens, sites, operation safety and the like, so that the training cost is lower, and the training period is shortened. But the interactive feeling of the operation is weaker, the direct feedback on the touch sense is lacking, the method is more suitable for the training of the operation flow, and the feeling culture of the actual operation is poorer.
In order to solve the above problems, the embodiments of the present application provide the following technical solutions.
Examples
Referring to fig. 1, fig. 1 is a flowchart of a haptic feedback method for a force feedback-based biological tissue structure according to an embodiment of the present application, wherein the method is generally performed by computer software. As shown in fig. 1, the method comprises the steps of:
s101: acquiring a corresponding real-time position of a stress point of force feedback equipment in a preset tissue structure information map; the tissue structure information map comprises area information corresponding to each tissue structure of the living being;
specifically, the force feedback device is a virtual 3D environment and an interoperable platform, and is mainly applied to virtual surgery, games, and the like. This embodiment may employ the openhaptics model 3.4 product offered by 3dsystems as the application base for haptic feedback. Of course, other similar devices may be used, and this is not a limitation. Furthermore, for simulated puncture surgery, the point of force applied by the force feedback device corresponds to the needle of the surgical instrument needle.
Biological tissue structures refer to biological subcutaneous tissue structures such as fat layers, cartilage layers, and the like.
In some embodiments, the process of creating the organizational structure information map includes:
performing section radiography on a part to be punctured of a living body to obtain a biological tissue structure section view; and dividing the biological tissue structure section according to the tissue structure to form a corresponding region information diagram.
The biological tissue structure section can be obtained directly from medical standard section image data, and can be manufactured by self according to a related method. Further, for better observation, different tissue structures may be represented by different colors, so that the region ranges of the respective colors represent the region ranges of the corresponding tissue structures.
In some embodiments, the actual process of obtaining the corresponding real-time position of the stress point of the force feedback device in the preset tissue structure information map includes: acquiring the position of the force feedback device in a 3D coordinate system of a working space; mapping the working space and tissue structure information of the force feedback equipment to coordinates, so that the position under the 3D coordinate system is corresponding to the position under the 2D coordinate system; and acquiring the real-time position of the stress point of the force feedback equipment in the 2D coordinate system where the tissue structure information map is positioned through a ray collision detection technology.
In the coordinate mapping, the plane in which the puncture direction of the puncture needle is located is usually mapped to the tissue structure information map. The ray collision detection technology is a collision detection technology in the Unity 3D engine, and can be used for detecting whether an object (collision body) is in contact with other objects (i.e. collision is generated). In this embodiment, the technique may be used to take the penetrating direction of the puncture needle as a ray, so as to obtain the position of the needle in the position corresponding to the tissue structure information map. The results are shown in fig. 2, wherein the arrow is shown on the left side as force feedback device (top view) and on the right side as tissue structure information map.
S102: determining a current organization structure corresponding to the real-time position;
specifically, on the basis of the above steps, if different tissue structures are represented by different colors, the corresponding tissue structures can be determined directly by acquiring the color parameters.
S103: based on preset tissue structure parameter information, calculating feedback force born by a stress point of force feedback equipment on a current tissue structure;
the process of calculating the feedback force in this embodiment includes: acquiring the operation force of a user on the force feedback equipment; i.e. the user holds the lever of the force feedback device, the applied operating force; calculating the product of the displacement of the stress point of the force feedback equipment in the current tissue structure and a preset resistance coefficient to obtain the resistance of the force feedback equipment; and calculating the difference between the operation force and the resistance force to obtain the feedback force. The resistance coefficient is determined by preset tissue structure parameter information. Further, the process of obtaining the operation force of the user on the force feedback device includes: collecting real-time data of force feedback equipment according to a preset first time interval; decoding the real-time data into data which can be identified during calculation; sequentially judging whether the decoded real-time data contains noise data or not; the noise data is discarded and the operating force is determined with the remaining data.
The first time interval may be 10 ms to 120 ms, the larger the time interval is, the higher the device performance is, but the lower the sampling accuracy is, 60 ms is generally taken as a preferable scheme. The noise data is sampling data interfered by other external signals, when judging the noise data, the current data and the data acquired before can be compared, and if the current data does not accord with the change rule, the noise data is needed to be discarded. Furthermore, it should be noted that the form of the data signal is different from that which can be recognized by the force feedback device (hardware) and the computing software. Thus, after the user's operating force on the force feedback device is obtained, and before the feedback force is transmitted to the force feedback device, the data needs to be decoded into software or encoded into hardware-recognizable data. Wherein the decoded and encoded protocol parameters are determined by the selected force feedback device.
Before calculating the feedback force, the stress condition of the tissue structure in the puncture operation will be described first, and as shown in fig. 3, when the surgical instrument enters the tissue structure layer of the living body with a certain operation force (F), the surface will not be punctured immediately due to a certain toughness, and a process of extrusion is provided in the middle, in which the operator will feel a relatively strong surface elasticity (F 1 ) When the surgical instrument is continuously forced, the puncture is completed after the operation force is greater than the maximum surface elastic force, and an inertia (F 2 ) The process, since each tissue structure has its own density, causes resistance to the penetration process (F 3 ) Inertial force (F) 3 ) And resistance (F) 2 ) And mutually abut against each other to finally penetrate the tissue.
The process of calculating the feedback force (F) described above is formulated as: f=f- (F) 1 +F 2 +F 3 )
In order to more intuitively explain the above-described process, a specific example will be described below.
In some embodiments, as shown in the following table, for each organization structure, the preset organization structure parameter information may include: bending strength coefficient_force, stress area coefficient_tool, maximum constraint force_maxf, toughness coefficient_fade, inertia coefficient_project, density coefficient_density, resistance_damp, and hierarchy identification Color value_color.
On this basis, with reference to fig. 3, the calculation formula of each parameter in the above formula is:
f＝_Force
F 1 ＝L 1 *_Fade
F 2 ＝(1-L 2 )*_Tension
F 3 ＝L 3 *_Density
wherein L is 1 、L 2 And L 3 And forming a stress area coefficient (Tough).
Based on this, the feedback force F is calculated as
F＝f-(F 1 +F 2 +F 3 )＝_Force-[L 1 *_Fade+(1-L 2 )*_Tension+L 3 *_Density]
For surface elasticity F 1 From a biomechanical point of view, the subcutaneous tissue structure portion can generally be regarded as being approximately isotropic (isotropy refers to a property of an object that does not change in terms of its physical, chemical, etc. properties, i.e. the measured property values of an object in different directions are exactly the same, also called homogeneity). Modeling of these tissue structures can build a linear elastic model of the soft tissue under this approximate condition. Assuming a mass-spring-damper system (the mass-spring-damper system can be represented by a simple mathematical model, namely, a numerical axis modeling method is adopted, a numerical axis parallel to the system is established, an actual displacement value of a spring or a damper is regarded as a rational number and is marked at a corresponding position on the numerical axis, then the magnitude and the direction of spring force or damping force borne by a corresponding mass block are determined according to the result of the rational number comparison, and further a differential equation of the system is obtained), mass points are represented by nodes in a geometric model, and an influence relation is established between the nodes m through virtual springs. The motion of each particle satisfies the LagrangianEquation of motion. The mass point-spring-damper model is an expansion of a viscoelastic vigot model, and can better describe the deformation and other characteristics of the subcutaneous tissue structure model. The deformation principle of the force feedback model built on the basis of the mass-spring model is shown in figure 4.
The model is provided with a spring and a damper attached to the surface of soft tissue, and the vertex of the virtual surgical instrument model generates elastic deformation once colliding with the surface, and by obtaining a vector formed by the original position and the new position of the vertex of the virtual instrument model, the intersecting contact point of the soft tissue surface can be obtained, and meanwhile, the length x of the spring stretching can be obtained, and the mathematical description calculated according to the surface elasticity of the model is as follows:
wherein M is the mass of the tissue surface, D is the damping (resistance) coefficient, K is the spring coefficient of the spring, x is the scaling displacement (needle penetration depth) of the spring, F 1 And (t) is the surface elasticity at a certain time.
It should be noted that, in the above model, mathematical description of the surface elasticity is used as an illustration principle, when the method of the present embodiment is applied, a toughness coefficient_fade may be preset for the tissue structure layer based on the principle, so that the toughness coefficient_fade and the displacement (needle insertion depth) L of the puncture needle may be calculated 1 To calculate the surface spring force F 1 . L before puncturing 2 And L 3 Are all 0, L 1 Gradually increase, thus F 1 Gradually increasing. The feedback force versus time in this process is shown in fig. 5.
Further, in order to simulate the sense of breakthrough at the time of skin puncture, a maximum restraining force (_maxf) is added to the feedback force of each layer, and when it is greater than the maximum restraining force, the force release process needs to be started. The relationship of feedback force to time after adding the maximum constraint force is shown in fig. 6.
Wherein, feedback force F satisfies: f= _maxf (t 1 -t 2 )*s
Wherein t is 2 Is opened toStart time, t 1 And s is a breakthrough scaling factor for adjusting the intensity of breakthrough force for the current time.
Further, in order to prevent the shaking of the force, it is necessary to linearly interpolate the calculated feedback force so as to have a linear characteristic. Interpolation is an important method of discrete function approximation, by which the approximation of a function at other points can be estimated from the value condition of the function at a limited number of points. The interpolation processing process can be implemented by Mathf. Lerp function, namely:
Mathf.lerp(min,max,F)＝newF
wherein min is the minimum value of the feedback force, max is the maximum value of the feedback force, F is the current feedback force, and newF is the final output result.
After the interpolation process, a graph as shown in fig. 7 is obtained, and the feedback force represented by the graph is the calculation result to be finally transmitted to the force feedback device.
S104: transmitting the feedback force to the force feedback device to drive the force feedback device to make a haptic force feedback response.
As described above, this step includes: and combining the calculated feedback forces in the first time interval, performing interpolation processing to obtain a continuous smooth feedback force curve, then encoding the continuous smooth feedback force curve into data which can be identified by the force feedback equipment, and finally transmitting the data to the force feedback equipment so as to enable the force feedback equipment to make a tactile force feedback response.
According to the technical scheme provided by the embodiment of the application, the operation force data of the user on the force feedback equipment is obtained, the feedback force is obtained by carrying out parameter calculation according to the preset biological tissue structure parameter information and is transmitted to the force feedback equipment for response, so that the puncture operation of the user on the real organ can be simulated. By the arrangement, in the teaching and training of the virtual puncture operation, a trainee can feel real and fine touch feedback force in interaction. The trainee interacts with the virtual organ model to complete a complete operation training interaction operation on the virtual organ model, and the training level of the puncture operation can be greatly improved as the operation of the real organ, so that the problem of actual operation feeling culture in the virtual puncture operation training is solved.
In addition, the computer software is also used for detecting and protecting the state of the force feedback equipment, such as idle standby, overheat protection and the like. Namely: if any operation data of the force feedback equipment is not obtained within a preset second time interval, controlling the force feedback equipment to enter an energy-saving standby state; and if the temperature of the force feedback equipment is detected to exceed the preset temperature threshold, controlling the force feedback equipment to reduce the performance or closing the force feedback equipment.
The second time interval may be set according to actual needs, for example, set to 30 seconds, that is, if the user does not operate the force feedback device within 30 seconds, the force feedback device enters an energy-saving standby state, so that energy consumption of the force feedback device is reduced. And protects the force feedback device when it is overheated.
In order to more fully describe the technical scheme of the application, the embodiment of the application also provides a tactile feedback device of the biological tissue structure based on force feedback, which corresponds to the tactile feedback method of the biological tissue structure based on force feedback provided by the embodiment of the application.
Referring to fig. 8, fig. 8 is a schematic structural diagram of a tactile feedback device with a biological tissue structure based on force feedback according to an embodiment of the application. As shown in fig. 8, the apparatus includes:
the acquiring module 81 is configured to acquire a real-time position of a stress point of the force feedback device corresponding to a preset tissue structure information map; the tissue structure information map comprises area information corresponding to each tissue structure of the living being;
a determining module 82, configured to determine a current organization structure corresponding to the real-time location;
the calculating module 83 is configured to calculate, based on preset tissue structure parameter information, a feedback force that the stress point of the force feedback device receives in the current tissue structure;
a transmission module 84 for transmitting the feedback force to the force feedback device for driving the force feedback device to make a haptic force feedback response.
In particular, the specific implementation of the function of each module may be implemented by referring to the content in the tactile feedback method of the biological tissue structure based on force feedback, which is not described in detail herein.
In order to more fully describe the technical scheme of the application, the embodiment of the application also provides a tactile feedback device of the biological tissue structure based on force feedback, which corresponds to the tactile feedback method of the biological tissue structure based on force feedback provided by the embodiment of the application.
Referring to fig. 9, fig. 9 is a schematic structural diagram of a haptic feedback device with a force feedback-based biological tissue structure according to an embodiment of the application. As shown in fig. 9, the apparatus includes:
a memory 91 and a processor 92 connected to the memory 91;
the memory 91 is used for storing a program at least for executing the above-described force feedback-based tactile feedback method of biological tissue structures;
the processor 92 is used to call and execute the program stored in the memory 91.
In particular, the device is usually a computer, and the implementation of the functions of the program is described with reference to the relevant content in the haptic feedback method of the biological tissue structure based on force feedback, which is not described in detail herein.
It is to be understood that the same or similar parts in the above embodiments may be referred to each other, and that in some embodiments, the same or similar parts in other embodiments may be referred to.
It should be noted that in the description of the present application, the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Furthermore, in the description of the present application, unless otherwise indicated, the meaning of "plurality" means at least two.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and further implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or a portion of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, and where the program, when executed, includes one or a combination of the steps of the method embodiments.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product.
The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, or the like.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the present application have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the application, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the application.
Claims (10)
1. A method of tactile feedback of a biological tissue structure based on force feedback, comprising:
acquiring a corresponding real-time position of a stress point of force feedback equipment in a preset tissue structure information map; the tissue structure information map comprises area information corresponding to each tissue structure of the living being;
determining a current organization structure corresponding to the real-time position;
based on preset tissue structure parameter information, calculating feedback force born by a stress point of force feedback equipment on a current tissue structure; the preset tissue structure parameter information comprises: bending strength coefficient_force, stress area coefficient_tool, maximum constraint force_maxf, toughness coefficient_fade, inertia coefficient_project, density coefficient_density, resistance_damp and hierarchy identification Color value_color;
transmitting the feedback force to a force feedback device to drive the force feedback device to make a haptic force feedback response;
after the feedback force is calculated, the method further comprises: adding a maximum constraint force to the feedback force of each layer, and when the feedback force is larger than the maximum constraint force, starting force unloading processing is needed, wherein the feedback force meets the following conditions: f= _maxf (t 1 -t 2 )*s；
Wherein t is 2 To start toTime t 1 S is a breakthrough scaling factor for adjusting the intensity of breakthrough force, maxF is the maximum constraint force, and F is the feedback force;
and carrying out linear interpolation processing on the feedback force, wherein the linear interpolation processing is as follows: mathf lerp (min, max, F) =newf;
wherein min is the minimum value of the feedback force, max is the maximum value of the feedback force, F is the current feedback force, and newF is the final output result.
2. The method of claim 1, wherein the obtaining the corresponding real-time position of the stress point of the force feedback device in the preset tissue structure information map comprises:
acquiring the position of the force feedback device in a 3D coordinate system of a working space;
mapping the working space and tissue structure information of the force feedback equipment to coordinates, so that the position under the 3D coordinate system is corresponding to the position under the 2D coordinate system;
and acquiring the real-time position of the stress point of the force feedback equipment in the 2D coordinate system where the tissue structure information map is positioned through a ray collision detection technology.
3. The method of claim 1, wherein the process of creating the organizational structure information map comprises:
performing section radiography on a part to be punctured of a living body to obtain a biological tissue structure section view;
and dividing the biological tissue structure section according to the tissue structure to form a corresponding region information diagram.
4. A method according to claim 3, further comprising:
each tissue structure is represented by a different color, and the region range of each color represents the region range of the corresponding tissue structure.
5. The method according to claim 1, wherein calculating the feedback force applied to the current tissue structure by the force-applying point of the force feedback device based on the preset tissue structure parameter information includes:
acquiring the operation force of a user on the force feedback equipment;
calculating the product of the displacement of the stress point of the force feedback equipment in the current tissue structure and a preset resistance coefficient to obtain the resistance of the force feedback equipment;
and calculating the difference between the operation force and the resistance force to obtain the feedback force.
6. The method of claim 5, wherein the obtaining the user's operating force on the force feedback device comprises:
collecting real-time data of force feedback equipment according to a preset first time interval;
decoding the real-time data into data which can be identified during calculation;
sequentially judging whether the decoded real-time data contains noise data or not;
the noise data is discarded and the operating force is determined with the remaining data.
7. The method of claim 6, wherein transmitting the feedback force to a force feedback device to drive the force feedback device to make a haptic force feedback response, comprising:
and encoding the feedback force into data which can be identified by the force feedback device, transmitting the data to the force feedback device, and enabling the force feedback device to make a tactile force feedback response.
8. The method of any one of claims 1-7, further comprising:
if any operation data of the force feedback equipment is not obtained within a preset second time interval, controlling the force feedback equipment to enter an energy-saving standby state;
and if the temperature of the force feedback equipment is detected to exceed the preset temperature threshold, controlling the force feedback equipment to reduce the performance or closing the force feedback equipment.
9. A force feedback-based tactile feedback device for biological tissue structures, comprising:
the acquisition module is used for acquiring the corresponding real-time position of the stress point of the force feedback equipment in the preset tissue structure information map; the tissue structure information map comprises area information corresponding to each tissue structure of the living being;
the determining module is used for determining the current organization structure corresponding to the real-time position;
the calculation module is used for calculating the feedback force of the stress point of the force feedback equipment on the current tissue structure based on the preset tissue structure parameter information; the preset tissue structure parameter information comprises: bending strength coefficient_force, stress area coefficient_tool, maximum constraint force_maxf, toughness coefficient_fade, inertia coefficient_project, density coefficient_density, resistance_damp and hierarchy identification Color value_color;
the transmission module is used for transmitting the feedback force to the force feedback equipment so as to enable the force feedback equipment to make a tactile force feedback response;
after the feedback force is calculated, the method further comprises: adding a maximum constraint force to the feedback force of each layer, and when the feedback force is larger than the maximum constraint force, starting force unloading processing is needed, wherein the feedback force meets the following conditions: f= _maxf (t 1 -t 2 )*s；
Wherein t is 2 To start time, t 1 S is a breakthrough scaling factor for adjusting the intensity of breakthrough force, maxF is the maximum constraint force, and F is the feedback force;
and carrying out linear interpolation processing on the feedback force, wherein the linear interpolation processing is as follows: mathf lerp (min, max, F) =newf;
wherein min is the minimum value of the feedback force, max is the maximum value of the feedback force, F is the current feedback force, and newF is the final output result.
10. A force feedback-based tactile feedback device for biological tissue structures, comprising:
a memory and a processor coupled to the memory;
the memory for storing a program for at least performing a force feedback based biological tissue structure haptic feedback method as claimed in any one of claims 1-8;
the processor is used for calling and executing the program stored in the memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911224558.7A CN111026269B (en) | 2019-12-04 | 2019-12-04 | Haptic feedback method, device and equipment for biological tissue structure based on force feedback |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911224558.7A CN111026269B (en) | 2019-12-04 | 2019-12-04 | Haptic feedback method, device and equipment for biological tissue structure based on force feedback |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111026269A CN111026269A (en) | 2020-04-17 |
| CN111026269B true CN111026269B (en) | 2023-10-10 |
Family
ID=70204181
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911224558.7A Active CN111026269B (en) | 2019-12-04 | 2019-12-04 | Haptic feedback method, device and equipment for biological tissue structure based on force feedback |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111026269B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111782066B (en) * | 2020-06-30 | 2021-11-16 | 联想(北京)有限公司 | Electronic equipment |
| CN112422829B (en) * | 2020-11-19 | 2022-04-26 | 北京字节跳动网络技术有限公司 | Method, device, terminal and storage medium for assisting in shooting image |
| CN112690757B (en) * | 2020-12-17 | 2022-06-14 | 成都柔电云科科技有限公司 | Pressure damage monitoring and feedback equipment and feedback regulation system |
| CN114533138B (en) * | 2021-05-31 | 2023-05-02 | 四川大学 | Automatic puncture mechanism integrating touch sense |
Citations (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101400306A (en) * | 2006-03-09 | 2009-04-01 | 画像诊断株式会社 | Medical 3-dimensional image display control program and medical 3-dimensional image display method |
| CN102207997A (en) * | 2011-06-07 | 2011-10-05 | 哈尔滨工业大学 | Force-feedback-based robot micro-wound operation simulating system |
| CN102254476A (en) * | 2011-07-18 | 2011-11-23 | 广州赛宝联睿信息科技有限公司 | Endoscopic minimally invasive surgery simulation training method and system |
| CN103269451A (en) * | 2013-04-27 | 2013-08-28 | 沈阳理工大学 | Automatic shutdown method using video monitoring |
| CN103400023A (en) * | 2013-06-28 | 2013-11-20 | 华北水利水电大学 | Soft tissue deformation simulation method |
| CN104272365A (en) * | 2012-04-13 | 2015-01-07 | 汤姆逊许可公司 | Method to render global 6 DoF motion effect with multiple local force-feedback |
| CN104537938A (en) * | 2014-12-21 | 2015-04-22 | 合肥德铭电子有限公司 | Endoscope simulative training system capable of achieving flexible force feedback |
| CN105069301A (en) * | 2015-08-14 | 2015-11-18 | 南通大学 | Lumbar puncture virtual simulation and training system supporting haptic interaction |
| CN105303605A (en) * | 2015-10-26 | 2016-02-03 | 哈尔滨理工大学 | Orthopedic surgery operation simulation system on the basis of force feedback |
| CN105446362A (en) * | 2015-12-07 | 2016-03-30 | 陆宁远 | Posture detection adjusting device and method based on assistance of computer science |
| CN105559887A (en) * | 2015-12-11 | 2016-05-11 | 哈尔滨工业大学 | Operation cutting training system and method based on force feedback and used for surgical robot |
| CN105913718A (en) * | 2016-07-08 | 2016-08-31 | 哈尔滨理工大学 | Thread lift plastic surgery simulation system |
| CN105931517A (en) * | 2016-06-17 | 2016-09-07 | 重庆金山科技（集团）有限公司 | Surgical operation virtual simulation method and system |
| CN106725762A (en) * | 2016-12-30 | 2017-05-31 | 西安中科微光影像技术有限公司 | A kind of automatic puncturing method |
| CN106781941A (en) * | 2016-11-24 | 2017-05-31 | 北京理工大学 | A kind of method and its system for simulating microtrauma puncture operation |
| CN107820715A (en) * | 2017-08-09 | 2018-03-20 | 北京小米移动软件有限公司 | Method for excessive heating protection, device, user equipment and the base station of user equipment |
| CN107992010A (en) * | 2017-12-14 | 2018-05-04 | 万泰机电工业(昆山)有限公司 | New-energy automobile part produces tracing information generation method and traceability system |
| CN108428142A (en) * | 2018-05-18 | 2018-08-21 | 广州质量云信息科技有限公司 | Quick Response Code traceability management system |
| CN108648548A (en) * | 2018-04-19 | 2018-10-12 | 浙江工业大学 | A kind of neuro-surgery virtual operation training system |
| CN109308739A (en) * | 2018-10-11 | 2019-02-05 | 南京工程学院 | A kind of soft tissue Minimally Invasive Surgery training method based on virtual reality |
| CN109756791A (en) * | 2017-11-07 | 2019-05-14 | 中兴通讯股份有限公司 | A kind of network equipment guard method, protection equipment and computer readable storage medium |
| CN109934479A (en) * | 2019-03-06 | 2019-06-25 | 南昌易美光电科技有限公司 | The management system and method for production material |
| CN110046406A (en) * | 2019-03-28 | 2019-07-23 | 天津大学 | A kind of soft tissue emulation mode with force feedback structure in anatomic teaching system |
| CN110279467A (en) * | 2019-06-19 | 2019-09-27 | 天津大学 | Ultrasound image under optical alignment and information fusion method in the art of puncture biopsy needle |
| CN111150461A (en) * | 2020-02-04 | 2020-05-15 | 赵天力 | Puncture needle positioning system and method |
-
2019
- 2019-12-04 CN CN201911224558.7A patent/CN111026269B/en active Active
Patent Citations (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101400306A (en) * | 2006-03-09 | 2009-04-01 | 画像诊断株式会社 | Medical 3-dimensional image display control program and medical 3-dimensional image display method |
| CN102207997A (en) * | 2011-06-07 | 2011-10-05 | 哈尔滨工业大学 | Force-feedback-based robot micro-wound operation simulating system |
| CN102254476A (en) * | 2011-07-18 | 2011-11-23 | 广州赛宝联睿信息科技有限公司 | Endoscopic minimally invasive surgery simulation training method and system |
| CN104272365A (en) * | 2012-04-13 | 2015-01-07 | 汤姆逊许可公司 | Method to render global 6 DoF motion effect with multiple local force-feedback |
| CN103269451A (en) * | 2013-04-27 | 2013-08-28 | 沈阳理工大学 | Automatic shutdown method using video monitoring |
| CN103400023A (en) * | 2013-06-28 | 2013-11-20 | 华北水利水电大学 | Soft tissue deformation simulation method |
| CN104537938A (en) * | 2014-12-21 | 2015-04-22 | 合肥德铭电子有限公司 | Endoscope simulative training system capable of achieving flexible force feedback |
| CN105069301A (en) * | 2015-08-14 | 2015-11-18 | 南通大学 | Lumbar puncture virtual simulation and training system supporting haptic interaction |
| CN105303605A (en) * | 2015-10-26 | 2016-02-03 | 哈尔滨理工大学 | Orthopedic surgery operation simulation system on the basis of force feedback |
| CN105446362A (en) * | 2015-12-07 | 2016-03-30 | 陆宁远 | Posture detection adjusting device and method based on assistance of computer science |
| CN105559887A (en) * | 2015-12-11 | 2016-05-11 | 哈尔滨工业大学 | Operation cutting training system and method based on force feedback and used for surgical robot |
| CN105931517A (en) * | 2016-06-17 | 2016-09-07 | 重庆金山科技（集团）有限公司 | Surgical operation virtual simulation method and system |
| CN105913718A (en) * | 2016-07-08 | 2016-08-31 | 哈尔滨理工大学 | Thread lift plastic surgery simulation system |
| CN106781941A (en) * | 2016-11-24 | 2017-05-31 | 北京理工大学 | A kind of method and its system for simulating microtrauma puncture operation |
| CN106725762A (en) * | 2016-12-30 | 2017-05-31 | 西安中科微光影像技术有限公司 | A kind of automatic puncturing method |
| CN107820715A (en) * | 2017-08-09 | 2018-03-20 | 北京小米移动软件有限公司 | Method for excessive heating protection, device, user equipment and the base station of user equipment |
| CN109756791A (en) * | 2017-11-07 | 2019-05-14 | 中兴通讯股份有限公司 | A kind of network equipment guard method, protection equipment and computer readable storage medium |
| CN107992010A (en) * | 2017-12-14 | 2018-05-04 | 万泰机电工业(昆山)有限公司 | New-energy automobile part produces tracing information generation method and traceability system |
| CN108648548A (en) * | 2018-04-19 | 2018-10-12 | 浙江工业大学 | A kind of neuro-surgery virtual operation training system |
| CN108428142A (en) * | 2018-05-18 | 2018-08-21 | 广州质量云信息科技有限公司 | Quick Response Code traceability management system |
| CN109308739A (en) * | 2018-10-11 | 2019-02-05 | 南京工程学院 | A kind of soft tissue Minimally Invasive Surgery training method based on virtual reality |
| CN109934479A (en) * | 2019-03-06 | 2019-06-25 | 南昌易美光电科技有限公司 | The management system and method for production material |
| CN110046406A (en) * | 2019-03-28 | 2019-07-23 | 天津大学 | A kind of soft tissue emulation mode with force feedback structure in anatomic teaching system |
| CN110279467A (en) * | 2019-06-19 | 2019-09-27 | 天津大学 | Ultrasound image under optical alignment and information fusion method in the art of puncture biopsy needle |
| CN111150461A (en) * | 2020-02-04 | 2020-05-15 | 赵天力 | Puncture needle positioning system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111026269A (en) | 2020-04-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111026269B (en) | Haptic feedback method, device and equipment for biological tissue structure based on force feedback | |
| US10417936B2 (en) | Hybrid physical-virtual reality simulation for clinical training capable of providing feedback to a physical anatomic model | |
| Basdogan et al. | VR-based simulators for training in minimally invasive surgery | |
| JP6049788B2 (en) | Virtual tool operation system | |
| CN104685551B (en) | Mixed reality emulation mode and system | |
| US11373553B2 (en) | Dynamic haptic robotic trainer | |
| CN105825752A (en) | Force feedback device-based virtual corneal surgery training system | |
| CN110807968A (en) | Puncture operation teaching system, realization method, teaching terminal and teaching equipment | |
| CN105096670B (en) | A kind of intelligent immersion tutoring system and device for nose catheter operation real training | |
| WO2012123943A1 (en) | Training, skill assessment and monitoring users in ultrasound guided procedures | |
| MX2013012830A (en) | System and method for performing a hybrid simulation of a medical procedure. | |
| Gopher | Skill training in multimodal virtual environments | |
| KR100551201B1 (en) | Virtual dental training and evaluation system using haptic interface based volumetric model | |
| Cakmak et al. | VS One, a virtual reality simulator for laparoscopic surgery | |
| Lamata et al. | SINERGIA laparoscopic virtual reality simulator: Didactic design and technical development | |
| Kwon et al. | Realistic force reflection in a spine biopsy simulator | |
| Tai et al. | Tissue and force modelling on multi-layered needle puncture for percutaneous surgery training | |
| Coles et al. | Haptic palpation for the femoral pulse in virtual interventional radiology | |
| Sclaverano et al. | BiopSym: a simulator for enhanced learning of ultrasound-guided prostate biopsy | |
| Tai et al. | A novel framework for visuo-haptic percutaneous therapy simulation based on patient-specific clinical trials | |
| Hamza-Lup et al. | Haptic simulator for liver diagnostics through palpation | |
| Crossan et al. | Multimodal feedback cues to aid veterinary training simulations | |
| Vaughan et al. | Haptic feedback from human tissues of various stiffness and homogeneity | |
| Feng et al. | Defining spatial regions in computer-assisted laparoscopic surgical training | |
| Crossan et al. | Multi-session VR Medical Training: The HOPS Simulator |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |