CN111026269A - Haptic feedback method, device and equipment of biological tissue structure based on force feedback - Google Patents
Haptic feedback method, device and equipment of biological tissue structure based on force feedback Download PDFInfo
- Publication number
- CN111026269A CN111026269A CN201911224558.7A CN201911224558A CN111026269A CN 111026269 A CN111026269 A CN 111026269A CN 201911224558 A CN201911224558 A CN 201911224558A CN 111026269 A CN111026269 A CN 111026269A
- Authority
- CN
- China
- Prior art keywords
- force
- force feedback
- tissue structure
- feedback
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B23/00—Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes
- G09B23/28—Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes for medicine
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Business, Economics & Management (AREA)
- Algebra (AREA)
- Computational Mathematics (AREA)
- Medical Informatics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Health & Medical Sciences (AREA)
- Pure & Applied Mathematics (AREA)
- Medicinal Chemistry (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- General Health & Medical Sciences (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Chemical & Material Sciences (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- User Interface Of Digital Computer (AREA)
- Instructional Devices (AREA)
Abstract
The application relates to a method, a device and equipment for tactile feedback of a biological tissue structure based on force feedback. Wherein the method comprises the following steps: acquiring a corresponding real-time position of a stress point of the force feedback equipment in a preset organizational structure information map; the tissue structure information map includes area information corresponding to each tissue structure of the living being; determining a current organizational structure corresponding to the real-time position; calculating the feedback force of the stress point of the force feedback equipment on the current tissue structure based on the preset tissue structure parameter information; transmitting the feedback force to the force feedback device to drive the force feedback device to make a haptic force feedback response. So set up, in virtual puncture operation teaching training, can make the person of being trained can feel real, exquisite sense of touch feedback power in the interaction, can improve the training level of puncture operation greatly, solve the difficult problem that the real behaviour sensation was cultivateed in the training of virtual puncture operation.
Description
Technical Field
The application relates to the field of medical teaching simulation training, in particular to a force feedback-based biological tissue structure tactile feedback method, device and equipment.
Background
Paracentesis is one of the commonly used clinical surgical procedures and important drug administration and treatment methods, and plays a decisive important role in diagnosis, treatment and rescue of critical cases.
In traditional surgical training, a trainee usually exercises a biological target such as an artificial model, a corpse or an animal after demonstration by a highly experienced doctor. However, artificial models are not realistic enough, cadaver resources are limited and expensive, and living animal tissues and structures are greatly different from those of human bodies. Therefore, it is difficult for beginners to grasp the strength of the operation and teachers to accurately transfer their experiences to students.
On the basis, a virtual surgery training scheme also exists, namely, the basic surgery skill of a doctor is helped to be trained by methods of constructing a three-dimensional virtual scene of a surgery, three-dimensional virtual patients, simulating basic procedures of the surgery and the like. However, the interactive feeling of the virtual operation training is weaker, direct feedback in touch is lacked, the virtual operation training is more suitable for the training of the operation process, and the feeling culture of the operation practice is poorer.
That is, the training method of the conventional virtual puncture surgery is difficult to cultivate the feeling of real operation.
Disclosure of Invention
The application provides a force feedback-based tactile feedback method, device and equipment for a biological tissue structure, and aims to solve the problem that the existing virtual puncture surgery training method is difficult to cultivate real operation feeling.
The above object of the present application is achieved by the following technical solutions:
in a first aspect, an embodiment of the present application provides a method for tactile feedback of a biological tissue structure based on force feedback, including:
acquiring a corresponding real-time position of a stress point of the force feedback equipment in a preset organizational structure information map; the tissue structure information map contains region information corresponding to each tissue structure of a living being;
determining a current tissue structure corresponding to the real-time position;
calculating the feedback force of the stress point of the force feedback equipment on the current tissue structure based on the preset tissue structure parameter information;
transmitting the feedback force to a force feedback device to drive the force feedback device to make a haptic force feedback response.
Optionally, the obtaining of the corresponding real-time position of the stress point of the force feedback device in the preset tissue structure information map includes:
acquiring the position of the force feedback equipment in a 3D coordinate system of a working space;
mapping the working space and the organizational structure information of the force feedback equipment to form coordinate mapping, so that the position under a 3D coordinate system corresponds to the position under a 2D coordinate system;
and acquiring the real-time position of the stress point of the force feedback equipment in the 2D coordinate system of the tissue structure information map by using a ray collision detection technology.
Optionally, the creating process of the organization structure information map includes:
carrying out section radiography on a part to be punctured of a living body to obtain a biological tissue structure section diagram;
and dividing the biological tissue structure section map according to the tissue structure to form a corresponding region information map.
Optionally, the method further includes:
each tissue structure is represented by a different color, and the region range of the corresponding tissue structure is represented by a region range of each color.
Optionally, the calculating, based on preset tissue structure parameter information, a feedback force applied to a force point of the force feedback device at a current tissue structure includes:
acquiring the operating force of a user on the force feedback equipment;
calculating the product of the displacement of the stress point of the force feedback equipment in the current organizational structure and a preset resistance coefficient to obtain the resistance of the force feedback equipment;
and calculating the difference between the operating force and the resistance force to obtain the feedback force.
Optionally, the acquiring the operation force of the user on the force feedback device includes:
acquiring real-time data of the force feedback equipment according to a preset first time interval;
decoding the real-time data into data which can be identified in calculation;
sequentially judging whether the decoded real-time data contains noise data;
the noisy point data is discarded and the remaining data is used to determine the operating force.
Optionally, the transmitting the feedback force to a force feedback device to drive the force feedback device to make a haptic force feedback response includes:
the feedback force is encoded into data recognizable by the force feedback device and transmitted to the force feedback device to drive the force feedback device to make a haptic force feedback response.
Optionally, the method further includes:
if no operation data of the force feedback equipment is acquired within a preset second time interval, controlling the force feedback equipment to enter an energy-saving standby state;
and if the temperature of the force feedback equipment is detected to exceed the preset temperature threshold value, controlling the force feedback equipment to reduce the performance or closing the force feedback equipment.
In a second aspect, embodiments of the present application further provide a haptic feedback device for a biological tissue structure based on force feedback, including:
the acquisition module is used for acquiring the corresponding real-time position of the stress point of the force feedback equipment in a preset organizational structure information map; the tissue structure information map contains region information corresponding to each tissue structure of a living being;
the determining module is used for determining the current tissue structure corresponding to the real-time position;
the calculation module is used for calculating the feedback force of the stress point of the force feedback equipment on the current tissue structure based on the preset tissue structure parameter information;
and the transmission module is used for transmitting the feedback force to the force feedback device so as to drive the force feedback device to make a tactile force feedback response.
In a third aspect, embodiments of the present application further provide a haptic feedback device for a biological tissue structure based on force feedback, including:
a memory and a processor coupled to the memory;
the memory is used for storing a program, and the program is at least used for executing the biological tissue structure tactile feedback method based on force feedback;
the processor is used for calling and executing the program stored in the memory.
The technical scheme provided by the embodiment of the application can have the following beneficial effects:
according to the technical scheme provided by the embodiment of the application, the operation force data of the user on the force feedback equipment is obtained, the parameter calculation is carried out according to the preset biological tissue structure parameter information to obtain the feedback force, and the feedback force is transmitted to the force feedback equipment to respond, so that the puncture operation of the user on the real organ can be simulated. By the arrangement, a trainee can feel real and fine tactile feedback force in interaction in the virtual puncture surgery teaching training. The trainee completes a complete operation training interactive operation on the virtual organ model through interacting with the virtual organ model, and as operating a real organ, the training level of the puncture operation can be greatly improved, and the problem of practical operation feeling culture in the virtual puncture operation training is solved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
FIG. 1 is a schematic flow chart illustrating a method for haptic feedback of a biological tissue structure based on force feedback according to an embodiment of the present disclosure;
fig. 2 is a schematic mapping diagram for converting 3D coordinates into 2D coordinates according to an embodiment of the present disclosure;
FIG. 3 is a schematic view of the force applied to the tissue structure in the puncture operation according to the embodiment of the present disclosure;
FIG. 4 is a schematic diagram illustrating a deformation principle of a force feedback model established based on a mass-spring model according to an embodiment of the present application;
FIG. 5 is a graph illustrating the feedback force before a tissue structure is pierced according to an embodiment of the present disclosure;
FIG. 6 is a graph illustrating the feedback force after a tissue structure has been punctured as provided by an embodiment of the present application;
fig. 7 is a feedback force curve graph after interpolation processing according to an embodiment of the present application;
FIG. 8 is a schematic structural diagram of a haptic feedback device for force feedback based biological tissue structure according to an embodiment of the present application;
fig. 9 is a schematic structural diagram of a haptic feedback device for a force feedback-based biological tissue structure according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
Currently, in the puncture surgery training, the following two methods are mainly used:
scheme of artificial model: the scheme mainly simulates the structural characteristics of a human body by adopting an artificial model which is similar to the operation requirement. The artificial model has strong tactile feedback due to the adoption of deformable materials such as silica gel and the like, but the model can be aged and damaged along with repeated puncture or cutting, and the repeated utilization rate is low. In addition, the scheme can not train the whole operation process, lacks statistics and evaluation of operation, can not be separated from guidance and supervision of teachers, and does not meet the requirements of medical teaching.
Virtual surgery scheme: the virtual operation system can help train the basic operation skill of a doctor by constructing an operation three-dimensional virtual scene, a three-dimensional virtual patient, simulating an operation basic process and the like. Different from the traditional training mode, the virtual surgery system can be used for repeatedly practicing for multiple times without worrying about the problems of specimen, field, surgery safety and the like, so that the training cost is lower, and the training period is shortened. But the interactive feeling of the operation is weaker, direct feedback on the touch sense is lacked, the operation flow training is more suitable, and the feeling culture on the operation practice is poorer.
In order to solve the above problem, embodiments of the present application provide the following technical solutions.
Examples
Referring to fig. 1, fig. 1 is a flow chart illustrating a method for tactile feedback of a biological tissue structure based on force feedback according to an embodiment of the present application, wherein the method is generally executed by computer software. As shown in fig. 1, the method comprises the steps of:
s101: acquiring a corresponding real-time position of a stress point of the force feedback equipment in a preset organizational structure information map; the tissue structure information map contains region information corresponding to each tissue structure of a living being;
specifically, the force feedback device is a virtual 3D environment and an interactive platform, and is mainly applied to virtual surgery, games, and the like. The present embodiment can use openhaptics model 3.4 product from 3dsystems as the application infrastructure for haptic feedback. Of course, other similar devices may be used, and are not limited thereto. In addition, for the simulated puncture operation, the stress point of the force feedback device corresponds to the needle head of the puncture needle of the surgical instrument.
The tissue structure of a living being refers to the subcutaneous tissue structure of the living being, such as the fat layer, the cartilage layer, and the like.
In some embodiments, the creation of the organization structure information map comprises:
carrying out section radiography on a part to be punctured of a living body to obtain a biological tissue structure section diagram; and dividing the biological tissue structure section map according to the tissue structure to form a corresponding region information map.
The sectional view of the biological tissue structure can be directly obtained from medical standard sectional image data, or can be made according to the related method. Further, for better viewing, different tissue structures may be represented in different colors, so that the region range of the corresponding tissue structure is represented by the region range of each color.
In some embodiments, the actual process of obtaining the corresponding real-time position of the force point of the force feedback device in the preset tissue structure information map includes: acquiring the position of the force feedback equipment in a 3D coordinate system of a working space; mapping the working space and the organizational structure information of the force feedback equipment to form coordinate mapping, so that the position under a 3D coordinate system corresponds to the position under a 2D coordinate system; and acquiring the real-time position of the stress point of the force feedback equipment in the 2D coordinate system of the tissue structure information map by using a ray collision detection technology.
In the coordinate mapping, a plane in which the puncture direction of the puncture needle is located is generally associated with the tissue structure information map. Ray collision detection technology, which is a collision detection technology in a Unity 3D engine, can be used to detect whether an object (collision volume) makes contact with other objects (i.e., generates a collision). In this embodiment, the technique may be adopted to obtain the position of the needle in the position corresponding to the tissue structure information map by taking the puncture direction of the puncture needle as a ray. The result is shown in fig. 2, where the force feedback device is on the left side of the arrow (top view) and the tissue structure information map is on the right side.
S102: determining a current tissue structure corresponding to the real-time position;
specifically, on the basis of the above steps, if different tissue structures are represented by different colors, the corresponding tissue structures can be determined directly by acquiring color parameters.
S103: calculating the feedback force of the stress point of the force feedback equipment on the current tissue structure based on the preset tissue structure parameter information;
in this embodiment, the process of calculating the feedback force includes: acquiring the operating force of a user on the force feedback equipment; namely the operating force applied by the user holding the operating lever of the force feedback device; calculating the product of the displacement of the stress point of the force feedback equipment in the current organizational structure and a preset resistance coefficient to obtain the resistance of the force feedback equipment; and calculating the difference between the operating force and the resistance force to obtain the feedback force. The resistance coefficient is determined by preset tissue structure parameter information. Further, the process of acquiring the operation force of the user on the force feedback device includes: acquiring real-time data of the force feedback equipment according to a preset first time interval; decoding the real-time data into data which can be identified in calculation; sequentially judging whether the decoded real-time data contains noise data; the noisy point data is discarded and the remaining data is used to determine the operating force.
The first time interval may be 10 ms to 120 ms, and the larger the time interval is, the higher the device performance is, but the lower the sampling precision is, and 60 ms is generally taken as a preferable scheme. The noise data refers to sampling data interfered by other external signals, and when the noise data is judged, the current data can be compared with the previously acquired data, and if the current data does not accord with the change rule, the noise data is the noise data and needs to be discarded. In addition, it should be noted that the form of the data signal that can be recognized by the force feedback device (hardware) and the calculation software is different. Therefore, after the operation force of the user on the force feedback device is acquired and before the feedback force is transmitted to the force feedback device, the data needs to be decoded into software or encoded into data that can be recognized by hardware. Wherein the decoded and encoded protocol parameters are determined by the selected force feedback device.
Before calculating the feedback force, the stress of the tissue structure during the puncture operation is first explained, as shown in fig. 3, when the surgical instrument enters the tissue structure layer of the living body with a certain operation force (F), because the surface has a certain toughness, the tissue structure layer will not be punctured immediately, and there is a squeezing process in the middle, during which the operator will feel a stronger surface elasticity (F)1) When the operating force is larger than the maximum surface elastic force when the surgical instrument is continuously applied, the puncture is completed, and an inertia (F) is generated along with the puncture2) The process, since each layer of the tissue structure has its own density, causes resistance to the penetration process (F)3) Inertial force (F)3) And resistance (F)2) Counteract each other and finally penetrate into the layer of tissue.
Formulating the above calculationThe process of the feedback force (F) is as follows: f ═ F- (F)1+F2+F3)
In order to more intuitively explain the above process, a description will be given below by a specific example.
In some embodiments, as shown in the following table, the preset tissue structure parameter information may include, for each tissue structure: the bending strength coefficient _ Force, the stress area coefficient _ gauge, the maximum restraining Force _ MaxF, the toughness coefficient _ Fade, the inertia coefficient _ Tension, the Density coefficient _ Density, the resistance _ Damp and the level identification Color value _ Color.
On the basis, with reference to fig. 3, the calculation formula of each parameter in the above formula is:
f＝_Force
F1＝L1*_Fade
F2＝(1-L2)*_Tension
F3＝L3*_Density
wherein L is1、L2And L3And forming a stress area coefficient _ Tough.
Based on this, the feedback force F is calculated by
F＝f-(F1+F2+F3)＝_Force-[L1*_Fade+(1-L2)*_Tension+L3*_Density]
For surface elasticity F1From a biomechanical point of view, the subcutaneous tissue structure can be generally regarded as approximately isotropic (isotropic refers to the property that the physical and chemical properties of an object do not change with different directions, i.e. the measured performance values of an object in different directions are identical, which is also called homogeneity). Modeling of these tissue structures allows for the creation of a linear elastic model of the soft tissue under this approximation. Suppose a particle-spring-damper system (the particle-spring-damper system can be represented by a simple mathematical model, namely a numerical axis modeling method, and established and connected withAnd marking the actual displacement value of the spring or the damper as a rational number on the corresponding position of the numerical axis of the system in the parallel direction, and then determining the magnitude and the direction of the spring force or the damping force borne by the corresponding mass point according to the result of the larger and smaller rational numbers so as to obtain a differential equation of the system. ) The mass point is represented by nodes in a geometric model, and the influence relationship is established between the nodes m through virtual springs. The motion of each particle satisfies the lagrange equation of motion. The particle-spring-damper model is an expansion of a viscoelasticity viogt model, and can better describe the characteristics of deformation and the like of a subcutaneous tissue structure model. The principle of deformation of the force feedback model based on the particle-spring model is shown in fig. 4.
The intersecting contact point of the soft tissue surface can be obtained by obtaining a vector formed by the original position and the new position of the vertex of the virtual instrument model, and the stretched length x of the spring can also be obtained, wherein the mathematical description of the surface elasticity calculation according to the model is as follows:
wherein M is the mass of the tissue surface, D is the damping (resistance) coefficient, K is the elastic coefficient of the spring, x is the retraction displacement (needle insertion depth) of the spring, and F is the retraction displacement (needle insertion depth) of the spring1(t) is the surface elasticity at a certain moment.
It should be noted that, the mathematical description of the surface elasticity in the above model is used for illustrating the principle, and when the method of the present embodiment is applied, a toughness coefficient _ Fade may be preset for the tissue structure layer based on the principle, so that the toughness coefficient _ Fade and the displacement (needle insertion depth) L of the puncture needle may be calculated1To calculate the surface spring force F1. Before lancing, L2And L3Are all 0, L1Gradually increase, therefore, F1And gradually increases. The feedback force versus time in this process is shown in fig. 5.
Further, in order to simulate the feeling of skin puncture, a maximum restraining force (_ MaxF) is added to the feedback force of each layer, and when the maximum restraining force is exceeded, the force-removing process is started. The feedback force versus time after the addition of the maximum restraining force is shown in fig. 6.
Wherein the feedback force F satisfies: f ═ MaxF (t)1-t2)*s
In the formula, t2To start time, t1And s is a breakthrough scaling coefficient for the current time and is used for adjusting the strength of the breakthrough force.
Further, in order to prevent the force from being jittered, it is necessary to linearly interpolate the calculated feedback force so as to have a linear characteristic. Interpolation is an important method for approximation of a discrete function, and can be used for estimating the approximate value of the function at other points through the value conditions of the function at a limited number of points. The interpolation processing process can be realized by a mathf.
Mathf.lerp(min,max,F)＝newF
In the formula, min is the minimum value of the feedback force, max is the maximum value of the feedback force, F is the current feedback force, and newF is the final output result.
After the interpolation process, a graph is obtained as shown in fig. 7, which represents the feedback force, i.e., the calculation result that is to be finally transmitted to the force feedback device.
S104: transmitting the feedback force to a force feedback device to drive the force feedback device to make a haptic force feedback response.
As described above, this step includes: and combining the calculated feedback forces in the first time interval, performing interpolation processing to obtain a continuous and smooth feedback force curve, encoding the continuous and smooth feedback force curve into data which can be identified by the force feedback equipment, and finally transmitting the data to the force feedback equipment so as to drive the force feedback equipment to make a tactile force feedback response.
According to the technical scheme provided by the embodiment of the application, the operation force data of the user on the force feedback equipment is obtained, the parameter calculation is carried out according to the preset biological tissue structure parameter information to obtain the feedback force, and the feedback force is transmitted to the force feedback equipment to respond, so that the puncture operation of the user on the real organ can be simulated. By the arrangement, a trainee can feel real and fine tactile feedback force in interaction in the virtual puncture surgery teaching training. The trainee completes a complete operation training interactive operation on the virtual organ model through interacting with the virtual organ model, and as operating a real organ, the training level of the puncture operation can be greatly improved, and the problem of practical operation feeling culture in the virtual puncture operation training is solved.
In addition, the computer software is also used for detecting and protecting the state of the force feedback device, such as idle standby, overheating protection and the like. Namely: if no operation data of the force feedback equipment is acquired within a preset second time interval, controlling the force feedback equipment to enter an energy-saving standby state; and if the temperature of the force feedback equipment is detected to exceed the preset temperature threshold value, controlling the force feedback equipment to reduce the performance or closing the force feedback equipment.
The second time interval may be set according to actual needs, for example, 30 seconds, that is, if the user does not operate the force feedback device within 30 seconds, the force feedback device is enabled to enter the energy-saving standby state, so as to reduce energy consumption of the force feedback device. And, the force feedback device is protected when it overheats.
In order to more fully describe the technical solution of the present application, embodiments of the present application also provide a haptic feedback device for a force feedback-based biological tissue structure, which corresponds to the haptic feedback method for a force feedback-based biological tissue structure provided in the above embodiments of the present application.
Referring to fig. 8, fig. 8 is a schematic structural diagram of a tactile feedback device for a force feedback-based biological tissue structure according to an embodiment of the present disclosure. As shown in fig. 8, the apparatus includes:
an obtaining module 81, configured to obtain a real-time position of a stress point of the force feedback device in a preset tissue structure information map; the tissue structure information map contains region information corresponding to each tissue structure of a living being;
a determining module 82, configured to determine a current tissue structure corresponding to the real-time location;
the calculating module 83 is configured to calculate a feedback force applied to the stress point of the force feedback device on the current tissue structure based on preset tissue structure parameter information;
a transmission module 84 for transmitting the feedback force to the force feedback device to drive the force feedback device to make a haptic force feedback response.
Specifically, the specific implementation manner of the functions of each module can be realized by referring to the content in the tactile feedback method of the biological tissue structure based on force feedback, and is not described in detail herein.
For more complete description of the technical solution of the present application, embodiments of the present application also provide a haptic feedback apparatus for a force feedback-based biological tissue structure, corresponding to the haptic feedback method for a force feedback-based biological tissue structure provided in the above embodiments of the present application.
Referring to fig. 9, fig. 9 is a schematic structural diagram of a haptic feedback device for a force feedback-based biological tissue structure according to an embodiment of the present application. As shown in fig. 9, the apparatus includes:
a memory 91 and a processor 92 connected to the memory 91;
the memory 91 is used for storing a program for executing at least the above-described force-feedback-based tactile feedback method for a biological tissue structure;
the processor 92 is used to call and execute the program stored in the memory 91.
Specifically, the device is generally a computer, and the functions of the program are implemented by referring to the relevant contents in the haptic feedback method based on the force feedback for the biological tissue structure, which is not described in detail herein.
It is understood that the same or similar parts in the above embodiments may be mutually referred to, and the same or similar parts in other embodiments may be referred to for the content which is not described in detail in some embodiments.
It should be noted that, in the description of the present application, the terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Further, in the description of the present application, the meaning of "a plurality" means at least two unless otherwise specified.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and the scope of the preferred embodiments of the present application includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.
Claims (10)
1. A method for haptic feedback of a biological tissue structure based on force feedback, comprising:
acquiring a corresponding real-time position of a stress point of the force feedback equipment in a preset organizational structure information map; the tissue structure information map contains region information corresponding to each tissue structure of a living being;
determining a current tissue structure corresponding to the real-time position;
calculating the feedback force of the stress point of the force feedback equipment on the current tissue structure based on the preset tissue structure parameter information;
transmitting the feedback force to a force feedback device to drive the force feedback device to make a haptic force feedback response.
2. The method according to claim 1, wherein the obtaining of the corresponding real-time position of the force point of the force feedback device in the preset tissue structure information map comprises:
acquiring the position of the force feedback equipment in a 3D coordinate system of a working space;
mapping the working space and the organizational structure information of the force feedback equipment to form coordinate mapping, so that the position under a 3D coordinate system corresponds to the position under a 2D coordinate system;
and acquiring the real-time position of the stress point of the force feedback equipment in the 2D coordinate system of the tissue structure information map by using a ray collision detection technology.
3. The method of claim 1, wherein the creation of the organization structure information map comprises:
carrying out section radiography on a part to be punctured of a living body to obtain a biological tissue structure section diagram;
and dividing the biological tissue structure section map according to the tissue structure to form a corresponding region information map.
4. The method of claim 3, further comprising:
each tissue structure is represented by a different color, and the region range of the corresponding tissue structure is represented by a region range of each color.
5. The method according to claim 1, wherein calculating the feedback force of the force point of the force feedback device on the current tissue structure based on the preset tissue structure parameter information comprises:
acquiring the operating force of a user on the force feedback equipment;
calculating the product of the displacement of the stress point of the force feedback equipment in the current organizational structure and a preset resistance coefficient to obtain the resistance of the force feedback equipment;
and calculating the difference between the operating force and the resistance force to obtain the feedback force.
6. The method of claim 5, wherein the obtaining of the user's operating force on the force feedback device comprises:
acquiring real-time data of the force feedback equipment according to a preset first time interval;
decoding the real-time data into data which can be identified in calculation;
sequentially judging whether the decoded real-time data contains noise data;
the noisy point data is discarded and the remaining data is used to determine the operating force.
7. The method of claim 6, wherein said transmitting the feedback force to a force feedback device to drive the force feedback device in a haptic force feedback response comprises:
the feedback force is encoded into data recognizable by the force feedback device and transmitted to the force feedback device to drive the force feedback device to make a haptic force feedback response.
8. The method of any one of claims 1-7, further comprising:
if no operation data of the force feedback equipment is acquired within a preset second time interval, controlling the force feedback equipment to enter an energy-saving standby state;
and if the temperature of the force feedback equipment is detected to exceed the preset temperature threshold value, controlling the force feedback equipment to reduce the performance or closing the force feedback equipment.
9. A force feedback based haptic feedback device for a biological tissue structure, comprising:
the acquisition module is used for acquiring the corresponding real-time position of the stress point of the force feedback equipment in a preset organizational structure information map; the tissue structure information map contains region information corresponding to each tissue structure of a living being;
the determining module is used for determining the current tissue structure corresponding to the real-time position;
the calculation module is used for calculating the feedback force of the stress point of the force feedback equipment on the current tissue structure based on the preset tissue structure parameter information;
and the transmission module is used for transmitting the feedback force to the force feedback device so as to drive the force feedback device to make a tactile force feedback response.
10. A haptic feedback device for force feedback based biological tissue structure, comprising:
a memory and a processor coupled to the memory;
the memory for storing a program for at least performing the force feedback based haptic feedback method of a biological tissue structure according to any one of claims 1 to 8;
the processor is used for calling and executing the program stored in the memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911224558.7A CN111026269B (en) | 2019-12-04 | 2019-12-04 | Haptic feedback method, device and equipment for biological tissue structure based on force feedback |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911224558.7A CN111026269B (en) | 2019-12-04 | 2019-12-04 | Haptic feedback method, device and equipment for biological tissue structure based on force feedback |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111026269A true CN111026269A (en) | 2020-04-17 |
| CN111026269B CN111026269B (en) | 2023-10-10 |
Family
ID=70204181
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911224558.7A Active CN111026269B (en) | 2019-12-04 | 2019-12-04 | Haptic feedback method, device and equipment for biological tissue structure based on force feedback |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111026269B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111782066A (en) * | 2020-06-30 | 2020-10-16 | 联想(北京)有限公司 | Electronic equipment |
| CN112422829A (en) * | 2020-11-19 | 2021-02-26 | 北京字节跳动网络技术有限公司 | Method, device, terminal and storage medium for assisting in shooting image |
| CN112690757A (en) * | 2020-12-17 | 2021-04-23 | 四川大学华西医院 | Pressure damage monitoring and feedback equipment and feedback regulation system |
| CN114533138A (en) * | 2021-05-31 | 2022-05-27 | 四川大学 | Automatic puncture mechanism integrating touch sensing |
Citations (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101400306A (en) * | 2006-03-09 | 2009-04-01 | 画像诊断株式会社 | Medical 3-dimensional image display control program and medical 3-dimensional image display method |
| CN102207997A (en) * | 2011-06-07 | 2011-10-05 | 哈尔滨工业大学 | Force-feedback-based robot micro-wound operation simulating system |
| CN102254476A (en) * | 2011-07-18 | 2011-11-23 | 广州赛宝联睿信息科技有限公司 | Endoscopic minimally invasive surgery simulation training method and system |
| CN103269451A (en) * | 2013-04-27 | 2013-08-28 | 沈阳理工大学 | Automatic shutdown method using video monitoring |
| CN103400023A (en) * | 2013-06-28 | 2013-11-20 | 华北水利水电大学 | Soft tissue deformation simulation method |
| CN104272365A (en) * | 2012-04-13 | 2015-01-07 | 汤姆逊许可公司 | Method to render global 6 DoF motion effect with multiple local force-feedback |
| CN104537938A (en) * | 2014-12-21 | 2015-04-22 | 合肥德铭电子有限公司 | Endoscope simulative training system capable of achieving flexible force feedback |
| CN105069301A (en) * | 2015-08-14 | 2015-11-18 | 南通大学 | Lumbar puncture virtual simulation and training system supporting haptic interaction |
| CN105303605A (en) * | 2015-10-26 | 2016-02-03 | 哈尔滨理工大学 | Orthopedic surgery operation simulation system on the basis of force feedback |
| CN105446362A (en) * | 2015-12-07 | 2016-03-30 | 陆宁远 | Posture detection adjusting device and method based on assistance of computer science |
| CN105559887A (en) * | 2015-12-11 | 2016-05-11 | 哈尔滨工业大学 | Operation cutting training system and method based on force feedback and used for surgical robot |
| CN105913718A (en) * | 2016-07-08 | 2016-08-31 | 哈尔滨理工大学 | Thread lift plastic surgery simulation system |
| CN105931517A (en) * | 2016-06-17 | 2016-09-07 | 重庆金山科技（集团）有限公司 | Surgical operation virtual simulation method and system |
| CN106781941A (en) * | 2016-11-24 | 2017-05-31 | 北京理工大学 | A kind of method and its system for simulating microtrauma puncture operation |
| CN106725762A (en) * | 2016-12-30 | 2017-05-31 | 西安中科微光影像技术有限公司 | A kind of automatic puncturing method |
| CN107820715A (en) * | 2017-08-09 | 2018-03-20 | 北京小米移动软件有限公司 | Method for excessive heating protection, device, user equipment and the base station of user equipment |
| CN107992010A (en) * | 2017-12-14 | 2018-05-04 | 万泰机电工业(昆山)有限公司 | New-energy automobile part produces tracing information generation method and traceability system |
| CN108428142A (en) * | 2018-05-18 | 2018-08-21 | 广州质量云信息科技有限公司 | Quick Response Code traceability management system |
| CN108648548A (en) * | 2018-04-19 | 2018-10-12 | 浙江工业大学 | A kind of neuro-surgery virtual operation training system |
| CN109308739A (en) * | 2018-10-11 | 2019-02-05 | 南京工程学院 | A kind of soft tissue Minimally Invasive Surgery training method based on virtual reality |
| CN109756791A (en) * | 2017-11-07 | 2019-05-14 | 中兴通讯股份有限公司 | A kind of network equipment guard method, protection equipment and computer readable storage medium |
| CN109934479A (en) * | 2019-03-06 | 2019-06-25 | 南昌易美光电科技有限公司 | The management system and method for production material |
| CN110046406A (en) * | 2019-03-28 | 2019-07-23 | 天津大学 | A kind of soft tissue emulation mode with force feedback structure in anatomic teaching system |
| CN110279467A (en) * | 2019-06-19 | 2019-09-27 | 天津大学 | Ultrasound image under optical alignment and information fusion method in the art of puncture biopsy needle |
| CN111150461A (en) * | 2020-02-04 | 2020-05-15 | 赵天力 | Puncture needle positioning system and method |
-
2019
- 2019-12-04 CN CN201911224558.7A patent/CN111026269B/en active Active
Patent Citations (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101400306A (en) * | 2006-03-09 | 2009-04-01 | 画像诊断株式会社 | Medical 3-dimensional image display control program and medical 3-dimensional image display method |
| CN102207997A (en) * | 2011-06-07 | 2011-10-05 | 哈尔滨工业大学 | Force-feedback-based robot micro-wound operation simulating system |
| CN102254476A (en) * | 2011-07-18 | 2011-11-23 | 广州赛宝联睿信息科技有限公司 | Endoscopic minimally invasive surgery simulation training method and system |
| CN104272365A (en) * | 2012-04-13 | 2015-01-07 | 汤姆逊许可公司 | Method to render global 6 DoF motion effect with multiple local force-feedback |
| CN103269451A (en) * | 2013-04-27 | 2013-08-28 | 沈阳理工大学 | Automatic shutdown method using video monitoring |
| CN103400023A (en) * | 2013-06-28 | 2013-11-20 | 华北水利水电大学 | Soft tissue deformation simulation method |
| CN104537938A (en) * | 2014-12-21 | 2015-04-22 | 合肥德铭电子有限公司 | Endoscope simulative training system capable of achieving flexible force feedback |
| CN105069301A (en) * | 2015-08-14 | 2015-11-18 | 南通大学 | Lumbar puncture virtual simulation and training system supporting haptic interaction |
| CN105303605A (en) * | 2015-10-26 | 2016-02-03 | 哈尔滨理工大学 | Orthopedic surgery operation simulation system on the basis of force feedback |
| CN105446362A (en) * | 2015-12-07 | 2016-03-30 | 陆宁远 | Posture detection adjusting device and method based on assistance of computer science |
| CN105559887A (en) * | 2015-12-11 | 2016-05-11 | 哈尔滨工业大学 | Operation cutting training system and method based on force feedback and used for surgical robot |
| CN105931517A (en) * | 2016-06-17 | 2016-09-07 | 重庆金山科技（集团）有限公司 | Surgical operation virtual simulation method and system |
| CN105913718A (en) * | 2016-07-08 | 2016-08-31 | 哈尔滨理工大学 | Thread lift plastic surgery simulation system |
| CN106781941A (en) * | 2016-11-24 | 2017-05-31 | 北京理工大学 | A kind of method and its system for simulating microtrauma puncture operation |
| CN106725762A (en) * | 2016-12-30 | 2017-05-31 | 西安中科微光影像技术有限公司 | A kind of automatic puncturing method |
| CN107820715A (en) * | 2017-08-09 | 2018-03-20 | 北京小米移动软件有限公司 | Method for excessive heating protection, device, user equipment and the base station of user equipment |
| CN109756791A (en) * | 2017-11-07 | 2019-05-14 | 中兴通讯股份有限公司 | A kind of network equipment guard method, protection equipment and computer readable storage medium |
| CN107992010A (en) * | 2017-12-14 | 2018-05-04 | 万泰机电工业(昆山)有限公司 | New-energy automobile part produces tracing information generation method and traceability system |
| CN108648548A (en) * | 2018-04-19 | 2018-10-12 | 浙江工业大学 | A kind of neuro-surgery virtual operation training system |
| CN108428142A (en) * | 2018-05-18 | 2018-08-21 | 广州质量云信息科技有限公司 | Quick Response Code traceability management system |
| CN109308739A (en) * | 2018-10-11 | 2019-02-05 | 南京工程学院 | A kind of soft tissue Minimally Invasive Surgery training method based on virtual reality |
| CN109934479A (en) * | 2019-03-06 | 2019-06-25 | 南昌易美光电科技有限公司 | The management system and method for production material |
| CN110046406A (en) * | 2019-03-28 | 2019-07-23 | 天津大学 | A kind of soft tissue emulation mode with force feedback structure in anatomic teaching system |
| CN110279467A (en) * | 2019-06-19 | 2019-09-27 | 天津大学 | Ultrasound image under optical alignment and information fusion method in the art of puncture biopsy needle |
| CN111150461A (en) * | 2020-02-04 | 2020-05-15 | 赵天力 | Puncture needle positioning system and method |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111782066A (en) * | 2020-06-30 | 2020-10-16 | 联想(北京)有限公司 | Electronic equipment |
| CN112422829A (en) * | 2020-11-19 | 2021-02-26 | 北京字节跳动网络技术有限公司 | Method, device, terminal and storage medium for assisting in shooting image |
| CN112422829B (en) * | 2020-11-19 | 2022-04-26 | 北京字节跳动网络技术有限公司 | Method, device, terminal and storage medium for assisting in shooting image |
| CN112690757A (en) * | 2020-12-17 | 2021-04-23 | 四川大学华西医院 | Pressure damage monitoring and feedback equipment and feedback regulation system |
| CN114533138A (en) * | 2021-05-31 | 2022-05-27 | 四川大学 | Automatic puncture mechanism integrating touch sensing |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111026269B (en) | 2023-10-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111026269B (en) | Haptic feedback method, device and equipment for biological tissue structure based on force feedback | |
| US10417936B2 (en) | Hybrid physical-virtual reality simulation for clinical training capable of providing feedback to a physical anatomic model | |
| Basdogan et al. | VR-based simulators for training in minimally invasive surgery | |
| CN104685551B (en) | Mixed reality emulation mode and system | |
| JP6049788B2 (en) | Virtual tool operation system | |
| US7241145B2 (en) | Birth simulator | |
| US11373553B2 (en) | Dynamic haptic robotic trainer | |
| CN110807968A (en) | Puncture operation teaching system, realization method, teaching terminal and teaching equipment | |
| CN105825752A (en) | Force feedback device-based virtual corneal surgery training system | |
| CN110400620B (en) | Heart three-dimensional model construction method and simulated heart operation guidance system | |
| JP4129527B2 (en) | Virtual surgery simulation system | |
| KR100551201B1 (en) | Virtual dental training and evaluation system using haptic interface based volumetric model | |
| Cakmak et al. | VS One, a virtual reality simulator for laparoscopic surgery | |
| Kwon et al. | Realistic force reflection in a spine biopsy simulator | |
| RU2687564C1 (en) | System for training and evaluating medical personnel performing injection and surgical minimally invasive procedures | |
| US20070105083A1 (en) | Rigid birth simulator having an interactive optical display | |
| Coles et al. | Haptic palpation for the femoral pulse in virtual interventional radiology | |
| Chen et al. | Dynamic touch‐enabled virtual palpation | |
| JPH11219100A (en) | Medical simulator system | |
| Székely | Surgical simulators | |
| Vaughan et al. | Haptic feedback from human tissues of various stiffness and homogeneity | |
| Kim et al. | Palpation simulator for laparoscopic surgery with haptic feedback | |
| McColl et al. | Haptic rendering & perception studies for laparoscopic surgery simulation | |
| CN115662234B (en) | Thoracic surgery teaching system based on virtual reality | |
| Suzuki et al. | Performing virtual surgery with a force feedback system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |