CN102207997A - Force-feedback-based robot micro-wound operation simulating system - Google Patents
Force-feedback-based robot micro-wound operation simulating system Download PDFInfo
- Publication number
- CN102207997A CN102207997A CN2011101506784A CN201110150678A CN102207997A CN 102207997 A CN102207997 A CN 102207997A CN 2011101506784 A CN2011101506784 A CN 2011101506784A CN 201110150678 A CN201110150678 A CN 201110150678A CN 102207997 A CN102207997 A CN 102207997A
- Authority
- CN
- China
- Prior art keywords
- module
- force
- force feedback
- information
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000002324 minimally invasive surgery Methods 0.000 claims abstract description 47
- 238000009877 rendering Methods 0.000 claims abstract description 34
- 230000033001 locomotion Effects 0.000 claims abstract description 25
- 238000012545 processing Methods 0.000 claims abstract description 22
- 238000004088 simulation Methods 0.000 claims abstract description 18
- 239000002245 particle Substances 0.000 claims description 34
- 238000006073 displacement reaction Methods 0.000 claims description 22
- 230000008859 change Effects 0.000 claims description 20
- 238000004364 calculation method Methods 0.000 claims description 15
- 210000000056 organ Anatomy 0.000 claims description 13
- 238000001514 detection method Methods 0.000 claims description 10
- 238000013016 damping Methods 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 230000005484 gravity Effects 0.000 claims description 4
- 238000005286 illumination Methods 0.000 claims description 4
- 239000000463 material Substances 0.000 claims description 3
- 230000001133 acceleration Effects 0.000 claims description 2
- 238000005192 partition Methods 0.000 claims 4
- 238000007781 pre-processing Methods 0.000 claims 4
- 230000003902 lesion Effects 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 6
- 238000011160 research Methods 0.000 abstract description 5
- 230000007812 deficiency Effects 0.000 abstract 1
- 238000002059 diagnostic imaging Methods 0.000 abstract 1
- 238000012549 training Methods 0.000 description 8
- 238000000034 method Methods 0.000 description 6
- 241001465754 Metazoa Species 0.000 description 5
- 210000004872 soft tissue Anatomy 0.000 description 5
- 238000001356 surgical procedure Methods 0.000 description 5
- 238000010276 construction Methods 0.000 description 4
- 201000010099 disease Diseases 0.000 description 4
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 241001269238 Data Species 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000002432 robotic surgery Methods 0.000 description 3
- 210000000683 abdominal cavity Anatomy 0.000 description 2
- 230000009471 action Effects 0.000 description 2
- 210000003484 anatomy Anatomy 0.000 description 2
- 210000004204 blood vessel Anatomy 0.000 description 2
- 230000006378 damage Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 239000012636 effector Substances 0.000 description 2
- 210000000232 gallbladder Anatomy 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000013334 tissue model Methods 0.000 description 2
- 210000001835 viscera Anatomy 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 206010044565 Tremor Diseases 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 206010016256 fatigue Diseases 0.000 description 1
- 210000000245 forearm Anatomy 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000003760 hair shine Effects 0.000 description 1
- 210000000936 intestine Anatomy 0.000 description 1
- 238000012977 invasive surgical procedure Methods 0.000 description 1
- 210000004185 liver Anatomy 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000000704 physical effect Effects 0.000 description 1
- 230000035479 physiological effects, processes and functions Effects 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000009958 sewing Methods 0.000 description 1
- 239000002689 soil Substances 0.000 description 1
- 210000002784 stomach Anatomy 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Landscapes
- Manipulator (AREA)
Abstract
基于力反馈的机器人微创手术仿真系统，涉及一种计算机虚拟仿真系统。本发明针对机器人微创手术的研究现状以及现有仿真技术存在的问题和不足而设计。本发明中的数据库模块用于存储医学影像原始数据、机器人微创手术器械模型数据和运动约束信息；图像处理模块用于将数据库模块中的影像数据转换成三维体数据；物理建模模块用于根据三维体数据构建几何模型；力反馈模块用于根据几何模型以及力反馈感知设备输出的参数计算出反馈力的大小和方向，并输出反馈力数据给力反馈感知设备，进而使操作者通过该力反馈感知设备感受到力；图形渲染模块用于根据力反馈模块发送的碰撞信息和数据库模块所发送的机器人运动信息获得渲染后的图像信息并输出。
A robot minimally invasive surgery simulation system based on force feedback relates to a computer virtual simulation system. The invention is designed aiming at the research status of robot minimally invasive surgery and the existing problems and deficiencies of the existing simulation technology. The database module in the present invention is used to store medical imaging original data, robot minimally invasive surgical instrument model data and motion constraint information; the image processing module is used to convert the image data in the database module into three-dimensional volume data; the physical modeling module is used to Construct a geometric model based on three-dimensional data; the force feedback module is used to calculate the magnitude and direction of the feedback force based on the geometric model and the parameters output by the force feedback sensing device, and output the feedback force data to the force feedback sensing device, so that the operator can pass the force The feedback sensing device senses the force; the graphics rendering module is used to obtain and output the rendered image information according to the collision information sent by the force feedback module and the robot motion information sent by the database module.
Description
Technical field
The present invention relates to a kind of computer virtual simulation system, particularly a kind of robot Minimally Invasive Surgery analogue system based on force feedback.
Technical background
In recent years, minimally-invasive surgery robot systems such as da Vinci are applied to clinical gradually, and it combines the traditional medical apparatus with infotech, Robotics, make surgical diagnosis and treatment reach Wicresoftization, microminiaturization, intellectuality.Micro-wound operation robot is compared with traditional operation has significant advantage: at first robot improves doctor's mode of operation, and the standard operation technique improves the operation quality, Minimally Invasive Surgery is developed and popularizes have important impetus; In addition, robot does not have human sense tired out and physiology restriction, is not subjected to environmental stimuli, and it is very little that work range can design, the precision height of executable operations, flexible operation, extend the functipnal capability of doctor's hand, eye, eliminated the intrinsic of staff and tremble, prolonged the surgical occupation life-span.Robot itself is not afraid of and is subjected to x radiation x.These advantages make robot become doctor's best assistant.
Yet micro-wound operation robot also has its hidden danger that need overcome.Compare with traditional operation, in the robotic surgery, the operator can't the Direct observation operative site, doctor's hand eye coordination difficulty, and the scope of activities of operating theater instruments is also unlike traditional operation, operation need rely on sight glass to judge focus in very little space fully, and the operating theater instruments of operating various complexity excises focus and sewing up a wound, and therefore, degree of accuracy is difficult to improve, the human factor of both hands slight jitter and so on also can't be got rid of, and this also has higher requirement for the doctor.Without training, the surgeon is difficult to competent this novel modus operandi.But traditional training and training except using corpse and animal, are not better trained and simulation means.The utilization animal gives training, and on the one hand, the anatomical structure of animal is different with the anatomical structure of human body, also can be subjected to the condemnation of Animal Protection Association on the other hand, and people's corpse is can not be nonexpondable.
If the characteristics of micro-wound operation robot are combined with the virtual operation technology, develop a cover robot Minimally Invasive Surgery analogue system, then can address the above problem.This technology simulates the human organ model from medical image in computing machine, create virtual medical environment and virtual operation robot, and dummy robot's minimal invasive surgical procedures of assisting.The multi-door subjects such as medical science, computer graphics, biomechanics, Mechanics of Machinery, materialogy that needed integrated application.Increasingly extensive along with the continuous development of micro-wound operation robot technology and clinical practice, the following training that needs a large amount of doctors to accept the micro-wound operation robot surgical technic.Therefore, the system of virtual operation based on micro-wound operation robot has become a novelty and urgent research direction.
At present, similar system is very rare, the work of some this respects has been done by Tokyo Ci Hui meeting medical university and Hong Kong Chinese University based on Leonardo da Vinci's operating robot, obtained certain achievement in research, but the system that develops is still very immature, and domestic operation emulation based on operating robot is not extensively carried out as yet, this respect research both domestic and external in a word also is in the starting stage, often has robot motion analog distortion, Soft Tissue Deformation distortion, lacks problems such as force feedback or force feedback distortion.
Summary of the invention
The present invention is directed to the present Research of robot Minimally Invasive Surgery,, design a kind of robot Minimally Invasive Surgery analogue system that can carry out repeatedly based on force feedback at its problem and shortage that exists at present.
Robot Minimally Invasive Surgery analogue system based on force feedback of the present invention is by the force feedback awareness apparatus, the force feedback module, database module, image processing module, physical modeling module and graph rendering module constitute, force feedback module output control signal is given the force feedback awareness apparatus, described force feedback awareness apparatus power output feeds back signal to the force feedback module, force feedback module output collision alarm is given graph rendering module, image processing module reads image data from database module, described image processing module output three-dimensional reconstruction information is given the physical modeling module, described physical modeling module output physical model information is given the force feedback module, and described force feedback module reads kinematic constraint information from database module; Described pattern rendering module also reads kinematic constraint information from database module;
Described database module is used for storing medical image raw data, robot Minimally Invasive Surgery apparatus model data and kinematic constraint information;
Described image processing module is used for the image data of reading of data library module, and converts the image data that reads to three-dimensional data;
Described physical modeling module is used for the three-dimensional data of reading images processing module, and makes up geometric model according to described three-dimensional data;
Described force feedback module, be used for going out the size and Orientation of feedback force according to the calculation of parameter that the geometric model and the force feedback awareness apparatus of physical modeling module construction are exported, and these feedback forces are outputed to the force feedback awareness apparatus after controlling and compensating, and then make the operator experience power by this force feedback awareness apparatus;
Described graph rendering module, the image information after being used for robot motion's information acquisition that the collision information that sends according to the force feedback module and database module sent and playing up, and realize the output of video information.
Described physical model information comprises particle postition, the elastic coefficient and ratio of damping.
Database module described in the present invention can be made up of image database, robot Minimally Invasive Surgery apparatus model bank and kinematic constraint unit, and wherein: image database is used for the storing medical image raw data; Robot Minimally Invasive Surgery apparatus model bank, the partial model and the various operating theater instruments model data that are used to store micro-wound operation robot; The kinematic constraint unit is used for storing each parts that robot Minimally Invasive Surgery apparatus model bank stores and the kinematical constraint connection relation and the characteristics of motion between the various operating theater instruments.
In the model data in the robot Minimally Invasive Surgery apparatus model bank described in the present invention, given different materials, different diffusing has been set for the model of each moving component.
Image processing module described in the present invention can be made up of image pretreatment unit and three-dimensional reconstruction unit, image pretreatment unit wherein, image data in the reading of data library module, and described image data carried out sending to after the image pre-service three-dimensional reconstruction unit; The three-dimensional reconstruction unit, the data that are used to adopt the MC algorithm to obtain after the image pre-service are carried out three-dimensional reconstruction, obtain three-dimensional data.
Force feedback module described in the present invention can be made up of collision detection unit, particle displacement change calculations unit, feedback force computing unit and power control and compensating unit, collision detection unit wherein, be used to judge whether the end of robot bumps with organ, and the positional information that accurate location model bumps under situation about bumping, and the positional information that will collide sends to graph rendering module and particle displacement change calculations unit simultaneously; Particle displacement change calculations unit, be used for when receiving the collision information that collision detection unit sends, calculate the change in displacement information of the particle of position of collision immediately, and then obtain the deformation data of organ model, and the change in displacement information of described particle is sent to the feedback force computing unit; The feedback force computing unit is used for calculating the information that obtains feedback force according to the particle displacement change information, and this feedback force information is sent to power control and compensating unit; Power control and compensating unit are used for obtaining force compensating information according to force feedback information, send power control information and force compensating information and send to the force feedback awareness apparatus.
Above-mentioned collision information comprises the direction and the position of collision.
Graph rendering module described in the present invention can be made up of image rendering unit and video output unit, and image such as wherein said image rendering unit is used to create viewpoint, illumination is set, data read, form renewal, form destruction is played up operation; Described video output unit is used for converting the image information of image rendering unit output to video information output.
Robot Minimally Invasive Surgery analogue system based on force feedback of the present invention based on a kind of micro-wound operation robot mechanism, and with the medical image data of the real human body construction basis as simulated environment, has following remarkable advantage:
1, broken through the training and the teaching pattern of traditional Minimally Invasive Surgery, avoided using the sample of anyone or animal, and can unlimitedly use repeatedly.
2, analogue system is based on a kind of minimally-invasive surgery robot system, and simulating scenes can be realized the switching between the operative scenario in robot motion's scene and the human body, that is can realize traditional Minimally Invasive Surgery emulation and two kinds of mode of operations of robot Minimally Invasive Surgery emulation simultaneously.The kinematics characteristics of operating robot and mechanism form can change effectively simulates real real machine people's various motions flexibly.
3, the function that has accurate force feedback can be felt during operation and the suitable power of true operation, can strengthen the sense of reality of doctor's operation effectively.
4, utilize advanced medical image technology and computer image processing technology,, utilize correlation technique structure internal organs, focus and whole surgery district physiological environments such as three-dimensional reconstruction based on the medical image of real human body.Realized the digitizing, visual of surgical field of view.
5, can utilize certain disease patient's image data to carry out modeling, design a kind of training and tutoring system at certain disease, help surgeon undergo surgery teaching, surgery planning and preview, training by repeatedly and demonstration make the surgeon be familiar with whole operation details at a kind of robot Minimally Invasive Surgery or the common Minimally Invasive Surgery of certain disease.
6, the soft tissue organs Visualization Model has the physical characteristics of tissue, can realize viscoelastic deformation true to nature when operating theater instruments interacts with it.
7, system delay is little, can realize real-time simulation.
Description of drawings
Fig. 1 is the Minimally Invasive Surgery emulate system architecture figure of robot based on force feedback.Fig. 2-12 is a micro-wound operation robot department pattern synoptic diagram, and wherein, Fig. 2 is passive transverse arm one, Fig. 3 is passive transverse arm two, Fig. 4 is passive upright arm, and Fig. 5 is a master arm one, and Fig. 6 is a master arm two, Fig. 7 is a master arm three, Fig. 8 is a master arm 4, and Fig. 9 is a slide unit, and Figure 10 is a micromechanics, Figure 11 is a base, and Figure 12 is the micro-wound operation robot wiring layout behind the unit construction shown in Fig. 2-11.Figure 13 and Figure 16 are the department patterns in the human body soft tissue model bank, and wherein Figure 13 and Figure 14 are gall-bladders, and Figure 15 is a blood vessel, and Figure 16 is the abdominal cavity, include liver, courage, intestines, stomach.Figure 17 is micro-wound operation robot operation simulating scenes, and Figure 18 is the partial enlarged drawing of the surgery location of soil 17.Figure 19 is a robotic surgery end effector principle of work.Figure 20-the 23rd, the simulating scenes synoptic diagram of operative site and operative site distortion is grasped in the micro-wound operation robot operation, wherein Figure 21 and 22 is the partial enlarged drawings among Figure 20, simulating scenes when wherein Figure 21 represents to prepare to grasp, Figure 22 are the simulating scenes when having grasped.Figure 23 is the simulating scenes that grasps back, the generation distortion of crawled position.
Embodiment
Embodiment one, present embodiment is described referring to Fig. 1.The described robot Minimally Invasive Surgery analogue system based on force feedback of present embodiment is by force feedback awareness apparatus 5, force feedback module 4, database module 1, image processing module 2, physical modeling module 3 and graph rendering module 6 constitute, force feedback module 4 output control signals are given force feedback awareness apparatus 5, described force feedback awareness apparatus 5 power outputs feed back signal to force feedback module 4, force feedback module 4 output collision alarms are given graph rendering module 6, image processing module 2 reads image data from database module 1, described image processing module 2 output three-dimensional reconstruction information are given physical modeling module 3, described physical modeling module 3 output physical model information are given force feedback module 4, and described force feedback module 4 reads kinematic constraint information from database module 1; Described pattern rendering module also reads kinematic constraint information, graph rendering module 6 from database module 1;
Described database module 1 is used for storing medical image raw data, robot Minimally Invasive Surgery apparatus model data and kinematic constraint information;
Described image processing module 2 is used for the image data of reading of data library module 1, and converts the image data that reads to three-dimensional data;
Described physical modeling module 3 is used for the three-dimensional data of reading images processing module 2, and makes up geometric model according to described three-dimensional data;
Described force feedback module 4, be used for going out the size and Orientation of feedback force according to the calculation of parameter that the geometric model and the force feedback awareness apparatus 5 of physical modeling module 3 structures are exported, and these feedback forces are outputed to the force feedback awareness apparatus after controlling and compensating, and then make the operator experience power by this force feedback awareness apparatus 5;
Described graph rendering module 6, the image information after being used for robot motion's information acquisition that the collision information that sends according to force feedback module 4 and database module 1 sent and playing up, and realize the output of video information.
Described physical model information comprises particle postition, the elastic coefficient and ratio of damping described in the present embodiment.
Force feedback awareness apparatus 5 described in the present embodiment can adopt the virtual haptic equipment of existing Omega.Choosing model in the present embodiment is the virtual haptic equipment of OMEGA.3.
Embodiment two, present embodiment is described referring to Fig. 1.Present embodiment is the explanation to a kind of embodiment of the database module 1 described in the embodiment one, database module 1 described in the present embodiment is made up of image database 1-1, the Minimally Invasive Surgery apparatus model bank 1-2 of robot and kinematic constraint unit 1-3, wherein:
Image database 1-1 is used for the storing medical image raw data;
The Minimally Invasive Surgery apparatus model bank 1-2 of robot, the partial model and the various operating theater instruments model data that are used to store micro-wound operation robot;
Kinematic constraint unit 1-3 is used for storing each parts that the Minimally Invasive Surgery apparatus model bank 1-2 of robot stores and the kinematical constraint connection relation and the characteristics of motion between the various operating theater instruments.
Medical image raw data described in the present embodiment comprises the image data of different sexes, all ages and classes, various disease.These data are all stored and are read according to the DICOM data standard.
The described medical image raw data of present embodiment can be the view data of gathering from medical supplies such as CT or MRI.
In the partial model and various operating theater instruments model of the micro-wound operation robot of storing among the described Minimally Invasive Surgery apparatus model bank 1-2 of robot, include the relevant kinematics information of Minimally Invasive Surgery apparatus, described relevant kinematics information is meant under certain coordinate definition, according to the robot end position, both can resolve in conjunction with above-mentioned data and to have obtained whole rod members each self-corresponding position and attitude, and just solve robot each rod member when action and how to have upgraded position and this problem of attitude of oneself.These data show as one group of matrix on mathematics, each rod member all has two matrixes, an expression attitude, an expression position.Interrelated between these matrixes, formed a kind of kinematic constraint, make virtual robot in the light of actual conditions to move.
Model data among the Minimally Invasive Surgery apparatus model bank 1-2 of robot described in the present embodiment, all be that these model datas export as the 3DS form and store by the data of actual object being surveyed acquisition, the model data of utilizing 3D modeling software such as 3D Studio MAX foundation to obtain.These model datas all are open, also are variable, and these model datas are to link up the operating robot of forming in the simulated environment according to kinematical constraint by kinematic constraint unit 1-3.
Embodiment three, present embodiment is described referring to Fig. 1.Present embodiment is further specifying Minimally Invasive Surgery apparatus model bank 1-2 of robot described in the embodiment two and kinematic constraint unit 1-3, in the model data among the Minimally Invasive Surgery apparatus model bank 1-2 of robot described in the present embodiment, given different materials for the model of each moving component, different diffusing has been set.Like this, can strengthen the demonstration of robot motion unit in virtual environment.
Kinematic constraint related data among the described kinematic constraint unit 1-3 is according to the DH algorithm position and the attitude of robot to be stipulated.
Embodiment four, present embodiment is described referring to Fig. 1.Present embodiment is the explanation to a kind of embodiment of the image processing module 2 described in the embodiment one, and the image processing module 2 described in the present embodiment is made up of image pretreatment unit 2-1 and three-dimensional reconstruction unit 2-2, wherein:
Image pretreatment unit 2-1, the image data in the reading of data library module 1, and described image data carried out sending to after the image pre-service three-dimensional reconstruction unit 2-2; Described image pre-service comprises the pretreated technological means of primary image such as gray scale enhancing, Threshold Segmentation.
Three-dimensional reconstruction unit 2-2, the data that are used to adopt the MC algorithm to obtain after the image pre-service are carried out three-dimensional reconstruction, obtain three-dimensional data.
Described image processing module 2 can obtain a three-dimensional data, gets ready for next step carries out physical modeling.Accompanying drawing 13-16 is a human body soft tissue model storehouse department pattern, is respectively the organize models in gall-bladder, blood vessel and abdominal cavity, and described model is based on that geometric model that above-mentioned three-dimensional data shines upon out sets up.
Embodiment five, present embodiment is described referring to Fig. 1.Present embodiment is the explanation to a kind of embodiment of the physical modeling module 3 described in the embodiment one, and the physical modeling module 3 described in the present embodiment is made up of subregion grid division unit 3-1 and improved particle spring modeling unit 3-2, wherein:
Subregion grid division unit 3-1 is used for according to focus distribution, internal organs characteristics and care zone three-dimensional data being carried out the subregion grid dividing;
Improved particle spring modeling unit 3-2 is used for carrying out preliminary modeling according to the three-dimensional data after the subregion grid dividing, obtains particle-spring model, and Newton second law is followed in the motion of the single particle in this particle-spring model, for particle i:
In the formula:
m
iThe quality of expression particle i; x
IjRelative displacement between expression particle i and the particle j, g represents acceleration of gravity; N is a positive integer, the quantity of the particle that expression and particle i have an effect.
The described physical modeling of present embodiment has been taken all factors into consideration gravity factor, damping factor etc. when modeling, and the elasticity coefficient of spring model is carried out regionality distribution setting, but also utilizes related function to consider some dynamicss of model.These work are that the physical property that virtual operation fully highlights surgical object when carrying out is got ready.
Embodiment six, present embodiment is described referring to Fig. 1.Present embodiment is the explanation to a kind of embodiment of the force feedback module 4 described in the embodiment one, force feedback module 4 described in the present embodiment is made up of collision detection unit 4-1, particle displacement change calculations unit 4-2, feedback force computing unit 4-3 and power control and compensating unit 4-4, wherein:
Collision detection unit 4-1, be used to judge whether the end of robot bumps with organ, and the positional information that accurate location model bumps under situation about bumping, and the positional information that will collide sends to graph rendering module 6 and particle displacement change calculations unit 4-2 simultaneously;
Particle displacement change calculations unit 4-2, be used for when receiving the collision information that collision detection unit 4-1 sent, calculate the change in displacement information of the particle of position of collision immediately, and then obtain the deformation data of organ model, and the change in displacement information of described particle is sent to feedback force computing unit 4-3;
Feedback force computing unit 4-3 is used for calculating the information that obtains feedback force according to the particle displacement change information, and this feedback force information is sent to power control and compensating unit 4-4;
Power control and compensating unit 4-4 are used for obtaining force compensating information according to force feedback information, send power control information and force compensating information and send to force feedback awareness apparatus 5.
Power control described in the present embodiment and compensating unit 4-4 are used for force feedback is carried out perfect, take all factors into consideration damping, gravity and kinetic factor force signal is compensated.
Collision information described in the present embodiment comprises the direction and the position of collision.
Embodiment seven, present embodiment is described referring to Fig. 1.Present embodiment is the explanation to a kind of embodiment of the graph rendering module 6 described in the embodiment one, and the graph rendering module 6 described in the present embodiment is made up of image rendering unit 6-1 and video output unit 6-2, wherein:
Images such as described image rendering unit 6-1 is used to create viewpoint, illumination is set, data read, form renewal, form destruction are played up operation;
Described video output unit 6-2 is used for converting the image information of image rendering unit 6-1 output to video information output.
The present invention can realize the micro-wound operation robot Analog Simulation System of carrying out repeatedly with the medical image data of the real human body construction basis as simulated environment.
The present invention adopts virtual haptic equipment as force feedback awareness apparatus 5, in the process that realizes analogue system of the present invention, can realize in conjunction with a developing instrument Chai3D of programming language C++, openGL and virtual haptic system, described developing instrument Chai3D is an OO kit based on C++, the abstract function of a lot of higher levels is provided, can need not too much to consider the physical layer interface function in the performance history, thereby can consider the generation of scene, calculating and the control that physical modeling feeds back with joint efforts by emphasis, embodiment is as follows:
Establishment for scene, at first to utilize cWorld, cLight, three classes of cCamera to define three objects, purpose is to create a virtual environment and illumination is set and view information, also to stipulate simultaneously information such as the size of form, position, style, demonstration, refresh cycle, also need simultaneously feedback awareness apparatus 5 is discerned, these work major parts are finished in graph rendering module 6, and each module begins to carry out work according to certain triggering and program circuit under this prerequisite.
To the motion simulation of robot, realize by canned data and force feedback awareness apparatus 5 and graph rendering module 6 among Minimally Invasive Surgery apparatus model bank 1-2 of robot and the kinematic constraint unit 1-3.
Referring to the set up process of Fig. 2-12 for the realistic model of micro-wound operation robot part, be implemented in the motion association of each parts in the virtual environment by programming, comprise big arm; forearm and terminal action; promptly: each robot Minimally Invasive Surgery apparatus model group load shown in Fig. 2-11 is connected together;; after little apparatus assembling of operation usefulness shown in Figure 10; be installed on the base shown in Figure 11, finally form the realistic model of the robot arm that is used for Minimally Invasive Surgery shown in Figure 12 with Fig. 2 and 3 described two passive transverse arms; passive upright arm shown in Figure 4; four master arms shown in Fig. 5-8; slide unit shown in Figure 9.
The realistic model of the robot arm that is used for Minimally Invasive Surgery that above-mentioned foundation obtains is in the motion of virtual environment, be under the constraint of kinematic constraint unit 1-3, to accept controlling of feedback awareness apparatus 5, and the characteristics of kinematic constraint have determined robot end 7-4 manipulater to move around the apocenter O shown in the robotic surgery end effector principle of work shown in the accompanying drawing 19, among Figure 19,7-1 is a sleeve pipe, 7-2 is little apparatus of laparoscope or other operations, 7-3 is the little of operation usefulness, 7-4 is meant the end of little apparatus, 7-5 is meant the soft tissue of human organ model, what the arrow among the figure was represented each parts can move direction, therefore the operator just can make robot end in the virtual environment reach any desired location in the operative space by this feedback awareness apparatus 5, and then implements virtual operation and operate.Effectively motion simulation has highlighted the work characteristics of micro-wound operation robot.
Force feedback module 4 in the analogue system of the present invention, when being used to realize that analog simulation is worked as operative site and robot end and do not bumped, referring to shown in Figure 20, robot in the micro-wound operation robot operation simulating scenes moves by the aforementioned machines people motion simulation characteristics of motion, does not have power to feel during doctor's operational feedback awareness apparatus 5 and experiences.
When robot in the virtual environment contacts with organ, force feedback module 4 is had an effect, terminal and the organ of operating robot has an effect, at first judges whether collision has taken place between virtual operation robot end and the organ at collision detection unit 4-1, and under situation about bumping the position that bumps of location model accurately.Enable particle displacement change calculations unit 4-2 then model is implemented deformation process, the visual context of enabling simultaneously after control of feedback force computing unit 4-3 and power and compensating unit 4-4 will interact shows by graph rendering module 6, accompanying drawing 21 and Figure 22 have shown that the end of little apparatus in the micro-wound operation robot surgical procedure grasps the view of process and distortion, Figure 23 grasps simulating scenes figure afterwards, when grasping, to the feedback awareness apparatus 5 stable force feedback of output in real time, power is felt refresh rate more than 1000Hz, and then feels impression effectively when making doctor's operational feedback awareness apparatus 5.
Claims (9)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110150678 CN102207997B (en) | 2011-06-07 | 2011-06-07 | Robot minimally invasive surgery simulation system based on force feedback |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110150678 CN102207997B (en) | 2011-06-07 | 2011-06-07 | Robot minimally invasive surgery simulation system based on force feedback |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102207997A true CN102207997A (en) | 2011-10-05 |
| CN102207997B CN102207997B (en) | 2013-10-16 |
Family
ID=44696825
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201110150678 Expired - Fee Related CN102207997B (en) | 2011-06-07 | 2011-06-07 | Robot minimally invasive surgery simulation system based on force feedback |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102207997B (en) |
Cited By (31)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102814815A (en) * | 2012-08-24 | 2012-12-12 | 广东工业大学 | Virtual engineering robot system and control method |
| CN102831289A (en) * | 2012-06-11 | 2012-12-19 | 董建增 | Atrial fibrillation catheter ablation simulator based on three-dimensional electroanatomic mapping system (CARTO) |
| CN103632595A (en) * | 2013-12-06 | 2014-03-12 | 合肥德易电子有限公司 | Medical education training system of multiple intracavity therapeutic endoscopic surgeries |
| CN104063540A (en) * | 2014-06-10 | 2014-09-24 | 南京信息工程大学 | Volute spiral spring model supporting haptic feedback enhancement |
| CN104281448A (en) * | 2014-09-03 | 2015-01-14 | 中国人民解放军63892部队 | Combined type scene simulation system and simulation method supporting function customizing |
| CN104598033A (en) * | 2015-02-05 | 2015-05-06 | 武汉大学 | Multi-coil electromagnetic type haptic feedback device and method |
| CN104950692A (en) * | 2015-05-20 | 2015-09-30 | 上海交通大学 | Grasping simulation system applicable to under-actuated manipulators |
| CN106446452A (en) * | 2016-10-19 | 2017-02-22 | 南京信息工程大学 | Model construction method for arched beam spring deformation model based on force sense and tactile sense interaction |
| CN106781941A (en) * | 2016-11-24 | 2017-05-31 | 北京理工大学 | A kind of method and its system for simulating microtrauma puncture operation |
| CN107550566A (en) * | 2011-11-30 | 2018-01-09 | 法国医疗科技公司 | By operating theater instruments with respect to the robot assisted device that patient body is positioned |
| CN108648821A (en) * | 2018-03-21 | 2018-10-12 | 北京理工大学 | Intelligent operation decision system and its application process towards puncturing operation robot |
| CN108777001A (en) * | 2018-06-26 | 2018-11-09 | 宁波极呈光电有限公司 | Surgical simulation method and device |
| CN109009434A (en) * | 2018-08-15 | 2018-12-18 | 苏州大学张家港工业技术研究院 | Abdominal minimally invasive Robot Virtual operation method |
| CN109192030A (en) * | 2018-09-26 | 2019-01-11 | 郑州大学第附属医院 | True hysteroscope Minimally Invasive Surgery simulation training system and method based on virtual reality |
| CN109243614A (en) * | 2018-09-11 | 2019-01-18 | 深圳先进技术研究院 | A kind of Surgery Simulation methods, devices and systems |
| CN109616211A (en) * | 2018-12-20 | 2019-04-12 | 南昌大学 | A hybrid seam model based on mass spring and its knotting method |
| CN109935311A (en) * | 2019-03-21 | 2019-06-25 | 刘伟民 | A kind of large size digital medical instrument method for managing security and system |
| CN110931121A (en) * | 2019-11-29 | 2020-03-27 | 重庆邮电大学 | Remote operation guiding device based on Hololens and operation method |
| CN110974425A (en) * | 2019-12-20 | 2020-04-10 | 哈尔滨工业大学 | A method for training a surgical instrument gripping force perception model |
| CN111026269A (en) * | 2019-12-04 | 2020-04-17 | 上海褚信医学科技有限公司 | Haptic feedback method, device and equipment of biological tissue structure based on force feedback |
| CN111599459A (en) * | 2020-05-15 | 2020-08-28 | 京东方科技集团股份有限公司 | Control method and control device for remote surgery and surgery system |
| CN112168361A (en) * | 2020-10-29 | 2021-01-05 | 上海工程技术大学 | A Pose Prediction Method of Catheter Surgery Robot Effectively Relieving the Influence of Time Delay |
| CN112989449A (en) * | 2021-03-26 | 2021-06-18 | 温州大学 | Motion stiffness optimized haptic force feedback simulation interaction method and device |
| CN113011590A (en) * | 2021-03-17 | 2021-06-22 | 华南理工大学 | Cross-test virtual force evaluation system based on micro-current stimulation |
| WO2021195916A1 (en) * | 2020-03-31 | 2021-10-07 | 西门子股份公司 | Dynamic hand simulation method, apparatus and system |
| CN114267219A (en) * | 2021-12-15 | 2022-04-01 | 上海交通大学医学院附属第九人民医院 | Virtual operation simulation device and simulation system |
| CN114924643A (en) * | 2022-05-12 | 2022-08-19 | 北京工商大学 | Dynamic force feedback mapping method based on video content detection |
| CN115019591A (en) * | 2022-08-05 | 2022-09-06 | 上海华模科技有限公司 | Operation simulation method, operation simulation device and storage medium |
| WO2024074016A1 (en) * | 2022-10-08 | 2024-04-11 | 网易（杭州）网络有限公司 | Deformation control method and apparatus for virtual model, and electronic device |
| CN118230967A (en) * | 2024-05-23 | 2024-06-21 | 四川大学华西医院 | A simulation method for flexible tissue with force feedback device |
| WO2024212046A1 (en) * | 2023-04-10 | 2024-10-17 | 深圳先进技术研究院 | Deformable object simulation method for robotic manipulation |
-
2011
- 2011-06-07 CN CN 201110150678 patent/CN102207997B/en not_active Expired - Fee Related
Non-Patent Citations (4)
| Title |
|---|
| 丁再珍: "基于虚拟现实技术的微创手术系统研究", 《中国优秀硕士学位论文全文数据库》 * |
| 付宜利等: "腹腔微创手术机器人虚拟手术仿真系统研究", 《机械与电子》 * |
| 冯美等: "腹腔微创手术机器人末端执行结构的设计和实现", 《机器人》 * |
| 孙银山等: "基于肝脏力模型的机器人辅助进针策略", 《机器人》 * |
Cited By (47)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107550566A (en) * | 2011-11-30 | 2018-01-09 | 法国医疗科技公司 | By operating theater instruments with respect to the robot assisted device that patient body is positioned |
| CN102831289A (en) * | 2012-06-11 | 2012-12-19 | 董建增 | Atrial fibrillation catheter ablation simulator based on three-dimensional electroanatomic mapping system (CARTO) |
| CN102831289B (en) * | 2012-06-11 | 2016-01-06 | 董建增 | The catheter ablation of atrial fibrillation simulator of Mapping System is cutd open based on three-dimensional electrolysis |
| CN102814815A (en) * | 2012-08-24 | 2012-12-12 | 广东工业大学 | Virtual engineering robot system and control method |
| CN103632595A (en) * | 2013-12-06 | 2014-03-12 | 合肥德易电子有限公司 | Medical education training system of multiple intracavity therapeutic endoscopic surgeries |
| CN103632595B (en) * | 2013-12-06 | 2016-01-13 | 合肥德易电子有限公司 | Multiple intracavitary therapy endoscopic surgery doctor religion training system |
| CN104063540A (en) * | 2014-06-10 | 2014-09-24 | 南京信息工程大学 | Volute spiral spring model supporting haptic feedback enhancement |
| CN104281448B (en) * | 2014-09-03 | 2018-01-05 | 中国人民解放军63892部队 | A kind of combined type vision emulation system for supporting that function is customized and emulation mode |
| CN104281448A (en) * | 2014-09-03 | 2015-01-14 | 中国人民解放军63892部队 | Combined type scene simulation system and simulation method supporting function customizing |
| CN104598033A (en) * | 2015-02-05 | 2015-05-06 | 武汉大学 | Multi-coil electromagnetic type haptic feedback device and method |
| CN104598033B (en) * | 2015-02-05 | 2017-05-10 | 武汉大学 | Multi-coil electromagnetic type haptic feedback device and method |
| CN104950692B (en) * | 2015-05-20 | 2017-08-11 | 上海交通大学 | A kind of crawl analogue system suitable for underactuated manipulator |
| CN104950692A (en) * | 2015-05-20 | 2015-09-30 | 上海交通大学 | Grasping simulation system applicable to under-actuated manipulators |
| CN106446452A (en) * | 2016-10-19 | 2017-02-22 | 南京信息工程大学 | Model construction method for arched beam spring deformation model based on force sense and tactile sense interaction |
| CN106446452B (en) * | 2016-10-19 | 2019-03-29 | 南京信息工程大学 | The modeling method of arched girder spring deformation model based on power haptic interaction |
| CN106781941A (en) * | 2016-11-24 | 2017-05-31 | 北京理工大学 | A kind of method and its system for simulating microtrauma puncture operation |
| CN108648821A (en) * | 2018-03-21 | 2018-10-12 | 北京理工大学 | Intelligent operation decision system and its application process towards puncturing operation robot |
| CN108648821B (en) * | 2018-03-21 | 2020-12-01 | 北京理工大学 | Intelligent surgical decision-making system for puncture surgical robot and its application method |
| CN108777001A (en) * | 2018-06-26 | 2018-11-09 | 宁波极呈光电有限公司 | Surgical simulation method and device |
| CN109009434A (en) * | 2018-08-15 | 2018-12-18 | 苏州大学张家港工业技术研究院 | Abdominal minimally invasive Robot Virtual operation method |
| CN109243614B (en) * | 2018-09-11 | 2020-07-31 | 深圳先进技术研究院 | Operation simulation method, device and system |
| CN109243614A (en) * | 2018-09-11 | 2019-01-18 | 深圳先进技术研究院 | A kind of Surgery Simulation methods, devices and systems |
| CN109192030A (en) * | 2018-09-26 | 2019-01-11 | 郑州大学第附属医院 | True hysteroscope Minimally Invasive Surgery simulation training system and method based on virtual reality |
| CN109616211B (en) * | 2018-12-20 | 2023-05-02 | 南昌大学 | Mixed suture line model based on mass point springs and knotting method thereof |
| CN109616211A (en) * | 2018-12-20 | 2019-04-12 | 南昌大学 | A hybrid seam model based on mass spring and its knotting method |
| CN109935311A (en) * | 2019-03-21 | 2019-06-25 | 刘伟民 | A kind of large size digital medical instrument method for managing security and system |
| CN110931121A (en) * | 2019-11-29 | 2020-03-27 | 重庆邮电大学 | Remote operation guiding device based on Hololens and operation method |
| CN111026269B (en) * | 2019-12-04 | 2023-10-10 | 上海褚信医学科技有限公司 | Haptic feedback method, device and equipment for biological tissue structure based on force feedback |
| CN111026269A (en) * | 2019-12-04 | 2020-04-17 | 上海褚信医学科技有限公司 | Haptic feedback method, device and equipment of biological tissue structure based on force feedback |
| CN110974425A (en) * | 2019-12-20 | 2020-04-10 | 哈尔滨工业大学 | A method for training a surgical instrument gripping force perception model |
| WO2021195916A1 (en) * | 2020-03-31 | 2021-10-07 | 西门子股份公司 | Dynamic hand simulation method, apparatus and system |
| CN111599459A (en) * | 2020-05-15 | 2020-08-28 | 京东方科技集团股份有限公司 | Control method and control device for remote surgery and surgery system |
| CN112168361A (en) * | 2020-10-29 | 2021-01-05 | 上海工程技术大学 | A Pose Prediction Method of Catheter Surgery Robot Effectively Relieving the Influence of Time Delay |
| CN112168361B (en) * | 2020-10-29 | 2021-11-19 | 上海工程技术大学 | Catheter surgical robot pose prediction method capable of effectively relieving time delay influence |
| CN113011590A (en) * | 2021-03-17 | 2021-06-22 | 华南理工大学 | Cross-test virtual force evaluation system based on micro-current stimulation |
| CN113011590B (en) * | 2021-03-17 | 2023-07-18 | 华南理工大学 | Cross-subject virtual force assessment system based on microcurrent stimulation |
| CN112989449B (en) * | 2021-03-26 | 2023-08-15 | 温州大学 | A tactile force feedback simulation interaction method and device for motion stiffness optimization |
| CN112989449A (en) * | 2021-03-26 | 2021-06-18 | 温州大学 | Motion stiffness optimized haptic force feedback simulation interaction method and device |
| CN114267219A (en) * | 2021-12-15 | 2022-04-01 | 上海交通大学医学院附属第九人民医院 | Virtual operation simulation device and simulation system |
| CN114267219B (en) * | 2021-12-15 | 2024-03-08 | 上海交通大学医学院附属第九人民医院 | Virtual operation simulation device and simulation system |
| CN114924643A (en) * | 2022-05-12 | 2022-08-19 | 北京工商大学 | Dynamic force feedback mapping method based on video content detection |
| CN114924643B (en) * | 2022-05-12 | 2024-09-06 | 北京工商大学 | Dynamic force feedback mapping method based on video content detection |
| CN115019591A (en) * | 2022-08-05 | 2022-09-06 | 上海华模科技有限公司 | Operation simulation method, operation simulation device and storage medium |
| CN115019591B (en) * | 2022-08-05 | 2022-11-04 | 上海华模科技有限公司 | Operation simulation method, device and storage medium |
| WO2024074016A1 (en) * | 2022-10-08 | 2024-04-11 | 网易（杭州）网络有限公司 | Deformation control method and apparatus for virtual model, and electronic device |
| WO2024212046A1 (en) * | 2023-04-10 | 2024-10-17 | 深圳先进技术研究院 | Deformable object simulation method for robotic manipulation |
| CN118230967A (en) * | 2024-05-23 | 2024-06-21 | 四川大学华西医院 | A simulation method for flexible tissue with force feedback device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102207997B (en) | 2013-10-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102207997A (en) | Force-feedback-based robot micro-wound operation simulating system | |
| Kühnapfel et al. | Endoscopic surgery training using virtual reality and deformable tissue simulation | |
| Escobar-Castillejos et al. | A review of simulators with haptic devices for medical training | |
| CN104778894B (en) | A kind of virtual emulation bone-setting manipulation training system and its method for building up | |
| Basdogan et al. | Virtual environments for medical training: graphical and haptic simulation of laparoscopic common bile duct exploration | |
| Basdogan et al. | VR-based simulators for training in minimally invasive surgery | |
| Kühnapfel et al. | The Karlsruhe endoscopic surgery trainer as an example for virtual reality in medical education | |
| Kühnapfel et al. | 3D modeling for endoscopic surgery | |
| Wang et al. | A computer-based real-time simulation of interventional radiology | |
| JP2008134373A (en) | Method and system of preparing biological data for operation simulation, operation simulation method, and operation simulator | |
| Zhang et al. | An optimized mass-spring model with shape restoration ability based on volume conservation | |
| Cakmak et al. | VS One, a virtual reality simulator for laparoscopic surgery | |
| Ayache et al. | Simulation of endoscopic surgery | |
| Abate et al. | A pervasive visual–haptic framework for virtual delivery training | |
| Tang et al. | Virtual laparoscopic training system based on VCH model | |
| Mi et al. | A multi-body mass-spring model for virtual reality training simulators based on a robotic guide wire operating system | |
| Aloisio et al. | Computer-based simulator for catheter insertion training | |
| De Paolis et al. | Virtual model of the human brain for neurosurgical simulation | |
| Zhu et al. | A real-time deformation modeling scheme of soft tissue for virtual surgical | |
| Eriksson | Haptic Milling Simulation in Six Degrees-of-Freedom: With Application to Surgery in Stiff Tissue | |
| Grecu et al. | A virtual system for simulation of human upper limb | |
| Suzuki et al. | Performing virtual surgery with a force feedback system | |
| Chen et al. | The Virtual Movable Human Upper Body for Palpatory Diagnostic Training | |
| Zhang et al. | A simulation framework for ultrasound-guided minimally invasive robotic breast surgery | |
| Ourahmoune et al. | A survey of echographic simulators. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20180625 Address after: 215163 Room 302, 13 building, 158 Jinfeng Road, science and Technology City, Suzhou, Jiangsu, -2 Patentee after: Suzhou CIGNA Tektronix Medical Technology Co Ltd Address before: 150001 No. 92 West straight street, Nangang District, Heilongjiang, Harbin Patentee before: Harbin Institute of Technology |
|
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20131016 Termination date: 20210607 |