CN109859849B - Soft tissue puncture force modeling method based on segmented artificial neural network - Google Patents
Soft tissue puncture force modeling method based on segmented artificial neural network Download PDFInfo
- Publication number
- CN109859849B CN109859849B CN201910155896.3A CN201910155896A CN109859849B CN 109859849 B CN109859849 B CN 109859849B CN 201910155896 A CN201910155896 A CN 201910155896A CN 109859849 B CN109859849 B CN 109859849B
- Authority
- CN
- China
- Prior art keywords
- puncture
- data
- soft tissue
- neural network
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Instructional Devices (AREA)
Abstract
A soft tissue puncture force modeling method based on a segmented artificial neural network comprises the following steps: 1) selecting a soft tissue sample to be modeled, performing a puncture experiment, and collecting sample data; 2) analyzing and sorting the sample data, dividing all the data into three groups according to the puncture stage, namely before puncture, after puncture until the deepest point and pulling out, and dividing each group of data into a training group and a testing group; 3) training a neural network by using the training set, and evaluating the obtained neural network by using the test set; 4) and calling the three neural networks obtained by training in a segmented manner according to the puncture stage to complete modeling. The modeling prediction error is small and is in a range which is difficult to be perceived by human hands; the modeling is easy, and the time and the labor are saved; the complete process of soft tissue puncture can be mechanically modeled, and modeling can be well completed even in the stage with the strongest nonlinearity; the model has certain universal applicability to the same tissues of different individuals, and the overall predictive trend is consistent with the experimental data.
Description
Technical Field
The invention belongs to the technical field of information, and relates to a biomechanics modeling method.
Background
In recent years, as the development of virtual reality technology is highly emphasized by the nation, virtual surgery systems are gradually accepted and applied to the medical field. At present, the technologies of the virtual surgery systems on the market in the aspects of interaction equipment, cutting image modeling, medical image processing and the like are mature, but most systems only stop at the visualization stage due to the complexity of biomechanics, and few complete systems with good tactile feedback exist. The lack of tactile force feedback causes virtual surgery to be different from clinical surgery, which is the biggest disadvantage of a virtual surgery system and directly causes the virtual surgery system to have poor teaching effect on doctors.
The existing soft tissue mechanics model mostly uses a mass point-spring-damping model or a finite element model, and the methods can achieve certain effect but the former is difficult to accurately model the whole course of puncture, and the latter is long in time consumption and poor in real-time performance. Therefore, it is necessary to establish a relatively accurate soft tissue mechanical model, which can lay a foundation for the further development of the future virtual surgery system.
Disclosure of Invention
The invention aims to provide a soft tissue puncture force modeling method based on a segmented artificial neural network, which provides good force feedback for a virtual surgery system and enhances the in-situ operation feeling of a user. The invention gets rid of the inherent framework of the existing modeling method, so that the method is easy to operate, short in time consumption and accurate in modeling result.
The invention is realized by the following technical scheme.
The invention relates to a soft tissue puncture force modeling method based on a segmented artificial neural network, which comprises the following steps of:
step 1: selecting a soft tissue sample block to be modeled, and flatly placing the soft tissue sample block below a mechanical arm with a puncture needle at the tail end and connected with a force sensor. The control arm punctures the soft tissue, because puncture this department soft tissue damage can not repeated puncture after the puncture, need adjust the puncture point of next time after each round puncture, avoid the accuracy that the same point puncture influences the data of gathering. And ensuring that the puncture frequency of the final experiment is more than 5 times and collecting enough sample data.
Step 2: the position, speed and acceleration data of the puncture needle are obtained through the mechanical arm, and the puncture force data are obtained through the ATI six-dimensional force sensor. The measured surface position coordinates of the soft tissue sample and the puncture needle position coordinates are used for making a difference, and soft tissue deformation size data can be roughly obtained. Correspondingly combining the data of deformation, puncture speed, puncture acceleration and puncture force into a data set with 4 columns.
And step 3: the data components are divided into three groups of data before puncturing the surface layer of the soft tissue, after puncturing until the deepest point and pulling back the puncture needle. The obtained three groups of data are reused to randomly extract 85% of data from each group of data as a training sample group, and the rest 15% of data are used as a test sample group.
And 4, step 4: putting a training sample of a data group before puncturing the surface layer of the soft tissue into an MATLAB working space, taking deformation, puncturing speed and acceleration as input of a neural network, taking puncturing force as output of the neural network, training sample data by utilizing a self-contained neural network tool kit or self-writing a neural network training program, and setting the learning rate to be 0.01 and the target mean square error to be 0.001 in training. And after the training is finished, evaluating the training network by using the test sample, and if the error is larger than the expected error, retraining the training network again, otherwise, storing the network. And (4) sequentially training the three groups of data in the step (3) according to the same method to finally obtain three neural networks corresponding to the three groups of sample data.
And 5: when the model is needed to be used, the read related data of the puncture needle is used as input, the puncture process is judged to be one of the three processes, and the threshold force for puncturing the soft tissue is set to be FmaxAnd calling a corresponding neural network according to the following segmented neural network, and calculating according to a calculation formula of BP neural network forward propagation to obtain an output layer result, so that more accurate model prediction puncture force can be obtained.
Where x represents the penetration depth of the needle,
the speed of penetration of the needle is indicated,
indicating the puncture acceleration of the needle.
The basic idea of the invention is as follows: experiments prove that the soft tissue puncture force has a certain relation with the puncture depth and the speed of the puncture needle and the acceleration, but the relation can not be described by a simple linear formula, so that the relation between the neural networks is used for learning and seeking a mapping relation. The idea of the invention is to set the three independent variables as the input of the neural network, and the puncture force as the output of the neural network; taking a soft tissue sample needing modeling to perform a puncture experiment, and collecting sample data; the mapping characteristics of the force and three variables in the whole puncture process of the soft tissue can go through several different stages, if all data are put into a neural network for training, an accurate mapping network is difficult to obtain, and therefore the puncture process is divided into three data sets according to the change characteristics of the force along with the puncture depth; initializing an initial weight w and a threshold b of the neural network randomly, and starting training by using a program to finally obtain three neural networks; and integrating the three neural networks to complete modeling, and calling the neural networks in a segmented manner according to the puncture stage when in use.
Compared with the prior art, the method has the following advantages: 1) the prediction force error of the modeling is small. Experiments show that when the interaction force of the same soft tissue of the same individual is predicted, the average error is far lower than 0.1N. And the maximum error is less than 0.3N even when predicting the same soft tissue of different individuals. Errors of this magnitude are difficult for a human hand to perceive and are not practical for virtual surgical systems. 2) Easy to model. It is very simple and time-saving, unlike other methods that require the calculation of various parameters. 3) The complete process of soft tissue puncture can be mechanically modeled, and modeling can be well completed even in the stage with the strongest nonlinearity; 4) the model has certain universal applicability to the same tissues of different individuals, and although the existence of errors cannot be avoided due to individual differences, the overall predictive force trend is consistent with experimental data.
Drawings
FIG. 1: experimental platform diagram.
FIG. 2 is a schematic diagram: the puncture force change curve chart of the whole process of the soft tissue puncture experiment.
FIG. 3: and comparing the model predicted force with the actual measured force by taking the puncture depth as an abscissa.
FIG. 4: and comparing the model predicted force with the actual measured force by taking the sampling time as an abscissa.
FIG. 5: the model predicts a force error graph.
FIG. 6: BP neural network structure sketch map.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
As shown in fig. 1, the steps of this embodiment are as follows:
step 1: as shown in fig. 1, a piece of soft tissue sample to be modeled is selected and laid flat under the arm having the puncture needle mounted at the tip and connected to the force sensor. The control arm punctures the soft tissue, because puncture this department soft tissue damage can not repeated puncture after the puncture, need adjust the puncture point of next time after each round puncture, avoid the accuracy that the same point puncture influences the data of gathering. And ensuring that the puncture frequency of the final experiment is more than 5 times and collecting enough sample data.
Step 2: the position, speed and acceleration data of the puncture needle are obtained through the mechanical arm, and the puncture force data are obtained through the ATI six-dimensional force sensor. The measured surface position coordinates of the soft tissue sample and the puncture needle position coordinates are used for making a difference, and soft tissue deformation size data can be roughly obtained. Correspondingly combining the data of deformation, puncture speed, puncture acceleration and puncture force into a data set with 4 columns.
And step 3: the data components are divided into three groups of data before puncturing the surface layer of the soft tissue, after puncturing until the deepest point and pulling back the puncture needle. Grouping is shown according to fig. 2, from the sampling starting point to point a, the stage before puncturing the surface layer of the soft tissue is obtained, the puncturing force at this stage is the rigid force of the soft tissue for resisting deformation, and therefore, the change of the force along with the sampling time is smoother. And until the point A has an obvious force drop back, because the point A reaches a puncture force threshold value, the stage from the point A to the point B is from the puncture point to the deepest point of the puncture, and the puncture force is changed from the rigid force to the resultant force of the friction force and the cutting force after the puncture, so that the nonlinear characteristic of the force becomes obvious later. And point B, the puncture force continuously falls back until the sampling end point, the puncture force is reversed and finally returns to zero, and the puncture needle begins to pull back out the soft tissue. The classification of the data sets can be accomplished by the above description of the phase features and the A, B two-phase turning points.
And 4, step 4: and (4) randomly extracting 85% of data from each group of data as a training sample group and taking the rest 15% of data as a test sample group by using a program for reusing the three groups of data obtained in the previous step.
And 5: putting a training sample of a data group before puncturing the surface layer of the soft tissue into an MATLAB working space, taking deformation, puncturing speed and acceleration as input of a neural network, taking puncturing force as output of the neural network, training sample data by utilizing a self-contained neural network tool box or self-writing a neural network training program, evaluating the training network by utilizing a test sample after training is finished, and storing the network if the error is larger than an expected error and retraining otherwise. And (4) sequentially training the three groups of data in the step (3) according to the same method to finally obtain three neural networks corresponding to the three groups of data.
The invention adopts BP neural network, and the training algorithm and formula are as follows:
as shown in fig. 6, X1, X2 and X3 are input layers of a neural network corresponding to the penetration depth, the penetration speed and the penetration acceleration of the soft tissue, H1 to H3 are hidden layers, and Y is an output layer corresponding to the soft tissue penetration force. w1 to w9 are connection weights from the input layer to the hidden layer, w10 to w12 are connection weights from the hidden layer to the output layer, and b1 to b4 are neuron thresholds. The back propagation algorithm of the BP neural network is suitable for training of a multilayer neural network based on a gradient descent method, and the main learning process is divided into a signal forward propagation process and an error correction back propagation process.
Firstly, randomly initializing connection weight values and threshold values, and calculating the output result of the hidden layer by using the following formula under the assumption that the activation functions used by the hidden layer and the output layer are Sigmoid functions f (x) is 1+1/(1+ e ^ x):
InH1＝X1*w1+X2*w2+X3*w3+b1
InH2＝X1*w4+X2*w5+X3*w6+b2
InH3＝X1*w7+X2*w8+X3*w9+b3
the output results of the output layer are then calculated in the same way:
InY＝OutH1*w10+OutH2*w11+OutH3*w12+b4
the calculated output is then compared to the expected output to obtain an error value. Based on the Widrow-Hoff learning rule, the error is propagated reversely, and the connection weight between the output layer and the hidden layer is adjusted and corrected according to a gradient descent method, wherein the derivative f' (x) ═ f (x) (1-f (x)) of the Sigmoid function is needed, and finally, the adjustment formulas of w10, w11, w12 and b4 can be obtained:
eY＝ExpectY-OutY
similarly, a general formula of the connection weight correction between the hidden layer and the input layer is derived:
η in the above equation represents the learning rate, the lower the learning rate is set, the more reliable the training is, but because of the small optimization of each step towards the minimum of the loss function, a longer time is consumed; the higher the learning rate is set, the shorter the training time is, but if the learning rate is too high, the result may not converge or even diverge at all, so the setting of the learning rate plays an important role in neural network training, the learning rate set in the experiment mentioned in the present invention is 0.01, and different learning rates required to be set for different sample data situations do not need to be bound by this value. And calculating and adjusting the weight and the threshold value of each layer by using the formula, updating the neural network, and finally obtaining the optimized network. The training algorithm is complex to describe, and the direct use of the correlation function and neural network toolbox in MATLAB may be considered to save programming time.
Step 6: when the model is needed, the read puncture needle related data is used as input and judgedThe puncture process of the fracture is one of the three processes, and the threshold force for puncturing the soft tissue is set as FmaxAnd calling a corresponding neural network according to the following segmented neural network formula to calculate according to the BP neural network forward propagation calculation formula mentioned in the step 5 to obtain an output layer result, so that more accurate model prediction puncture force can be obtained.
As shown in fig. 3 and 4, which are graphs comparing the model predicted force with the actual measured force, it can be seen that the error is small and the model predicted force is accurate, and the error graph of fig. 5 shows that the maximum error in the whole process is less than 0.07N, so that the error force applied to the virtual surgery system in this size can be ignored.
Claims (1)
1. A soft tissue puncture force modeling method based on a segmented artificial neural network is characterized by comprising the following steps:
step 1: selecting a soft tissue sample to be modeled, and flatly placing the soft tissue sample under a mechanical arm of which the tail end is provided with a puncture needle and is connected with a force sensor; the mechanical arm is controlled to puncture soft tissues, and the next puncture point is adjusted after each puncture, so that the influence of the same-point puncture on the accuracy of collected data is avoided; ensuring that the experiment puncture times are more than 5, and collecting enough sample data;
step 2: obtaining the position, speed and acceleration data of the puncture needle through a mechanical arm, and obtaining puncture force data through an ATI six-dimensional force sensor; the measured surface position coordinates of the soft tissue sample and the puncture needle position coordinates are used for making a difference, so that soft tissue deformation size data can be roughly obtained; correspondingly combining the data of deformation, puncture speed, puncture acceleration and puncture force into a data set with 4 rows;
and step 3: dividing the data components into three groups of data before puncturing the surface layer of the soft tissue, after puncturing until the deepest point and pulling back the puncture needle; randomly extracting 85% of data from each group of data as a training sample group, and taking the rest 15% of data as a test sample group;
and 4, step 4: putting a training sample of a data group before puncturing the surface layer of the soft tissue into an MATLAB working space, taking deformation, puncturing speed and acceleration as input of a neural network, taking puncturing force as output of the neural network, training sample data by utilizing a self-contained neural network tool box or self-writing a neural network training program, and setting the learning rate to be 0.01 and the target mean square error to be 0.001 in training; after the training is finished, evaluating the training network by using the test sample, if the error is larger than the expected error, retraining again, otherwise, storing the network; training the three groups of data in the step 3 according to the same method in sequence to finally obtain three neural networks corresponding to the three groups of sample data;
and 5: when the model is used, read related data of the puncture needle is used as input, the puncture process is judged to be one of the three processes, and the threshold force for puncturing the soft tissue is set to be FmaxAnd calling a corresponding neural network according to the following segmented neural network, and calculating according to a calculation formula of BP neural network forward propagation to obtain an output layer result so as to obtain more accurate model prediction puncture force:
where x represents the penetration depth of the needle,
the speed of penetration of the needle is indicated,
indicating the puncture acceleration of the needle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910155896.3A CN109859849B (en) | 2019-03-01 | 2019-03-01 | Soft tissue puncture force modeling method based on segmented artificial neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910155896.3A CN109859849B (en) | 2019-03-01 | 2019-03-01 | Soft tissue puncture force modeling method based on segmented artificial neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109859849A CN109859849A (en) | 2019-06-07 |
| CN109859849B true CN109859849B (en) | 2022-05-20 |
Family
ID=66899550
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910155896.3A Active CN109859849B (en) | 2019-03-01 | 2019-03-01 | Soft tissue puncture force modeling method based on segmented artificial neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109859849B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113761796B (en) * | 2021-08-24 | 2023-04-18 | 中国人民解放军总医院第一医学中心 | Simulation system and method of heart hemorrhage and hemostasis model based on virtual reality |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014002805A1 (en) * | 2012-06-26 | 2014-01-03 | Canon Kabushiki Kaisha | Puncture control system and method therefor |
| CN104007027A (en) * | 2014-06-17 | 2014-08-27 | 河北大学 | Biological soft tissue characteristic test and mechanical parameter calibration system |
| CN104720812A (en) * | 2013-12-21 | 2015-06-24 | 青海大学 | Real-time soft tissue deformation amount measurement method in robot-assisted flexible needle penetration experiments on soft tissues |
| CN107808698A (en) * | 2017-11-08 | 2018-03-16 | 哈尔滨理工大学 | A kind of mechanics and flexural deformation modeling method of flexible pin puncture soft tissue |
| EP3425519A1 (en) * | 2017-07-05 | 2019-01-09 | Siemens Aktiengesellschaft | Method for the computer-assisted configuring a data-based model based on training data |
-
2019
- 2019-03-01 CN CN201910155896.3A patent/CN109859849B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014002805A1 (en) * | 2012-06-26 | 2014-01-03 | Canon Kabushiki Kaisha | Puncture control system and method therefor |
| CN104720812A (en) * | 2013-12-21 | 2015-06-24 | 青海大学 | Real-time soft tissue deformation amount measurement method in robot-assisted flexible needle penetration experiments on soft tissues |
| CN104007027A (en) * | 2014-06-17 | 2014-08-27 | 河北大学 | Biological soft tissue characteristic test and mechanical parameter calibration system |
| EP3425519A1 (en) * | 2017-07-05 | 2019-01-09 | Siemens Aktiengesellschaft | Method for the computer-assisted configuring a data-based model based on training data |
| CN107808698A (en) * | 2017-11-08 | 2018-03-16 | 哈尔滨理工大学 | A kind of mechanics and flexural deformation modeling method of flexible pin puncture soft tissue |
Non-Patent Citations (4)
| Title |
|---|
| Experimental study of needle-tissue interaction forces effect of needle geometries, insertion methods and tissue characteristics;Shan Jiang等;《Journal of Biomechanics》;20141017;第47卷(第13期);第3344-3353页 * |
| 柔性针穿刺软组织的力学建模及变形测量;宿志亮;《中国优秀博硕士学位论文全文数据库（硕士）医药卫生科技辑》;20150515(第五期);第E060-71页 * |
| 针对按压与穿刺的生物组织力学建模算法研究;周文锦;《中国优秀博硕士学位论文全文数据库（硕士）医药卫生科技辑》;20210115(第1期);第E080-34页 * |
| 针穿刺软组织实验与力学建模技术研究;郭菁莘;《中国优秀博硕士学位论文全文数据库（硕士）医药卫生科技辑》;20150315(第三期);第E060-171页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109859849A (en) | 2019-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108463271B (en) | System and method for motor skill analysis and skill enhancement and prompting | |
| Bianchi et al. | Simultaneous topology and stiffness identification for mass-spring models based on fem reference deformations | |
| JP6049788B2 (en) | Virtual tool operation system | |
| CN100415159C (en) | Dynamic characteristic analysis method of real-time tendency of heart state | |
| CN106231528B (en) | Personalized head related transfer function generation system and method based on segmented multiple linear regression | |
| CN106600000A (en) | Method and system for human-robot motion data mapping | |
| CN105205867B (en) | A kind of collision checking method in minimally invasive virtual abdominal aorta vascular surgery | |
| CN110289104B (en) | Soft tissue compression and deformation recovery simulation method | |
| US11250726B2 (en) | System for simulation of soft bodies | |
| CN102614061A (en) | Human body upper limb functional rehabilitation training implement method based on muscle tone signals | |
| CN106156398A (en) | For the operating equipment of area of computer aided simulation and method | |
| CN109859849B (en) | Soft tissue puncture force modeling method based on segmented artificial neural network | |
| CN109446952A (en) | A kind of piano measure of supervision, device, computer equipment and storage medium | |
| CN106339071A (en) | Method and device for identifying behaviors | |
| CN111026269A (en) | Haptic feedback method, device and equipment of biological tissue structure based on force feedback | |
| CN103908361A (en) | Method for acquiring and operating artificial limb joint movement coupling drive signals | |
| KR20150075909A (en) | Method and apparatus for editing 3D character motion | |
| CN113408443B (en) | Gesture posture prediction method and system based on multi-view images | |
| CN104887241A (en) | Shimmer based cerebral thrombosis patient movement function rehabilitation assessment method | |
| CN109598799B (en) | Virtual cutting method based on CycleGAN | |
| CN111192688B (en) | Biomechanics-based ingrowing nail prediction method | |
| CN111339654A (en) | Soft tissue pressing and deformation recovery method in virtual surgery system | |
| KR20230056004A (en) | Apparatus and Method for Providing a Surgical Environment based on a Virtual Reality | |
| CN116595411A (en) | Real-time maneuver identification method and device based on early time sequence classification | |
| Gick et al. | Coarticulation in a whole event model of speech production |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |