CN117516353A - Continuum shape sensing system and method based on flexible magnet - Google Patents
Continuum shape sensing system and method based on flexible magnet Download PDFInfo
- Publication number
- CN117516353A CN117516353A CN202311344554.9A CN202311344554A CN117516353A CN 117516353 A CN117516353 A CN 117516353A CN 202311344554 A CN202311344554 A CN 202311344554A CN 117516353 A CN117516353 A CN 117516353A
- Authority
- CN
- China
- Prior art keywords
- continuum
- shape
- magnet
- magnetic field
- flexible
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000005259 measurement Methods 0.000 claims abstract description 33
- 230000008859 change Effects 0.000 claims abstract description 30
- 238000010801 machine learning Methods 0.000 claims abstract description 13
- 238000001514 detection method Methods 0.000 claims abstract description 7
- 230000008447 perception Effects 0.000 claims abstract description 4
- 239000013598 vector Substances 0.000 claims description 15
- 238000004422 calculation algorithm Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 9
- 239000000696 magnetic material Substances 0.000 claims description 8
- 239000000463 material Substances 0.000 claims description 7
- 239000011159 matrix material Substances 0.000 claims description 5
- 230000001133 acceleration Effects 0.000 claims description 3
- 230000006698 induction Effects 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 239000013013 elastic material Substances 0.000 claims description 2
- 230000007704 transition Effects 0.000 claims description 2
- 230000006870 function Effects 0.000 description 17
- 238000013528 artificial neural network Methods 0.000 description 6
- 238000012549 training Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000006467 substitution reaction Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000002324 minimally invasive surgery Methods 0.000 description 2
- 229920000642 polymer Polymers 0.000 description 2
- 238000001356 surgical procedure Methods 0.000 description 2
- QJVKUMXDEUEQLH-UHFFFAOYSA-N [B].[Fe].[Nd] Chemical compound [B].[Fe].[Nd] QJVKUMXDEUEQLH-UHFFFAOYSA-N 0.000 description 1
- WYTGDNHDOZPMIW-RCBQFDQVSA-N alstonine Natural products C1=CC2=C3C=CC=CC3=NC2=C2N1C[C@H]1[C@H](C)OC=C(C(=O)OC)[C@H]1C2 WYTGDNHDOZPMIW-RCBQFDQVSA-N 0.000 description 1
- 238000005452 bending Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000005489 elastic deformation Effects 0.000 description 1
- 229920001971 elastomer Polymers 0.000 description 1
- 239000000806 elastomer Substances 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 229910001172 neodymium magnet Inorganic materials 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000037361 pathway Effects 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000035939 shock Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000008733 trauma Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B7/00—Measuring arrangements characterised by the use of electric or magnetic techniques
- G01B7/003—Measuring arrangements characterised by the use of electric or magnetic techniques for measuring position, not involving coordinate determination
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B7/00—Measuring arrangements characterised by the use of electric or magnetic techniques
- G01B7/28—Measuring arrangements characterised by the use of electric or magnetic techniques for measuring contours or curvatures
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Measurement Of Length, Angles, Or The Like Using Electric Or Magnetic Means (AREA)
Abstract
The invention discloses a continuum shape sensing system and method based on a flexible magnet, wherein the system comprises: a flexible magnet covering the surface of the continuum or directly serving as part of the continuum body for generating a magnetic field required for shape perception; a magnetic field detection device for detecting magnetic field change information caused by movement and shape change of the flexible magnet; and the processor is used for calculating the pose and the shape of the flexible magnet according to the detected magnetic field change information and obtaining the shape information of the continuum. The method comprises the following steps: establishing a global coordinate system; detecting magnetic field change information caused by the flexible magnet when the flexible magnet moves and deforms; and acquiring the shape information of the continuum by adopting a discrete magnet positioning fitting method or a direct shape sensing method based on machine learning according to the magnetic field change information. The invention combines the flexible magnet with the measured instrument, and realizes the non-contact sensing, the safe, stable, accurate and real-time pose measurement and shape sensing of the measured instrument.
Description
Technical Field
The invention relates to the field of position tracking and shape sensing, in particular to a continuum shape sensing system and method based on a flexible magnet.
Background
Unlike conventional rigid skeletal structures, the continuum does not contain rigid links and defined rotational joints, but rather produces continuous bending by elastic deformation and movement by the generation of smooth curves, thus providing greater flexibility, flexibility in deformation, and greater resistance to shock and vibration. Continuum structures are commonly used in the design of flexible robots, where their inherent compliance and high flexibility enable them to perform flexible tasks such as object grasping, surgical manipulation, etc. through tortuous pathways. The continuum robot is applied to the minimally invasive surgery, so that the accuracy, safety and automation level of the minimally invasive medical operation can be greatly improved, and the trauma to a patient and the recovery time are further reduced. Continuum is also used as seals and hoses in hydraulic systems, such as serpentine robots for search and rescue based on hydraulic drives, powered entirely by the pressure of fire hose, can reach dangerous or narrow areas that are not reachable by firefighters.
Real-time shape sensing of the continuum during its application is important. For example, in minimally invasive surgery, real-time detection of end position and shape information of a flexible robotic arm is required. On one hand, during surgery, the flexible robot interacts with human biological tissues, and the shape and the pose of the robot are monitored in real time, so that the damage to the human tissues can be avoided; on the other hand, providing pose information feedback to the controller in real time facilitates more precise manipulation of the instrument, which is critical to performing accurate control during surgery.
The prior continuum shape sensing method is widely used as a method based on a kinematic model. The conventional kinematic model is generally based on the assumption of piecewise constant curvature, and is not suitable for the situation that the local curvature of the continuum is changed due to unknown external force or load, but the force and load of the continuum robot are unknown when the continuum moves in the human body cavity, so that the shape of the continuum robot moving in the human body cannot be accurately perceived by the methods. The medical image-based method has the disadvantages of safety and accuracy. The method based on other sensors has respective limitations and disadvantages in terms of safety, stability, practicability, sensing precision and the like, for example, the method based on the fiber bragg grating sensor has the problems of higher economic cost and larger influence on sensing precision by temperature. The magnetic field can safely penetrate the human body, the pose tracking method based on the magnetic field sensing principle can be used for tracking a continuum robot moving in the human body, and further, the shape of the continuum can be estimated by using the acquired pose information. However, the existing positioning method based on the magnetic field sensing principle needs to fix an electromagnetic coil or a rigid permanent magnet on the continuous body, which affects the flexible structure and the deformability of the continuous body.
Disclosure of Invention
In order to solve at least one of the technical problems existing in the prior art to a certain extent, the invention aims to provide a continuum shape sensing system and method based on a flexible magnet.
The technical scheme adopted by the invention is as follows:
a flexible magnet-based continuum shape sensing system, comprising:
a flexible magnet covering the surface of the continuum or directly serving as part of the continuum body for generating a magnetic field required for shape perception;
a magnetic field detection device for detecting magnetic field change information caused by movement and shape change of the flexible magnet;
and the processor is used for calculating the pose and the shape of the flexible magnet according to the detected magnetic field change information and obtaining the shape information of the continuum.
Further, the continuum is a continuum robot, a soft robot, a continuum-like instrument with multiple joints, a flexible instrument, a flexible catheter, or a flexible endoscope.
Further, the material of the flexible magnet is a high polymer elastic material doped with a magnetic material (such as neodymium iron boron);
the combination configuration of the flexible magnet and the continuum is that the hollow flexible magnet is tightly covered on the surface of the continuum to be tested, or the flexible instrument body is directly doped with magnetic materials.
Further, the magnetic field detection device includes:
a magnetic sensor for converting the magnetic induction intensity into a voltage signal;
and the signal processing unit is used for preprocessing the voltage signal and outputting a processing result.
The invention adopts another technical scheme that:
a method of continuum shape sensing based on flexible magnets, comprising the steps of:
establishing a global coordinate system to represent all devices by using uniform coordinates;
detecting magnetic field change information caused by the flexible magnet when the flexible magnet moves and deforms;
and acquiring the shape information of the continuum by adopting a discrete magnet positioning fitting method or a direct shape sensing method based on machine learning according to the magnetic field change information.
Further, the method for obtaining the shape information of the continuum by adopting the discrete magnet positioning fitting method comprises the following steps:
discretizing the continuous flexible magnet into a plurality of segments of independent rigid magnets (magnetic dipoles);
establishing a kinematic model of the multi-section rigid magnet;
establishing a mathematical relationship model between the magnetic field measurement value and the pose of the discrete rigid magnet;
analyzing magnetic field change information by combining a kinematic model and a magnetic field measurement mathematical relation model, and obtaining the pose of the multi-section rigid magnet by adopting a pose estimation algorithm (such as a Kalman filtering algorithm);
and establishing a parameterized model of the continuum shape, and obtaining the continuum shape by adopting a curve fitting method (such as Bezier curve fitting method and polynomial curve fitting method) according to the pose of the multi-section rigid magnet.
Further, the method for obtaining the shape information of the continuum by adopting the direct shape sensing method based on machine learning comprises the following steps:
establishing a parameterized model of the continuum shape;
establishing a mathematical relationship model between magnetic field measurement values and flexible magnet shape parameters based on a machine learning model (such as a neural network model);
and analyzing the magnetic field change information by using a mathematical relationship model, and directly predicting the shape parameters of the continuum to obtain the continuum shape information.
Further, the parameterized model of the continuum shape is obtained by building a polynomial representation or a Bezier curve representation.
Further, in the polynomial representation, the continuum is projected onto two orthogonal planes, using polynomial functions, respectivelyAnd->Representing the projection curve of a continuum in two planes, where a i And b i Is the parameter to be determined.
Further, the bezier curve representation uses a bezier curve explicit expression to represent the continuum:
wherein P is i For the parameter to be determined.
Further, the modeling of the mathematical relationship between the magnetic field measurements and the pose of the discrete rigid magnet includes:
the continuous flexible magnet is considered as a multi-segment rigid magnet and approximated using a magnetic dipole model for a single permanent magnet:
the magnetic sensor actually measures the sum magnetic field generated by a plurality of rigid permanent magnets, namely:
wherein mu 0 Is the vacuum dielectric constant, l represents the number of discrete rigid magnets, r j Represents the j thPosition vector of center of each permanent magnet to center of sensor, M j Representing the magnetic moment produced by the jth permanent magnet.
Further, the kinematic model of the multi-section rigid magnet is built based on a constant speed model, and the state quantity parameters to be estimated comprise three-dimensional coordinates of the flexible magnet, three-axis speed components and attitude vectors of the flexible magnet, and are expressed as follows:
X＝[p v u] T
wherein p= [ x y z ]] T Representing the triaxial position vector of the flexible magnet, v= [ v ] x v y v z ] T Representing the triaxial velocity component of a flexible magnet, u= [ mnp ]] T A pose vector representing the flexible magnet;
the magnetic field measurements are given by the magnetic sensor array:
Y＝[B x1 B y1 B z1 B x2 B y2 B z2 ... B xs B ys B zs ] T
wherein s represents the number of sensors used, [ B ] xi B yi B zi ]Representing three-axis magnetic field measurements of the ith sensor;
establishing a state equation and a magnetic field measurement equation of the system:
X k ＝Φ k-1 X k-1 +Gw k-1
Y k ＝h(X k )+c k
wherein X is k Represents the state parameter to be estimated at the moment k, phi k-1 State transition matrix Gw representing time (k-1) k-1 Representing a state estimation error at time (k-1); y is Y k Representing the actual measurement at time k, c k Representing measurement noise at time k;
the state equation is developed as:
wherein p is k Bits representing time kV is set up k Represents the velocity at time k, u k Posture vector u representing time k k ＝[m k n k p k ] T WhereinI 3 Representing a third-order identity matrix, dt representing a sampling time interval; h (X) k ) The calculation model of h is the mathematical relation model between the magnetic field measurement value and the pose of the discrete rigid magnet, and w and c respectively represent process noise and measurement noise, w a And w ω Noise representing acceleration and angular velocity, respectively.
Further, the curve fitting method is a polynomial curve fitting method or a Bezier curve fitting method.
Wherein, the polynomial curve fitting method projects the coordinates of points on a continuum in three-dimensional space into two perpendicular planes to obtain two-dimensional coordinates (x i ,y i ) Or (x) i ,z i )；
In one of the planes, a polynomial is usedRepresents a continuum, wherein a i Is a parameter to be determined, deriving a polynomial to obtain +.>Direction information representing a polynomial curve at a specific point;
defining an error function asWhere α is a coefficient that determines the error ratio of the position and direction information, calculated from the certainty of the position and direction information itself;
error function E for each coefficient a i Obtaining the deviationLet->Obtaining the coefficient with the minimum error function by solving the equation set, and determining the curve fitting result on the plane;
repeating the steps in another plane perpendicular to the plane to obtain a curve fitting result in the perpendicular plane;
therefore, any point of the continuum can be fitted by the two curve fitting results to obtain three-dimensional coordinates in space, and curve fitting of the continuum is realized.
Bezier curve fitting method, which uses a Bezier curve explicit expression to represent a continuum:
Be(t)＝(1-t) 3 P 0 +3(1-t) 2 tP 1 +3(1-t)t 2 P 2 +t 3 P 3 ，t∈[0,1]
wherein P is 0 And P 3 Respectively represent the coordinates of the start point and the end point of the curve, P 1 And P 2 Representing control point coordinates, P 1 And P 2 Respectively at P 0 (origin) and P 3 Tangential direction of (end point);
defining an error function f (||P) 0 P 1 ||,||P 2 P 3 ||)＝L-L true Wherein L represents the length of a third-order Bezier curve, L ture Representing the actual length of the flexible magnet, the length parameter P can be determined by solving a minimized error function 0 P 1 And P 2 P 3 ；
Tracking P by pose 0 And P 3 Is known in combination with the determined length parameter P 0 P 1 And P 2 P 3 I.e. the control point P can be represented 1 And P 2 Coordinates. The shape of the continuum can thus be determined.
Further, the machine learning is achieved by:
neural network: collecting and preparing magnetic field measurement and shape parameter data sets for model training, ensuring that the data sets are correctly marked; using a feedforward neural network structure and using a mean square error as a loss function; initializing parameters of the neural network, calculating and outputting a result along a forward path of the network, comparing the result with an actual label, and calculating a loss value; calculating the gradient of each parameter through a back propagation algorithm, and updating network parameters according to a gradient descent algorithm so as to minimize a loss function; and (5) repeating training to obtain a better model.
The beneficial effects of the invention are as follows: the invention combines the flexible magnet with the measured instrument, and realizes the non-contact sensing, the safe, stable, accurate and real-time pose measurement and shape sensing of the measured instrument.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description is made with reference to the accompanying drawings of the embodiments of the present invention or the related technical solutions in the prior art, and it should be understood that the drawings in the following description are only for convenience and clarity of describing some embodiments in the technical solutions of the present invention, and other drawings may be obtained according to these drawings without the need of inventive labor for those skilled in the art.
FIG. 1 is a schematic diagram of a flexible magnet-based continuum shape sensing system in an embodiment of the invention;
FIG. 2 is a flow chart of steps of a method for sensing a shape of a continuum based on a flexible magnet in an embodiment of the invention;
FIG. 3 is a flow chart of shape sensing using discrete magnet positioning fitting in an embodiment of the present invention;
FIG. 4 is a flow chart of shape sensing using a direct shape sensing method based on machine learning in an embodiment of the present invention;
fig. 5 is a schematic diagram showing direct comparison of shape estimation values and reference values obtained in the embodiment of the present invention.
Reference numerals: a flexible magnet 1, a magnetic sensor 2, a signal processing module 3, a computer 4 and a flexible catheter 5.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the invention. The step numbers in the following embodiments are set for convenience of illustration only, and the order between the steps is not limited in any way, and the execution order of the steps in the embodiments may be adaptively adjusted according to the understanding of those skilled in the art.
In the description of the present invention, it should be understood that references to orientation descriptions such as upper, lower, front, rear, left, right, etc. are based on the orientation or positional relationship shown in the drawings, are merely for convenience of description of the present invention and to simplify the description, and do not indicate or imply that the apparatus or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus should not be construed as limiting the present invention.
In the description of the present invention, a number means one or more, a number means two or more, and greater than, less than, exceeding, etc. are understood to not include the present number, and above, below, within, etc. are understood to include the present number. The description of the first and second is for the purpose of distinguishing between technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.
Furthermore, in the description of the present invention, unless otherwise indicated, "a plurality" means two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.
In the description of the present invention, unless explicitly defined otherwise, terms such as arrangement, installation, connection, etc. should be construed broadly and the specific meaning of the terms in the present invention can be reasonably determined by a person skilled in the art in combination with the specific contents of the technical scheme.
The flexible magnet has better magnetic property and mechanical property, and is easy to process into flexible bonded magnetic plates, magnetic strips, magnetic rings and other devices with complex shapes. Therefore, the flexible magnet may be formed in a sleeve shape to cover the surface of the continuous body, or may be directly doped with a magnetic material in the elastic body of the continuous body. When the continuum robot moves or changes shape, the shape of the flexible magnet changes accordingly, the magnetic field generated by the flexible magnet changes accordingly, and the magnetic field signals acquired by the sensor array change accordingly. And estimating the pose of the flexible magnet by combining the kinematic model and the magnetic field measurement mathematical relation model, and calculating the shape of the flexible magnet through pose information. Therefore, after knowing the relation between the pose of the flexible magnet and the magnetic field, the shape of the flexible catheter or other continuous bodies can be accurately estimated by combining a shape sensing method.
Referring to fig. 1, the present embodiment provides a flexible catheter shape sensing system based on a flexible magnet, comprising:
a flexible magnet covering the surface of the continuum or directly serving as part of the continuum body for generating a magnetic field required for shape perception;
a magnetic field detection device for detecting magnetic field change information caused by movement and shape change of the flexible magnet;
and the processor is used for calculating the pose and the shape of the flexible magnet according to the detected magnetic field change information and obtaining the shape information of the continuum.
As an alternative embodiment, the continuum includes, but is not limited to: continuum robots, soft robots, continuum-like instruments with multiple joints, flexible instruments, flexible catheters, flexible endoscopes.
As an alternative embodiment, the material of the flexible magnet is a polymeric elastomer material doped with a magnetic material. The combination configuration of the flexible magnet and the continuum can be that the hollow flexible magnet is tightly covered on the surface of the continuum to be tested or the flexible instrument body is directly doped with magnetic materials.
As an alternative embodiment, the magnetic field detection device includes:
a magnetic sensor for converting the magnetic induction intensity into a voltage signal;
and the signal processing unit is used for preprocessing the voltage signal and outputting a processing result.
In this embodiment, referring to fig. 1, the magnetic sensor used is a three-axis tunnel magneto-resistive sensor, nine sensors are arranged in a square shape in the same plane, and the distance between the two sensors is 50mm.
The above system is explained in detail below with reference to the drawings and the specific embodiments.
In this embodiment, a sleeve-shaped magnet made of a polymer flexible material doped with a magnetic material is covered on the surface of the flexible catheter.
As shown in fig. 1, the present embodiment provides a flexible catheter shape sensing system based on a flexible magnet, which comprises a flexible magnet 1, a magnetic sensor 2, a signal processing module 3, a computer 4 and a flexible catheter 5.
The flexible magnet 1 is prepared in a sleeve shape and covers the outside of the flexible catheter 5, and the flexible catheter 5 moves in the effective working area of the magnetic sensor 2.
Because the flexible catheter 5 moves, the position and shape of the flexible magnet 1 are correspondingly changed to cause magnetic field change, the magnetic field change is detected by the magnetic sensor 2, and the detected change signal is filtered by the signal processing module 3, takes an effective value and performs analog-digital conversion and is transmitted to the computer 4 through a cable in a serial port communication mode.
The computer 4 receives signals from the magnetic sensor 2, and estimates the pose information of the flexible magnet according to the established mathematical model between the magnetic field measurement value and the pose of the flexible magnet 1 and the kinematic model of the multi-section independent rigid permanent magnet discretized by the flexible magnet sleeve. The shape information of the flexible magnet can be estimated by combining pose information with a shape fitting algorithm. Since the flexible magnet 1 is completely covered on the surface of the flexible catheter 5, the shape sensing of the flexible catheter 5 can be achieved by estimating the shape information of the flexible magnet 1.
Based on the above system, as shown in fig. 2, the present embodiment provides a continuum shape sensing method based on a flexible magnet, which includes the following steps:
s1, establishing a global coordinate system, and representing all devices by using unified coordinates;
s2, detecting magnetic field change information caused by the flexible magnet when moving and deforming;
s3, acquiring shape information of the continuum by adopting a discrete magnet positioning fitting method or a direct shape sensing method based on machine learning according to the magnetic field change information.
In some embodiments, referring to fig. 3, the discrete magnet positioning fitting method is used to obtain the shape information of the continuum, and specifically includes the following steps:
a1, dispersing a continuous flexible magnet into a plurality of sections of independent rigid magnets;
a2, establishing a kinematic model of the multi-section rigid magnet;
a3, establishing a mathematical relation model between the magnetic field measured value and the pose of the discrete rigid magnet;
a4, analyzing magnetic field change information by combining a kinematic model and a magnetic field measurement mathematical relation model, and obtaining the pose of the multi-section rigid magnet by adopting a pose estimation algorithm;
a5, establishing a parameterized model of the continuum shape, and obtaining the continuum shape by adopting a curve fitting method according to the pose of the multi-section rigid magnet.
As an alternative embodiment, the kinematic model established in step A2 is based on a constant velocity model, and the state quantity to be estimated includes three-dimensional coordinates of the flexible magnet, velocity components in three-axis directions, and attitude vectors of the flexible magnet, which are expressed as follows:
X＝[p v u] T
wherein p= [ x y z ]] T Representing the triaxial position vector of the flexible magnet, v= [ v ] x v y v z ] T Representing the triaxial velocity component of a flexible magnet, u= [ mnp ]] T Representing the attitude vector of the flexible magnet.
The magnetic field measurements are given by the magnetic sensor array:
Y＝[B x1 B y1 B z1 B x2 B y2 B z2 ... B xs B ys B zs ] T
wherein s represents the number of sensors used, [ B ] xi B yi B zi ]Representing the triaxial magnetic field measurement of the ith sensor. Establishing a state equation and a magnetic field measurement equation of the system:
X k ＝Φ k-1 X k-1 +Gw k-1
Y k ＝h(X k )+c k
the state equation is developed as:
p k represents the position at time k, v k Represents the velocity at time k, u k Posture vector u representing time k k ＝[m k n k p k ] T WhereinI 3 Representing a third-order identity matrix, dt representing a sampling time interval; h (X) k ) The calculation model of h is the mathematical relation model between the magnetic field measurement value and the pose of the discrete rigid magnet as set forth in claim 9, w and c represent process noise and measurement noise, respectively, w a And w ω Noise representing acceleration and angular velocity, respectively.
As an alternative embodiment, the mathematical relationship model between the magnetic field measurement value and the pose of the discrete rigid magnet established in step A3 is:
discretizing the flexible magnet, treating the continuous flexible magnet as a multi-segment rigid magnet, and approximating a single permanent magnet with a magnetic dipole model:the magnetic sensor actually measures the sum magnetic field generated by a plurality of rigid permanent magnets, namely
Wherein mu 0 Is the vacuum dielectric constant, l represents the number of discrete rigid magnets, r j Representing the position vector from the center of the jth permanent magnet to the center of the sensor, M j Representing the magnetic moment produced by the jth permanent magnet.
As an alternative embodiment, in step A5, the parameterized model of the continuum shape may be established by several means including, but not limited to:
1) Polynomial representation of a continuum projected onto two orthogonal planes using polynomial functions, respectivelyAnd->Representing the projection curve of a continuum in two planes, where a i And b i Is the parameter to be determined.
2) Bezier curve representation, i.e., using the Bezier curve explicit expression to represent a continuum:
wherein P is i For the parameter to be determined.
As an alternative embodiment, in step A5, the curve fitting method includes, but is not limited to, the following:
1) Polynomial curve fitting method, which projects coordinates of points on a continuum in three-dimensional space into two perpendicular planes to obtain two-dimensional coordinates (x i ,y i ) Or (x) i ,z i )；
In one of the planes, multiple terms are usedAndRepresents a continuum, wherein a i Is a parameter to be determined, deriving a polynomial to obtain +.>Direction information representing a polynomial curve at a specific point;
defining an error function asWhere α is a coefficient that determines the error ratio of the position and direction information, calculated from the certainty of the position and direction information itself;
error function E for each coefficient a i Obtaining the deviationLet->Obtaining the coefficient with the minimum error function by solving the equation set, and determining the curve fitting result on the plane;
repeating the steps in another plane perpendicular to the plane to obtain a curve fitting result in the perpendicular plane;
therefore, any point of the continuum can be fitted by the two curve fitting results to obtain three-dimensional coordinates in space, and curve fitting of the continuum is realized.
2) Bezier curve fitting method, which uses a Bezier curve explicit expression to represent a continuum:
Be(t)＝(1-t) 3 P 0 +3(1-t) 2 tP 1 +3(1-t)t 2 P 2 +t 3 P 3 ，t∈[0,1]
wherein P is 0 And P 3 Respectively represent the coordinates of the start point and the end point of the curve, P 1 And P 2 Representing control point coordinates, P 1 And P 2 Respectively at P 0 (starting point)) And P 3 Tangential direction of (end point);
defining an error function f (||P) 0 P 1 ||,||P 2 P 3 ||)＝L-L true Wherein L represents the length of a third-order Bezier curve, L ture Representing the actual length of the flexible magnet, the length parameter P can be determined by solving a minimized error function 0 P 1 And P 2 P 3 ；
Tracking P by pose 0 And P 3 Is known in combination with the determined length parameter P 0 P 1 And P 2 P 3 I.e. the control point P can be represented 1 And P 2 Coordinates. The shape of the continuum can thus be determined.
In some embodiments, referring to fig. 4, shape sensing is performed directly by a machine learning based method, specifically comprising the steps of:
b1, establishing a parameterized model of the shape of the continuum;
b2, establishing a mathematical relation model between a magnetic field measured value based on a machine learning model and a flexible magnet shape parameter;
and B3, analyzing the magnetic field change information by using a mathematical relation model, and directly predicting the shape parameters of the continuum to obtain the continuum shape information.
As an alternative embodiment, the machine learning method includes, but is not limited to, the following:
neural network: collecting and preparing magnetic field measurement and shape parameter data sets for model training, ensuring that the data sets are correctly marked; using a feedforward neural network structure and using a mean square error as a loss function; initializing parameters of the neural network, calculating and outputting a result along a forward path of the network, comparing the result with an actual label, and calculating a loss value; calculating the gradient of each parameter through a back propagation algorithm, and updating network parameters according to a gradient descent algorithm so as to minimize a loss function; and (5) repeating training to obtain a better model.
Through simulation experiments, fig. 5 is a schematic diagram showing the comparison of the estimated shape obtained by the above method with the actual shape. As can be seen from fig. 5, the method of the present embodiment can accurately estimate the shape of the continuum flexible catheter.
In the foregoing description of the present specification, reference has been made to the terms "one embodiment/example", "another embodiment/example", "certain embodiments/examples", and the like, means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that: many changes, modifications, substitutions and variations may be made to the embodiments without departing from the spirit and principles of the invention, the scope of which is defined by the claims and their equivalents.
While the preferred embodiment of the present invention has been described in detail, the present invention is not limited to the above embodiments, and various equivalent modifications and substitutions can be made by those skilled in the art without departing from the spirit of the present invention, and these equivalent modifications and substitutions are intended to be included in the scope of the present invention as defined in the appended claims.
Claims (10)
1. A flexible magnet-based continuum shape sensing system, comprising:
a flexible magnet covering the surface of the continuum or directly serving as part of the continuum body for generating a magnetic field required for shape perception;
a magnetic field detection device for detecting magnetic field change information caused by movement and shape change of the flexible magnet;
and the processor is used for calculating the pose and the shape of the flexible magnet according to the detected magnetic field change information and obtaining the shape information of the continuum.
2. The flexible magnet-based continuum shape sensing system of claim 1, wherein the continuum is a continuum robot, a soft robot, a continuum-like instrument with multiple joints, a flexible instrument, a flexible catheter, or a flexible endoscope.
3. A continuum shape sensing system based on flexible magnets according to claim 1, wherein the material of the flexible magnets is a polymeric elastic material doped with a magnetic material;
the combination configuration of the flexible magnet and the continuum is that the hollow flexible magnet is tightly covered on the surface of the continuum to be tested, or the flexible instrument body is directly doped with magnetic materials.
4. A flexible magnet based continuum shape sensing system according to claim 1, wherein said magnetic field detection means comprises:
a magnetic sensor for converting the magnetic induction intensity into a voltage signal;
and the signal processing unit is used for preprocessing the voltage signal and outputting a processing result.
5. A method for sensing a shape of a continuum based on a flexible magnet, comprising the steps of:
establishing a global coordinate system to represent all devices by using uniform coordinates;
detecting magnetic field change information caused by the flexible magnet when the flexible magnet moves and deforms;
and acquiring the shape information of the continuum by adopting a discrete magnet positioning fitting method or a direct shape sensing method based on machine learning according to the magnetic field change information.
6. The method for sensing the shape of a continuum based on a flexible magnet according to claim 5, wherein the step of obtaining the shape information of the continuum by using a discrete magnet positioning fitting method comprises the steps of:
discretizing the continuous flexible magnet into a plurality of sections of independent rigid magnets;
establishing a kinematic model of the multi-section rigid magnet;
establishing a mathematical relationship model between the magnetic field measurement value and the pose of the discrete rigid magnet;
analyzing magnetic field change information by combining a kinematic model and a magnetic field measurement mathematical relation model, and obtaining the pose of the multi-section rigid magnet by adopting a pose estimation algorithm;
and establishing a parameterized model of the continuum shape, and obtaining the continuum shape by adopting a curve fitting method according to the pose of the multi-section rigid magnet.
7. The method of claim 5, wherein the step of obtaining the shape information of the continuum using the direct shape sensing method based on machine learning comprises:
establishing a parameterized model of the continuum shape;
establishing a mathematical relation model between a magnetic field measured value and a flexible magnet shape parameter based on a machine learning model;
and analyzing the magnetic field change information by using a mathematical relationship model, and directly predicting the shape parameters of the continuum to obtain the continuum shape information.
8. A method of sensing a continuum shape based on a flexible magnet according to claim 6 or 7, wherein said parameterized model of the continuum shape is constructed using a polynomial representation or a bezier curve representation.
9. The flexible magnet-based continuum shape sensing method of claim 6, wherein modeling a mathematical relationship between magnetic field measurements and discrete rigid magnet pose comprises:
the continuous flexible magnet is considered as a multi-segment rigid magnet and approximated using a magnetic dipole model for a single permanent magnet:
the magnetic sensor actually measures the sum magnetic field generated by a plurality of rigid permanent magnets, namely:
wherein mu 0 Is the vacuum dielectric constant, l represents the number of discrete rigid magnets, r j Representing the position vector from the center of the jth permanent magnet to the center of the sensor, M j Representing the magnetic moment produced by the jth permanent magnet.
10. The flexible magnet-based continuum shape sensing method of claim 6, wherein the kinematic model of the multi-segment rigid magnet is built based on a constant velocity model, and the state quantity parameters to be estimated comprise three-dimensional coordinates of the flexible magnet, velocity components in three-axis directions, and attitude vectors of the flexible magnet, expressed as follows:
X＝[p v u] T
wherein p= [ x y z ]] T Representing the triaxial position vector of the flexible magnet, v= [ v ] x v y v z ] T Representing the triaxial velocity component of a flexible magnet, u= [ mnp ]] T A pose vector representing the flexible magnet;
the magnetic field measurements are given by the magnetic sensor array:
Y＝[B x1 B y1 B z1 B x2 B y2 B z2 ... B xs B ys B zs ] T
wherein s represents the number of sensors used, [ B ] xi B yi B zi ]Representing three-axis magnetic field measurements of the ith sensor;
establishing a state equation and a magnetic field measurement equation of the system:
X k ＝Φ k-1 X k-1 +Gw k-1
Y k ＝h(X k )+c k
wherein X is k Represents the state parameter to be estimated at the moment k, phi k-1 State transition matrix, gw, representing time k-1 k-1 Representing the state estimation error at time k-1; y is Y k Representing the actual measurement at time k, c k Representing measurement noise at time k;
the state equation is developed as:
wherein p is k Represents the position at time k, v k Represents the velocity at time k, u k Posture vector u representing time k k ＝[m k n k p k ] T WhereinI 3 Representing a third-order identity matrix, dt representing a sampling time interval; h (X) k ) Representing a magnetic field measurement model, w a And w ω Noise representing acceleration and angular velocity, respectively.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311344554.9A CN117516353A (en) | 2023-10-17 | 2023-10-17 | Continuum shape sensing system and method based on flexible magnet |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311344554.9A CN117516353A (en) | 2023-10-17 | 2023-10-17 | Continuum shape sensing system and method based on flexible magnet |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117516353A true CN117516353A (en) | 2024-02-06 |
Family
ID=89750286
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311344554.9A Pending CN117516353A (en) | 2023-10-17 | 2023-10-17 | Continuum shape sensing system and method based on flexible magnet |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117516353A (en) |
-
2023
- 2023-10-17 CN CN202311344554.9A patent/CN117516353A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6799658B2 (en) | Systems and methods for adaptive input mapping | |
| US9333650B2 (en) | Method and system for contact detection and contact localization along continuum robots | |
| EP2121099B1 (en) | System and method for local deformable registration of a catheter navigation system to image data or a model | |
| EP2849669B1 (en) | Systems and methods for deformation compensation using shape sensing | |
| EP2148629B1 (en) | Frame mapping and force feedback methods, devices and systems | |
| Santaera et al. | Low-cost, fast and accurate reconstruction of robotic and human postures via IMU measurements | |
| EP2684519B1 (en) | Position and orientation algorithm for a single axis sensor | |
| CN113347936A (en) | Flexible multi-coil tracking sensor | |
| KR20150007020A (en) | Surgical robot system and method of controlling the same | |
| CN103211563A (en) | Methods and system for performing 3-D tool tracking by fusion of sensor and/or camera derived data during minimally invasive robotic surgery | |
| KR101666232B1 (en) | Autonomous system and method for determining information representative of the movement of an articulated chain | |
| Alambeigi et al. | Scade: Simultaneous sensor calibration and deformation estimation of fbg-equipped unmodeled continuum manipulators | |
| US20130211763A1 (en) | Method for checking position data of a medical instrument, and corresponding medical instrument | |
| Tully et al. | Constrained filtering with contact detection data for the localization and registration of continuum robots in flexible environments | |
| Zhang | Hand/eye calibration for electronic assembly robots | |
| Sanan et al. | Simultaneous Compliance and Registration Estimation for Robotic Surgery. | |
| Song et al. | Real-time multi-object magnetic tracking for multi-arm continuum robots | |
| US20230028689A1 (en) | System and method for inter-arm registration | |
| CN117516353A (en) | Continuum shape sensing system and method based on flexible magnet | |
| Lu et al. | Fbg-based variable-length estimation for shape sensing of extensible soft robotic manipulators | |
| Lin et al. | Whisker-inspired tactile sensing for contact localization on robot manipulators | |
| Howe | Model-based error correction for flexible robotic surgical instruments | |
| EP4280996A1 (en) | Method for tracking a medical tool during a medical procedure using deep learning | |
| Zhang et al. | Shape tracking and navigation for continuum surgical robot based on magnetic tracking | |
| Zhang et al. | Robust tracking of dexterous continuum robots: Fusing fbg shape sensing and stereo vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |