CN112432723A - Puncture force measuring device and method based on laser speckle interference principle - Google Patents
Puncture force measuring device and method based on laser speckle interference principle Download PDFInfo
- Publication number
- CN112432723A CN112432723A CN201910787195.1A CN201910787195A CN112432723A CN 112432723 A CN112432723 A CN 112432723A CN 201910787195 A CN201910787195 A CN 201910787195A CN 112432723 A CN112432723 A CN 112432723A
- Authority
- CN
- China
- Prior art keywords
- puncture
- laser
- force
- image
- interference
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000003384 imaging method Methods 0.000 claims abstract description 17
- 239000000835 fiber Substances 0.000 claims abstract description 14
- 230000008859 change Effects 0.000 claims description 22
- 238000001914 filtration Methods 0.000 claims description 21
- 238000006073 displacement reaction Methods 0.000 claims description 20
- 230000009466 transformation Effects 0.000 claims description 19
- 210000004872 soft tissue Anatomy 0.000 claims description 18
- 238000013178 mathematical model Methods 0.000 claims description 14
- 238000009826 distribution Methods 0.000 claims description 13
- 238000012545 processing Methods 0.000 claims description 13
- 238000005286 illumination Methods 0.000 claims description 10
- 230000001681 protective effect Effects 0.000 claims description 10
- 210000001519 tissue Anatomy 0.000 claims description 10
- 230000005540 biological transmission Effects 0.000 claims description 8
- 230000008030 elimination Effects 0.000 claims description 5
- 238000003379 elimination reaction Methods 0.000 claims description 5
- 230000001131 transforming effect Effects 0.000 claims description 5
- 150000001875 compounds Chemical class 0.000 claims description 4
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 claims description 3
- 230000000149 penetrating effect Effects 0.000 claims description 3
- 238000001356 surgical procedure Methods 0.000 abstract description 4
- 230000002452 interceptive effect Effects 0.000 abstract description 3
- 238000005259 measurement Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 9
- 238000003780 insertion Methods 0.000 description 5
- 230000037431 insertion Effects 0.000 description 5
- 238000005305 interferometry Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 239000002245 particle Substances 0.000 description 3
- 230000035515 penetration Effects 0.000 description 3
- 238000002360 preparation method Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 239000003814 drug Substances 0.000 description 2
- 210000001835 viscera Anatomy 0.000 description 2
- 229910052724 xenon Inorganic materials 0.000 description 2
- FHNFHKCVQCLJFQ-UHFFFAOYSA-N xenon atom Chemical compound [Xe] FHNFHKCVQCLJFQ-UHFFFAOYSA-N 0.000 description 2
- 238000004971 IR microspectroscopy Methods 0.000 description 1
- 206010028980 Neoplasm Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000001574 biopsy Methods 0.000 description 1
- 201000011510 cancer Diseases 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000012060 immune response imaging Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 210000004185 liver Anatomy 0.000 description 1
- 210000005228 liver tissue Anatomy 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000011410 subtraction method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01L—MEASURING FORCE, STRESS, TORQUE, WORK, MECHANICAL POWER, MECHANICAL EFFICIENCY, OR FLUID PRESSURE
- G01L1/00—Measuring force or stress, in general
- G01L1/24—Measuring force or stress, in general by measuring variations of optical properties of material when it is stressed, e.g. by photoelastic stress analysis using infrared, visible light, ultraviolet
- G01L1/242—Measuring force or stress, in general by measuring variations of optical properties of material when it is stressed, e.g. by photoelastic stress analysis using infrared, visible light, ultraviolet the material being an optical fibre
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B17/00—Surgical instruments, devices or methods, e.g. tourniquets
- A61B17/34—Trocars; Puncturing needles
- A61B17/3403—Needle locating or guiding means
Abstract
The invention discloses a puncture force measuring device and a measuring method based on a laser speckle interference principle, which comprises a needle inserting unit, a laser irradiation unit and a camera unit; the needle inserting unit comprises a puncture needle body, and the puncture needle body is driven by a motor to realize automatic needle inserting; the laser irradiation unit comprises a laser fiber, a spectroscope, a beam expander I and a beam expander II, the spectroscope divides laser into two beams, one beam passes through the beam expander I and then irradiates the surface of the punctured tissue and is collected by a camera pipeline to form reflected light, and the other beam passes through the beam expander II to form reference light; the camera shooting unit comprises an imaging lens, a reflecting mirror, a semi-transparent semi-reflecting mirror, a CCD photoelectric sensor and a CCD conducting line, and is used for interfering reference light and reflected light to form a speckle interference image and transmitting the speckle interference image to the outside of a human body. According to the invention, the stress of the tip of the puncture needle is measured by laser, so that the accuracy of measuring the puncture force is improved, the needle tip of the surgical needle is accurately controlled to a certain target position area, and the success rate of the surgery is ensured.
Description
Technical Field
The invention relates to a puncture surgery medical technology, in particular to a puncture force measuring device based on a laser speckle interference principle and a puncture force measuring method thereof.
Background
The percutaneous puncture of a surgical needle into soft tissue is an important component of modern clinical medicine, and has more and more application requirements in clinical medicine, such as the processes of biopsy, injection, neurosurgery, cancer treatment and the like, all use similar minimally invasive treatment means. The most critical point of the puncture operation is to precisely control the needle point of the surgical needle to a certain target location area, but due to the interaction between the surgical needle and the surrounding tissue and the complex environment inside the tissue during the puncture process, the tissue and the surgical needle are deformed, the surgical needle deviates from the predetermined movement path and finally cannot precisely reach the target location area. In order to improve the accuracy of puncturing, people try to replace hands with mechanical equipment such as an operation robot and the like so as to eliminate human errors and improve the accuracy of puncturing.
To better understand the penetration force and provide a reference for robotic-assisted penetration control, many researchers have developed different penetration force models. The research result of Maurinetal 2004 shows the distribution diagram of the axial force along with the time and the puncture depth in the puncturing and withdrawing process of the surgical needle, Simone and Okamura work in 2002, the puncture force in the process of puncturing the liver of a cow by the surgical needle is modeled based on the distribution diagram, the puncture force can be divided into two stages of pre-puncture and post-puncture according to the distribution diagram and the model, and the puncture needle is mainly subjected to the rigid force generated by the deformation of the liver tissue before puncture; after piercing, the needle moves in the tissue, mainly subject to axial cutting forces and frictional forces, and the piercing force increases with increasing piercing depth, while the piercing force remains substantially constant as the piercing depth is constant. The simulation of soft tissue deformation by needle puncture established by Dimaio et al shows that the larger the puncture depth, the larger the puncture force and the larger the deformation of the tissue surface.
At present, the puncture force is mainly obtained by a sensor arranged at the needle tail, the precision of the measuring mode is low, interference factors are large in the measuring process of the puncture force of the tip, and the stress condition close to the puncture tip cannot be accurately obtained. With the deep research and development of surgical robots, the precision of puncture surgery needs to be based on the high-precision measurement of puncture force, so that the method for measuring the puncture force close to the tip of a puncture needle has important significance for further improving the success rate and reliability of the surgery.
Disclosure of Invention
The invention aims to overcome the defects in the prior art and provide a puncture force measuring device and a puncture force measuring method thereof based on the laser speckle interference principle.
The technical scheme adopted by the invention is as follows: a puncture force measuring device based on the laser speckle interference principle is characterized by comprising:
an outer protective sheath;
the inner soft sleeve is arranged inside the outer protective sleeve, and a needle outlet pipeline, an illumination pipeline and a camera shooting pipeline are arranged in the outer protective sleeve;
the needle inserting unit comprises a puncture needle body, the puncture needle body is arranged in the needle outlet pipeline, and the puncture needle body is driven by a motor to move along the needle outlet pipeline so as to realize automatic needle inserting;
the laser irradiation unit comprises a laser fiber, a spectroscope, a beam expander I and a beam expander II, the laser fiber is arranged in the illumination pipeline, the spectroscope and the beam expander I are sequentially arranged at the front end of the laser fiber, the beam expander II is arranged on the side face of the spectroscope, laser transmitted by the laser fiber is divided into two beams after passing through the beam expander, one beam passes through the beam expander I and then irradiates the surface of the punctured tissue and is collected by the camera pipeline to form reflected light, and the other beam passes through the beam expander II to form reference light; and the number of the first and second groups,
the camera shooting unit comprises an imaging lens, a reflector, a semi-transparent semi-reflecting mirror, a CCD photoelectric sensor and a CCD conducting wire, the CCD conducting line is arranged in the camera pipeline, the CCD photoelectric sensor, the semi-transmitting and semi-reflecting mirror, the reflecting mirror and the imaging lens are sequentially arranged at the front end of the CCD conducting line, the CCD photoelectric sensor is connected with the CCD conducting wire, the semi-transparent semi-reflecting mirror is opposite to the beam expander II, the imaging lens is used for imaging an object image on a lens of the CCD photoelectric sensor, the reflecting mirror is used for adjusting the reflected light to be parallel to the lens of the CCD photoelectric sensor, the semi-transparent semi-reflecting mirror is used for interfering the reference light and the reflected light to form a speckle interference image, and the CCD photoelectric sensor is used for transmitting the collected speckle interference image to the outside of a human body through the CCD conducting wire.
Furthermore, the needle outlet pipeline, the illuminating pipeline and the camera shooting pipeline are arranged in parallel along the longitudinal direction and are arranged in a triangular shape along the transverse direction, the needle outlet pipeline is located at the top angle of the triangle, and the illuminating pipeline and the camera shooting pipeline are respectively located at the bottom angle of the triangle.
Furthermore, a communicating channel is arranged in the inner soft sleeve and between the illuminating pipeline and the camera shooting pipeline, one end of the communicating channel is located at the spectroscope position, the other end of the communicating channel is located at the semi-transparent semi-reflecting mirror position, and the beam expander II is arranged in the communicating channel.
The other technical scheme adopted by the invention is as follows: a puncture force measuring method of the puncture force measuring device based on the laser speckle interference principle comprises the following steps:
and 4, establishing a mathematical model between the puncture force and the maximum phase change value of the stripes in the phase information based on the phase information obtained in the step 3, and calculating according to the established mathematical model to obtain the puncture force.
Further, in step 2, the removing noise of the obtained interference fringe pattern by using a fringe pattern nonlinear filtering method based on orthogonal wavelet transform includes:
step 2-1, processing the interference fringe pattern with the internal boundary to obtain a label pattern containing internal boundary high-frequency component position information;
step 2-2, performing multilayer wavelet transformation on the interference fringe pattern with the internal boundary to obtain a spectrogram of the multilayer wavelet transformation;
step 2-3, filtering the spectrogram of the multilayer wavelet transform obtained in the step 2-2 according to the label graph obtained in the step 2-1, and filtering noise of high-frequency components;
and 2-4, performing wavelet inverse transformation on the spectrogram of the multilayer wavelet transformation after the high-frequency component noise is filtered in the step 2-3, and reconstructing an image to obtain an interference fringe pattern which is used for filtering speckle noise and protecting an internal boundary.
In step 2-1, the processing of the interference fringe pattern with the internal boundary to obtain the label map containing the internal boundary high-frequency component position information specifically includes:
and subtracting the original interference fringe image from the fully filtered smooth image to obtain a distribution map of high-frequency component loss in the original interference fringe image, and marking the time-frequency information of the maximum frequency of the distribution map to obtain a marking map containing the position information of the internal boundary high-frequency component.
Further, in step 3, the obtaining of the phase information of the interference fringe pattern after the noise is removed in step 2 by using the spatial carrier fourier transform method specifically includes:
transforming the interference fringe image after noise elimination from a space domain to a frequency domain through Fourier transformation, removing high-frequency noise and carrier terms on the frequency domain, leaving the frequency of deformed fringes, transforming the frequency domain to the space domain through inverse Fourier transformation to obtain complex fringe distribution, and calculating a phase value delta phi of the interference fringe image after noise elimination through complex operation, wherein the phase value is a phase change value caused by object deformation.
Further, in step 4, the establishing a mathematical model between the magnitude of the puncturing force and the maximum phase change value of the fringes in the phase information includes:
step 4-1, establishing a mathematical model between the puncture force and the maximum out-of-plane displacement of the object:
wherein F (W)max) For the magnitude of the puncturing force, fForce of rigidityThe magnitude of the pre-puncture stiffness force, ErAlpha is the tip deflection angle of the puncture needle body (13), W is the equivalent modulusmaxMaximum out-of-plane displacement, W1Maximum out-of-plane displacement just before soft tissue puncture, fFrictional forceThe magnitude of the friction force after puncture, fTangential forceThe magnitude of the tangential force after puncture, W2The maximum value of the out-of-plane displacement when just puncturing soft tissues, mu is the friction coefficient between the puncture needle body (13) and the soft tissues, D is the outer diameter of the puncture needle body (13), I is the inertia moment of the puncture needle body (13), and E1Is the Young's modulus of the piercing needle body (13), E2Is the elastic modulus of soft tissue, upsilon2Is the poisson's ratio of the soft tissue, C is the magnitude of the tangential force, and is a constant associated with the penetrating medium;
step 4-2, the relationship between the maximum out-of-plane displacement of the object and the maximum phase change value of the stripe is as follows:
in the formula (I), the compound is shown in the specification,
the maximum phase change value of the fringes in the phase information obtained in the step 3 is shown as lambda, wherein lambda is the laser wavelength;
step 4-3, establishing a mathematical model between the puncture force and the maximum phase change value of the stripes in the phase information:
and (4) calculating according to the formula (10) to obtain the puncture force.
The invention has the beneficial effects that: the invention provides a puncture force measuring device and a puncture force measuring method based on the laser speckle interference principle, which have the advantages of high accuracy, high precision and small wound surface. Based on the medical endoscope structure, the designed laser irradiation and camera receiving structure enables the system to project laser on the puncture surface and receive scattered light, so that the puncture force can be measured in real time, and the measurement accuracy is improved; by adopting electronic speckle interferometry, non-contact measurement is realized, the measurement precision is improved, and the method has the advantages of rapidness, convenience, low environmental requirement and convenience for storage and analysis; by adopting a fringe pattern nonlinear filtering method based on orthogonal wavelet transform, most speckle noises of an image can be filtered, information loss can be reduced to the greatest extent, the internal boundary of the image is still kept clear, and the measurement precision is improved; the space carrier Fourier transform method is adopted, the optical system is simple, the requirement on the measurement environment is low, dynamic measurement can be realized, full-field measurement is realized through phase measurement, and the measurement precision is high. By exploring the defects of the existing measuring method and system, a more reliable guarantee is provided for the improvement of the measuring accuracy and the application and popularization of the surgical robot, and the method has important practical significance.
Drawings
FIG. 1: the invention relates to a front view structure schematic diagram of a puncture force measuring device based on a laser speckle interference principle;
FIG. 2: the invention relates to a puncture force measuring device based on the laser speckle interference principle, which has a right-view structure schematic diagram;
FIG. 3: the invention relates to a rear view structure schematic diagram of a puncture force measuring device based on a laser speckle interference principle;
FIG. 4: the invention relates to a working schematic diagram (overlook) of a puncture force measuring device based on the laser speckle interference principle;
FIG. 5: the invention relates to a working schematic diagram (front view) of a puncture force measuring device based on the laser speckle interference principle;
FIG. 6: the invention relates to an electronic speckle pattern interference schematic diagram;
the attached drawings are marked as follows: 1. an outer protective sheath; 2. an inner soft sleeve; 3. a needle outlet pipeline; 4. a laser fiber; 5. a beam splitter; 6. a beam expander I; 7. a beam expanding lens II; 8. an imaging lens; 9. a mirror; 10. a semi-transparent semi-reflective mirror; 11. a CCD photoelectric sensor; 12. a CCD conductive line; 13. puncturing the needle body;
b1, experimental spectroscope; b2, experimental half mirror; m1, experimental mirror I; m2, experimental mirror II; m3, experimental mirror iii; l1, an experimental beam expander I; l2, experimental imaging lens; l3, an experimental beam expander II; o, a measured object; la, a laser; dc. Experiment CCD photoelectric sensor.
Detailed Description
In order to further understand the contents, features and effects of the present invention, the following embodiments are illustrated and described in detail with reference to the accompanying drawings:
as shown in fig. 1 to 3, a puncture force measuring device based on the principle of laser speckle interference includes a housing, a needle insertion unit, a laser irradiation unit, and an image pickup unit.
(1) Outer casing
The shell comprises an outer protective sleeve 1 and an inner soft sleeve 2, wherein the inner soft sleeve 2 is arranged inside the outer protective sleeve 1. A needle outlet pipeline 3, an illumination pipeline and a camera shooting pipeline are arranged in the outer protective sleeve 1, the needle outlet pipeline 3, the illumination pipeline and the camera shooting pipeline are longitudinally arranged in parallel and transversely arranged in a triangular shape, the needle outlet pipeline 3 is positioned at the vertex angle of the triangle, and the illumination pipeline and the camera shooting pipeline are respectively positioned at the bottom angle of the triangle; a communicating channel is arranged in the inner soft sleeve 2 and between the illuminating pipeline and the camera shooting pipeline, one end of the communicating channel is positioned at the position of the spectroscope 5, and the other end of the communicating channel is positioned at the position of the semi-transparent semi-reflecting mirror 10.
(2) Needle insertion unit
The needle inserting unit comprises a puncture needle body 13, and the puncture needle body 13 is arranged in the needle outlet pipeline 3. The needle insertion unit body adopts a wired electric energy transmission driving mode, a micro motor is arranged at the tail end region of the puncture needle body 13, and the needle insertion unit body is driven by the wired electric energy transmission driving mode, so that the puncture needle body 13 moves along the needle outlet pipeline 3, and the automatic needle insertion function is realized.
(3) Laser irradiation unit
Laser irradiation unit includes laser fiber 4, spectroscope 5, beam expander I6 and beam expander II 7, laser fiber 4 sets up in the lighting tube, laser fiber 4's front end sets gradually spectroscope 5 with beam expander I6, beam expander II 7 sets up spectroscope 5's side is located communicate in the passageway.
The laser source is a medical xenon lamp cold light source with high power, high color temperature and long service life, the laser is transmitted to the front end of the device through the laser fiber 4, the laser is divided into two beams by the spectroscope 5, one beam passes through the beam expander I6 and then irradiates to the surface of the puncture tissue and is collected by the image pickup pipeline to form reflected light, and the other beam passes through the beam expander II 7 and then irradiates to the semi-transparent semi-reflecting mirror 10 of the image pickup pipeline to form reference light.
(4) Image pickup unit
The unit of making a video recording includes imaging lens 8, speculum 9, semi-transparent semi-reflecting mirror 10, CCD photoelectric sensor 11 and CCD conduction line 12, CCD conduction line 12 sets up in the pipeline of making a video recording, the front end of CCD conduction line 12 sets gradually CCD photoelectric sensor 11 semi-transparent semi-reflecting mirror 10 speculum 9 with imaging lens 8, CCD photoelectric sensor 11 with CCD conduction line 12 is connected, semi-transparent semi-reflecting mirror 10 with beam expander II 7 is relative.
The imaging lens 8 is used for imaging an object image on a lens of the CCD photoelectric sensor 11, the reflecting mirror 9 is used for adjusting the reflected light to be parallel to the lens of the CCD photoelectric sensor 11, the semi-transmitting and semi-reflecting mirror 10 is used for interfering the reference light and the reflected light to form a speckle interference image, the CCD photoelectric sensor 11 is used for transmitting the collected speckle interference image to the outside of the human body through the CCD transmission line 12 for further image processing and data analysis, and finally the magnitude of the puncture force can be displayed through a display screen.
According to the puncture force measuring method of the puncture force measuring device based on the laser speckle interference principle, the tiny deformation of soft tissues during puncture is accurately measured through a laser speckle interference method, an electronic speckle interference pattern (ESPI) is adopted to obtain a laser speckle interference pattern, the noise is eliminated through a fringe pattern nonlinear filtering method based on orthogonal wavelet transformation, the visibility and the resolution of interference fringe fringes are improved, phase measurement is carried out through spatial carrier Fourier transformation, and the quantitative information of tissue deformation is obtained. And finally, calculating to obtain a specific numerical value of the puncture force by combining a puncture force model obtained based on experimental data. The method comprises the following steps:
Step 1-1, preoperative preparation:
before a puncture operation is carried out by using a puncture force measuring device, a puncture viscus needs to be evaluated, a proper rigid puncture needle body 13 is selected to be arranged in the puncture force measuring device, a medical xenon lamp cold light source is used as a laser light source, and a CCD conducting wire 12 is connected to external equipment to process an image collected by a CCD photoelectric sensor 11.
Step 1-2, the puncture process:
as shown in fig. 4 and 5, after the preoperative preparation is completed, the tip of the puncture force measuring device is used to puncture the skin of the human body, and the puncture force measuring device is punctured into the interior of the human body, and is still at a certain distance from the internal organs, and the rigid puncture needle body 13 is driven by a micro motor to automatically insert the needle.
Meanwhile, a laser light source is turned on, the spectroscope 5 divides laser into two beams, one beam of laser is emitted in parallel and is projected to the surface of the viscera through the beam expander I6, diffuse reflection light is collected by a receiving device consisting of the imaging lens 8, the reflector 9 and the like, the other beam of laser is projected to the semi-transparent semi-reflecting mirror 10 through the beam expander II 7 and is interfered with the diffuse reflection light to form a speckle interference pattern, the speckle interference pattern is collected by the CCD photoelectric sensor 11, and the speckle interference pattern is transmitted to external equipment for processing through the CCD conducting wire 12.
Because the speckle interference image collected by the CCD photoelectric sensor 11 has high frequency speckle noise (in the present invention, the frequency is higher than 30 Hz), and the image quality is poor, the speckle interference image needs to be further processed by filtering and phase processing methods in the external device.
The acquired speckle interference images before and after deformation are processed by a subtraction mode to obtain an interference fringe image, and the synthesized image comprises two parts of information: fringe information corresponding to the distortion and noise information of the speckle particles. In the frequency domain, the speckle fringe information is a low-frequency part (in the invention, the frequency is lower than 25 Hz), the change is slow, and the high-frequency part is particle noise, and the change is prominent.
Because the fringe pattern nonlinear filtering method based on orthogonal wavelet transform can selectively select different filtering modes in different areas of the image, most speckle noises of the image can be filtered, the loss of information can be reduced to the maximum extent, the internal boundary of the image still keeps clear, the measurement precision is improved, and therefore the fringe pattern nonlinear filtering method is selected to carry out speckle image processing. The treatment process comprises the following steps:
step 2-1, inputting an interference fringe pattern with an internal boundary.
And 2-2, processing the interference fringe pattern with the internal boundary to obtain a label pattern containing the position information of the high-frequency component of the internal boundary. And subtracting the original interference fringe image from the fully filtered smooth image to obtain a distribution diagram of high-frequency component loss in the original interference fringe image, wherein the place with large gray value means that the high-frequency loss is large, and the place with the maximum gray value is around the boundary in the original interference fringe image. And marking the time-frequency information of the maximum frequency of the distribution graph to obtain a marked graph containing the position information of the high-frequency components of the internal boundary.
And 2-3, performing multilayer wavelet transformation on the interference fringe pattern with the internal boundary to obtain a spectrogram of the multilayer wavelet transformation, wherein the spectrogram contains time information and frequency information and can be regarded as a time-frequency pattern.
And 2-4, according to the label graph obtained in the step 2-2, performing selective filtering processing on the spectrogram of the multilayer wavelet transform obtained in the step 2-3, and filtering out noise of high-frequency components, namely filtering out high-frequency components at all points which do not correspond to the positions of the internal boundary of the label graph, and reserving the high-frequency components at the internal boundary.
And 2-5, performing wavelet inverse transformation on the spectrogram of the multilayer wavelet transformation subjected to the high-frequency component noise filtering in the step 2-4, and reconstructing an image.
And 2-6, finishing the steps to obtain an interference fringe image which not only filters speckle noise but also protects the internal boundary, namely obtaining a clear interference fringe image.
And 3, calculating the puncture force.
After a clear interference fringe image is obtained, the principle of measuring the micro out-of-plane displacement of an object by using electronic speckle interferometry (ESPI) is utilized, micro deformation information of the tissue surface can be obtained through speckle interference images, and then a corresponding value between the puncture force and the maximum deformation information of the object is obtained based on a large amount of experimental data calibration, so that the puncture force can be obtained.
First, the principle of electronic speckle interferometry is given as follows:
the basic principle of electronic speckle interferometry (ESPI) is to introduce reference light to interfere with a speckle light field generated by an object, record interference light fields before and after the object is deformed, subtract the interference light fields and take absolute values to obtain speckle related fringes modulated by high-frequency speckle noise, wherein the phase reflects the deformation information of the object.
As shown in fig. 6, laser emitted by a laser La is divided into two beams by an experimental beam splitter B1, one beam of laser is irradiated to a measured object O by an experimental beam expander il 1 and reflected to form object light, and the object light is imaged in front of a lens of an experimental CCD photoelectric sensor Dc by an experimental imaging lens L2; the other laser beam forms reference light through an experimental reflector I M1, an experimental reflector II M2, an experimental reflector III M3 and an experimental beam expander II L3 to an experimental half-mirror B2, and the experimental half-mirror B2 interferes the reference light and object light to form a speckle interference image.
A laser beam irradiates the surface of an object to be measured O, because of the random distribution of surface particles, the interference of different scattered lights on each point generates a plurality of corresponding randomly distributed bright and dark spots, and after the experimental imaging lens L2 images, the complex amplitude of object light on an image plane can be expressed as:
in the formula of U0(r) is the complex amplitude of the object light, u0(r) is the amplitude of the object light,
is the phase of the object light.
The complex amplitude expression of the reference light is:
in the formula of UR(r) is the complex amplitude of the reference light, uR(r) is the amplitude of the reference light,
is the phase of the reference light.
The intensity of interference light formed by the object light and the reference light on the surface of the experimental CCD photoelectric sensor Dc is as follows:
in the formula IRIs the intensity of interference light formed by the object light and the reference light on the surface of the experimental CCD photosensor Dc.
Amplitude u of speckle field at each point of surface when object O to be measured deforms0(r) will be substantially constant and the phase will become
Wherein the content of the first and second substances,
the phase difference of speckle fields before and after deformation is adopted, so the synthesized light intensity after deformation is as follows;
in the formula IR' is the intensity of the interference light after deformation.
The light intensity before and after the measured object O is deformed is subtracted by adopting a digital image subtraction method, namely:
in the formula (I), the compound is shown in the specification,
the difference in interference light intensity before and after deformation.
As can be seen from the above, the low-frequency fringes are determined by the phase change of the optical wave caused by the deformation of the object to be measured O, and the phase change of the optical wave and the deformation information of the object to be measured O can be derived from the propagation theory of light:
in the formula, λ' is experimental laser wavelength, W is out-of-plane displacement, U is in-plane displacement, and θ is an included angle between the irradiation light and the normal line of the surface of the object to be measured O. If θ is very small, the relationship between the phase and the object displacement can be obtained:
step 3-1, phase acquisition: and (3) acquiring the phase information of the interference fringe pattern after the noise is eliminated in the step (2) by adopting a space carrier Fourier transform method.
The invention adopts a space carrier Fourier transform method to obtain the phase information of the interference fringe pattern after noise elimination. Transforming the interference fringe pattern after eliminating noise from space domain to frequency domain by Fourier transform, removing high-frequency noise and carrier wave term in frequency domain, only leaving the frequency of deformed fringe, and using inverse Fourier transformThe vertical transformation is converted from the frequency domain to the space domain to obtain a complex fringe distribution, and the phase value of the interference fringe pattern after the noise is eliminated can be calculated through complex operation
The phase value is a phase change value caused by deformation of the object.
Step 3-2, obtaining puncture force: and (3) establishing a mathematical model between the puncture force and the maximum phase change value of the stripes in the phase information based on the phase information obtained in the step (3-1), and calculating according to the established mathematical model to obtain the puncture force.
And 3-2-1, selecting the maximum out-of-plane displacement as a reference, wherein the puncture force before puncture is rigid force, and the puncture force after puncture is friction force and tangential force, so that a mathematical model between the puncture force and the maximum out-of-plane displacement of the object can be obtained through fitting based on a large amount of experimental data:
wherein F (W)max) For the magnitude of the puncturing force, fForce of rigidityThe magnitude of the pre-puncture stiffness force, ErAlpha is the tip deflection angle of the puncture needle body 13 and W is the equivalent modulusmaxMaximum out-of-plane displacement, W1Maximum out-of-plane displacement just before soft tissue puncture, fFrictional forceThe magnitude of the friction force after puncture, fTangential forceThe magnitude of the tangential force after puncture, W2The maximum value of the out-of-plane displacement when just puncturing soft tissues, mu is the friction coefficient between the puncture needle body 13 and the soft tissues, D is the outer diameter of the puncture needle body 13, I is the inertia moment of the puncture needle body 13, E1Young's modulus of the puncture needle body 13, E2Is the elastic modulus of soft tissue, upsilon2Poisson's ratio for soft tissue, C is the magnitude of tangential force, and is a constant associated with penetrating media, typically 0.0776 + -0.0139N.
Step 3-2-2, the relationship between the maximum out-of-plane displacement of the object and the maximum phase change value of the stripe is as follows:
in the formula (I), the compound is shown in the specification,
and (4) the maximum phase change value of the fringes in the phase information obtained in the step (3), wherein lambda is the laser wavelength.
3-2-3, establishing a mathematical model between the puncture force and the maximum phase change value of the stripes in the phase information:
the maximum phase change value of the fringes is obtained by a space carrier Fourier transform method, and the puncture force can be calculated based on a formula (10).
Although the preferred embodiments of the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and those skilled in the art can make many modifications without departing from the spirit and scope of the present invention as defined in the appended claims.
Claims (8)
1. A puncture force measuring device based on the laser speckle interference principle is characterized by comprising:
an outer protective sheath (1);
the inner soft sleeve (2) is arranged inside the outer protective sleeve (1), and a needle outlet pipeline (3), an illumination pipeline and a camera shooting pipeline are arranged in the outer protective sleeve (1);
the needle inserting unit comprises a puncture needle body (13), the puncture needle body (13) is arranged in the needle outlet pipeline (3), and the puncture needle body (13) is driven by a motor to move along the needle outlet pipeline (3) to realize automatic needle inserting;
the laser irradiation unit comprises a laser fiber (4), a spectroscope (5), a beam expander I (6) and a beam expander II (7), the laser fiber (4) is arranged in the illumination pipeline, the spectroscope (5) and the beam expander I (6) are sequentially arranged at the front end of the laser fiber (4), the beam expander II (7) is arranged on the side face of the spectroscope (5), laser transmitted through the laser fiber (4) is divided into two beams after passing through the spectroscope (5), one beam is irradiated to the surface of a punctured tissue after passing through the beam expander I (6) and collected through the image pickup pipeline to form reflected light, and the other beam passes through the beam expander II (7) to form reference light; and the number of the first and second groups,
the image pickup unit comprises an imaging lens (8), a reflector (9), a half mirror (10), a CCD photoelectric sensor (11) and a CCD transmission line (12), the CCD transmission line (12) is arranged in the image pickup pipeline, the front end of the CCD transmission line (12) is sequentially provided with the CCD photoelectric sensor (11), the half mirror (10), the reflector (9) and the imaging lens (8), the CCD photoelectric sensor (11) is connected with the CCD transmission line (12), the half mirror (10) is opposite to the beam expander II (7), the imaging lens (8) is utilized to image an object on a lens of the CCD photoelectric sensor (11), the reflector (9) is utilized to adjust the reflected light to be parallel to the lens of the CCD photoelectric sensor (11), and the half mirror (10) is utilized to interfere the reference light and the reflected light to form an interference image, and the collected speckle interference image is transmitted to the outside of the human body through the CCD transmission line (12) by utilizing the CCD photoelectric sensor (11).
2. The puncture force measuring device based on the laser speckle interference principle according to claim 1, wherein the needle outlet pipe (3), the illumination pipe and the camera pipe are arranged in parallel along the longitudinal direction and arranged in a triangle along the transverse direction, the needle outlet pipe (3) is located at the vertex angle of the triangle, and the illumination pipe and the camera pipe are respectively located at the base angle of the triangle.
3. The puncture force measuring device based on the laser speckle interference principle as claimed in claim 1, wherein a communicating channel is arranged in the inner soft sleeve (2) and between the illumination pipeline and the image pickup pipeline, one end of the communicating channel is located at the spectroscope (5), the other end of the communicating channel is located at the half-transmitting and half-reflecting mirror (10), and the beam expander II (7) is arranged in the communicating channel.
4. A puncturing force measuring method based on the puncturing force measuring device based on the laser speckle interference principle according to any one of the claims 1 to 3, comprising the steps of:
step 1, acquiring a speckle interference image by a puncture force measuring device based on a laser speckle interference principle according to any one of claims 1 to 3;
step 2, obtaining an interference fringe pattern after processing the speckle interference image obtained in the step 1 by adopting a subtraction mode, and eliminating noise of the obtained interference fringe pattern by adopting a fringe pattern nonlinear filtering method based on orthogonal wavelet transform;
step 3, acquiring phase information of the interference fringe pattern after the noise is eliminated in the step 2 by adopting a space carrier Fourier transform method;
and 4, establishing a mathematical model between the puncture force and the maximum phase change value of the stripes in the phase information based on the phase information obtained in the step 3, and calculating according to the established mathematical model to obtain the puncture force.
5. The method according to claim 4, wherein the step 2 of eliminating noise from the obtained interference fringe pattern by using a fringe pattern nonlinear filtering method based on orthogonal wavelet transform comprises:
step 2-1, processing the interference fringe pattern with the internal boundary to obtain a label pattern containing internal boundary high-frequency component position information;
step 2-2, performing multilayer wavelet transformation on the interference fringe pattern with the internal boundary to obtain a spectrogram of the multilayer wavelet transformation;
step 2-3, filtering the spectrogram of the multilayer wavelet transform obtained in the step 2-2 according to the label graph obtained in the step 2-1, and filtering noise of high-frequency components;
and 2-4, performing wavelet inverse transformation on the spectrogram of the multilayer wavelet transformation after the high-frequency component noise is filtered in the step 2-3, and reconstructing an image to obtain an interference fringe pattern which is used for filtering speckle noise and protecting an internal boundary.
6. The method according to claim 5, wherein in step 2-1, the step of processing the interference fringe pattern with the inner boundary to obtain the label pattern containing the position information of the high-frequency component of the inner boundary comprises:
and subtracting the original interference fringe image from the fully filtered smooth image to obtain a distribution map of high-frequency component loss in the original interference fringe image, and marking the time-frequency information of the maximum frequency of the distribution map to obtain a marking map containing the position information of the internal boundary high-frequency component.
7. The method according to claim 4, wherein in step 3, the step of obtaining the phase information of the interference fringe pattern after the noise is removed in step 2 by using the spatial carrier fourier transform method specifically comprises:
transforming the interference fringe image after noise elimination from a space domain to a frequency domain through Fourier transformation, removing high-frequency noise and carrier terms on the frequency domain, leaving the frequency of deformed fringes, transforming the frequency domain to the space domain through inverse Fourier transformation to obtain complex fringe distribution, and calculating a phase value delta phi of the interference fringe image after noise elimination through complex operation, wherein the phase value is a phase change value caused by object deformation.
8. The method according to claim 4, wherein the step 4 of establishing the mathematical model between the magnitude of the puncturing force and the maximum phase change value of the fringes in the phase information comprises:
step 4-1, establishing a mathematical model between the puncture force and the maximum out-of-plane displacement of the object:
wherein F (W)max) For the magnitude of the puncturing force, fForce of rigidityThe magnitude of the pre-puncture stiffness force, ErAlpha is the tip deflection angle of the puncture needle body (13), W is the equivalent modulusmaxMaximum out-of-plane displacement, W1Maximum out-of-plane displacement just before soft tissue puncture, fFrictional forceThe magnitude of the friction force after puncture, fTangential forceThe magnitude of the tangential force after puncture, W2The maximum value of the out-of-plane displacement when just puncturing soft tissues, mu is the friction coefficient between the puncture needle body (13) and the soft tissues, D is the outer diameter of the puncture needle body (13), I is the inertia moment of the puncture needle body (13), and E1Is the Young's modulus of the piercing needle body (13), E2Is the elastic modulus of soft tissue, upsilon2Is the poisson's ratio of the soft tissue, C is the magnitude of the tangential force, and is a constant associated with the penetrating medium;
step 4-2, the relationship between the maximum out-of-plane displacement of the object and the maximum phase change value of the stripe is as follows:
in the formula (I), the compound is shown in the specification,
the maximum phase change value of the fringes in the phase information obtained in the step 3 is shown as lambda, wherein lambda is the laser wavelength;
step 4-3, establishing a mathematical model between the puncture force and the maximum phase change value of the stripes in the phase information:
and (4) calculating according to the formula (10) to obtain the puncture force.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910787195.1A CN112432723B (en) | 2019-08-25 | 2019-08-25 | Puncture force measuring device and method based on laser speckle interference principle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910787195.1A CN112432723B (en) | 2019-08-25 | 2019-08-25 | Puncture force measuring device and method based on laser speckle interference principle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112432723A true CN112432723A (en) | 2021-03-02 |
| CN112432723B CN112432723B (en) | 2021-12-07 |
Family
ID=74690013
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910787195.1A Active CN112432723B (en) | 2019-08-25 | 2019-08-25 | Puncture force measuring device and method based on laser speckle interference principle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112432723B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114440783A (en) * | 2021-12-31 | 2022-05-06 | 西安交通大学 | Transformer oil tank body deformation monitoring device and method |
| CN114767236A (en) * | 2022-06-24 | 2022-07-22 | 北京微刀医疗科技有限公司 | Light output puncture needle, light sensing puncture needle and puncture distance measuring system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090192522A1 (en) * | 2007-12-18 | 2009-07-30 | Intuitive Surgical, Inc | Force sensor temperature compensation |
| CN103239241A (en) * | 2012-02-07 | 2013-08-14 | 森斯奥普蒂克股份公司 | Optical force measuring element and microsurgical device |
| CN103278272A (en) * | 2013-05-31 | 2013-09-04 | 哈尔滨工业大学 | Soldered joint stress concentration testing device and stress concentration testing method based on laser speckle technology |
| CN104248471A (en) * | 2013-06-27 | 2014-12-31 | 中国科学院沈阳自动化研究所 | Robot-assisted oblique-tip flexible needle puncture system and method |
| CN106052565A (en) * | 2016-06-12 | 2016-10-26 | 天津大学 | A dual-optical-path three-dimensional speckle interference system based on spectroscopes |
| CN108908337A (en) * | 2018-07-26 | 2018-11-30 | 苏州科技大学 | Manipulator repetitive positioning accuracy measuring device and method based on digital speckle interference |
| CN110118537A (en) * | 2019-06-12 | 2019-08-13 | 合肥工业大学 | Deformation and strain synchronized measurement system and measurement method based on speckle interference |
-
2019
- 2019-08-25 CN CN201910787195.1A patent/CN112432723B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090192522A1 (en) * | 2007-12-18 | 2009-07-30 | Intuitive Surgical, Inc | Force sensor temperature compensation |
| CN103239241A (en) * | 2012-02-07 | 2013-08-14 | 森斯奥普蒂克股份公司 | Optical force measuring element and microsurgical device |
| CN103278272A (en) * | 2013-05-31 | 2013-09-04 | 哈尔滨工业大学 | Soldered joint stress concentration testing device and stress concentration testing method based on laser speckle technology |
| CN104248471A (en) * | 2013-06-27 | 2014-12-31 | 中国科学院沈阳自动化研究所 | Robot-assisted oblique-tip flexible needle puncture system and method |
| CN106052565A (en) * | 2016-06-12 | 2016-10-26 | 天津大学 | A dual-optical-path three-dimensional speckle interference system based on spectroscopes |
| CN108908337A (en) * | 2018-07-26 | 2018-11-30 | 苏州科技大学 | Manipulator repetitive positioning accuracy measuring device and method based on digital speckle interference |
| CN110118537A (en) * | 2019-06-12 | 2019-08-13 | 合肥工业大学 | Deformation and strain synchronized measurement system and measurement method based on speckle interference |
Non-Patent Citations (1)
| Title |
|---|
| 李兆璋: "激光医学的进展", 《中国医学物理学杂志》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114440783A (en) * | 2021-12-31 | 2022-05-06 | 西安交通大学 | Transformer oil tank body deformation monitoring device and method |
| CN114440783B (en) * | 2021-12-31 | 2023-03-31 | 西安交通大学 | Transformer oil tank body deformation monitoring device and method |
| CN114767236A (en) * | 2022-06-24 | 2022-07-22 | 北京微刀医疗科技有限公司 | Light output puncture needle, light sensing puncture needle and puncture distance measuring system |
| CN114767236B (en) * | 2022-06-24 | 2022-09-02 | 北京微刀医疗科技有限公司 | Light output puncture needle, light-sensitive puncture needle and puncture distance measuring system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112432723B (en) | 2021-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230414176A1 (en) | Intravascular Measurement And Data Collections Systems, Apparatus And Methods | |
| US7949385B2 (en) | System and method for reconstruction of the human ear canal from optical coherence tomography scans | |
| JP6899878B2 (en) | Fiber optic sensor-guided navigation for vascular visualization and monitoring | |
| JP7437294B2 (en) | Detection of lumens, stents and/or artifacts in one or more images, such as optical coherence tomography images | |
| JP6874055B2 (en) | Devices, methods and storage media for the detection of lumens and artifacts in one or more images, such as optical coherence tomography images. | |
| US20150226537A1 (en) | System and method for optical coherence tomography and positioning element | |
| CN112432723B (en) | Puncture force measuring device and method based on laser speckle interference principle | |
| CN108670239B (en) | Three-dimensional blood flow imaging method and system based on feature space | |
| EP0581704A1 (en) | Method for determining the position of an organ | |
| JP2008542758A (en) | System, method and apparatus capable of using spectrally encoded heterodyne interferometry for imaging | |
| AU2007211061A1 (en) | Method and apparatus for measurement of optical properties in tissue | |
| CN102333477A (en) | Optical tomographic imaging method and optical tomographic imaging apparatus | |
| EP2001362A2 (en) | Quantitative optoacoustic tomography with enhanced contrast | |
| WO2008144766A1 (en) | Porcine biliary tract imaging | |
| US9200886B2 (en) | Method and system for optical coherence tomography | |
| CN114965470A (en) | Light intensity transmission diffraction chromatography microimaging method based on non-interference synthetic aperture | |
| US9042963B2 (en) | System and method for acquiring images from within a tissue | |
| CN109984731A (en) | Real-time angioradiographic system and method based on whole audience time domain OCT technology | |
| KR20170039784A (en) | Optical coherence tomography device for the skin diagnostic | |
| US8379945B2 (en) | Optical apparatus for acquiring structure information and its processing method of optical interference signal | |
| Edwards | Medical optical imaging: Signal processing leads to new methods of detecting life-threatening situations [Special Reports] | |
| JP5812785B2 (en) | Optical tomographic image processing apparatus and method of operating optical tomographic image processing apparatus | |
| CN201208249Y (en) | Three dimensional endoscope measurement device based amplitude transmission grating projection | |
| CN116530914A (en) | Optical coherence tomography endoscope imaging method and system based on spectrum section | |
| CN117850018A (en) | Automatic focusing method of optical fiber endoscope and lens-free optical fiber endoscope system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |