WO2021142138A1 - Laser speckle force feedback estimation - Google Patents
Laser speckle force feedback estimation Download PDFInfo
- Publication number
- WO2021142138A1 WO2021142138A1 PCT/US2021/012524 US2021012524W WO2021142138A1 WO 2021142138 A1 WO2021142138 A1 WO 2021142138A1 US 2021012524 W US2021012524 W US 2021012524W WO 2021142138 A1 WO2021142138 A1 WO 2021142138A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- target tissue
- tissue region
- images
- region
- deformation
- Prior art date
Links
- 238000000034 method Methods 0.000 claims abstract description 100
- 210000001519 tissue Anatomy 0.000 claims description 312
- 238000005259 measurement Methods 0.000 claims description 75
- 238000004422 calculation algorithm Methods 0.000 claims description 60
- 238000010801 machine learning Methods 0.000 claims description 51
- 239000012530 fluid Substances 0.000 claims description 48
- 230000010412 perfusion Effects 0.000 claims description 42
- 238000012549 training Methods 0.000 claims description 36
- 230000000007 visual effect Effects 0.000 claims description 22
- 238000013528 artificial neural network Methods 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 16
- 210000004369 blood Anatomy 0.000 claims description 10
- 239000008280 blood Substances 0.000 claims description 10
- 230000003287 optical effect Effects 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 9
- 210000000941 bile Anatomy 0.000 claims description 8
- 229940088597 hormone Drugs 0.000 claims description 8
- 239000005556 hormone Substances 0.000 claims description 8
- 238000000338 in vitro Methods 0.000 claims description 8
- 238000001727 in vivo Methods 0.000 claims description 8
- 210000004080 milk Anatomy 0.000 claims description 8
- 239000008267 milk Substances 0.000 claims description 8
- 235000013336 milk Nutrition 0.000 claims description 8
- 210000003097 mucus Anatomy 0.000 claims description 8
- 210000004915 pus Anatomy 0.000 claims description 8
- 210000003296 saliva Anatomy 0.000 claims description 8
- 210000000582 semen Anatomy 0.000 claims description 8
- 210000004243 sweat Anatomy 0.000 claims description 8
- 210000002700 urine Anatomy 0.000 claims description 8
- 230000003211 malignant effect Effects 0.000 claims description 7
- 210000004872 soft tissue Anatomy 0.000 claims description 7
- 238000006073 displacement reaction Methods 0.000 claims description 3
- 230000003993 interaction Effects 0.000 abstract 1
- 239000000975 dye Substances 0.000 description 11
- MOFVSTNWEDAEEK-UHFFFAOYSA-M indocyanine green Chemical compound [Na+].[O-]S(=O)(=O)CCCCN1C2=CC=C3C=CC=CC3=C2C(C)(C)C1=CC=CC=CC=CC1=[N+](CCCCS([O-])(=O)=O)C2=CC=C(C=CC=C3)C3=C2C1(C)C MOFVSTNWEDAEEK-UHFFFAOYSA-M 0.000 description 7
- 229960004657 indocyanine green Drugs 0.000 description 7
- 238000001356 surgical procedure Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000012800 visualization Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000002093 peripheral effect Effects 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 239000004973 liquid crystal related substance Substances 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 238000012706 support-vector machine Methods 0.000 description 3
- 229920001621 AMOLED Polymers 0.000 description 2
- 238000002679 ablation Methods 0.000 description 2
- 230000001427 coherent effect Effects 0.000 description 2
- 238000004883 computer application Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000001802 infusion Methods 0.000 description 2
- 238000012417 linear regression Methods 0.000 description 2
- 238000007477 logistic regression Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 210000000056 organ Anatomy 0.000 description 2
- APTZNLHMIGJTEW-UHFFFAOYSA-N pyraflufen-ethyl Chemical compound C1=C(Cl)C(OCC(=O)OCC)=CC(C=2C(=C(OC(F)F)N(C)N=2)Cl)=C1F APTZNLHMIGJTEW-UHFFFAOYSA-N 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009141 biological interaction Effects 0.000 description 1
- 230000017531 blood circulation Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 210000002808 connective tissue Anatomy 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 210000000981 epithelium Anatomy 0.000 description 1
- 239000007850 fluorescent dye Substances 0.000 description 1
- 238000009802 hysterectomy Methods 0.000 description 1
- 238000010348 incorporation Methods 0.000 description 1
- 238000012880 independent component analysis Methods 0.000 description 1
- 230000001939 inductive effect Effects 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003387 muscular Effects 0.000 description 1
- 229920001690 polydopamine Polymers 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 238000009418 renovation Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000010454 slate Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 239000010409 thin film Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0062—Arrangements for scanning
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0062—Arrangements for scanning
- A61B5/0068—Confocal scanning
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
- A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
- A61B1/000096—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope using artificial intelligence
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/313—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for introducing through surgical openings, e.g. laparoscopes
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0082—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence adapted for particular medical purposes
- A61B5/0084—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence adapted for particular medical purposes for introduction into the body, e.g. by catheters
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/026—Measuring blood flow
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/026—Measuring blood flow
- A61B5/0261—Measuring blood flow using optical means, e.g. infrared light
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
- A61B5/7207—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal of noise induced by motion artifacts
- A61B5/721—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal of noise induced by motion artifacts using a separate sensor to detect motion or using motion information derived from signals other than the physiological signal to be measured
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/48—Laser speckle optics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/269—Analysis of motion using gradient-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
- G06T2207/10021—Stereoscopic video; Stereoscopic image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20076—Probabilistic image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30101—Blood vessel; Artery; Vein; Vascular
- G06T2207/30104—Vascular flow; Blood flow; Perfusion
Definitions
- a computer-implemented method for determining an estimated force applied on a target tissue region comprising: obtaining a set of images of the target tissue region; determining a perfusion property, a set of spatial measurement, or both of the target tissue region based at least on the set of images; determining a deformation of the target tissue region based at least on the set of spatial measurements; determining a viscoelastic property of the target tissue region based at least on the deformation of the target tissue region, the perfusion property of the target tissue region, or both; and determining the estimated force applied on the target tissue region based at least on the viscoelastic property of the target tissue region.
- the set of images comprises a laser speckle image, an RGB image, an RGB-Depth image, or any combination thereof.
- the laser speckle image is a subjective laser speckle image, an objective laser speckle image, a near-field laser speckle image, or any combination thereof.
- the set of images is obtained while emitting two or more different wavelengths of light at the target tissue region. [0005] In some embodiments, the set of images is obtained while emitting light at the target tissue region having a number of different wavelengths of about 10 to about 1,000.
- the set of images is obtained while emitting light at the target tissue region having a number of different wavelengths of about 10 to about 50, about 10 to about 100, about 10 to about 200, about 10 to about 300, about 10 to about 400, about 10 to about 500, about 10 to about 600, about 10 to about 700, about 10 to about 800, about 10 to about 900, about 10 to about 1,000, about 50 to about 100, about 50 to about 200, about 50 to about 300, about 50 to about 400, about 50 to about 500, about 50 to about 600, about 50 to about 700, about 50 to about 800, about 50 to about 900, about 50 to about 1,000, about 100 to about 200, about 100 to about 300, about 100 to about 400, about 100 to about 500, about 100 to about 600, about 100 to about 700, about 100 to about 800, about 100 to about 900, about 100 to about 1,000, about 200 to about 300, about 200 to about 400, about 200 to about 500, about 200 to about 600, about 200 to about 700, about 200 to about 800, about 200 to about 900, about 100 to about
- the set of images is obtained while emitting light at the target tissue region having a number of different wavelengths of about 10, about 50, about 100, about 200, about 300, about 400, about 500, about 600, about 700, about 800, about 900, or about 1,000. In some embodiments, the set of images is obtained while emitting light at the target tissue region having a number of different wavelengths of at least about 10, about 50, about 100, about 200, about 300, about 400, about 500, about 600, about 700, about 800, or about 900.
- the set of images is obtained while emitting light at the target tissue region having a number of different wavelengths of at most about 50, about 100, about 200, about 300, about 400, about 500, about 600, about 700, about 800, about 900, or about 1,000.
- the set of images of the target issue region and the set spatial measurements of the target tissue region are obtained simultaneously in real time as the target issue region undergoes the deformation.
- the set of images of the target issue region is obtained in-vitro.
- the set of images of the target issue region is obtained in-vivo.
- at least one of the set of images of the target issue region is obtained while the target tissue region undergoes a known deformation by a pre determined force.
- the target tissue region is a soft tissue region.
- determining the mechanical property, the viscoelastic property, or both of the target tissue region is performed by a machine learning algorithm.
- the viscoelastic property comprises a viscous property, an elastic property, a fluid mechanics property, or any combination thereof.
- the method further comprises obtaining depth measurements from a depth sensor, and wherein the deformation of the target tissue region is further based on the depth measurements.
- the spatial measurements are one-dimensional, two-dimensional, or three-dimensional.
- the depth sensor comprises a stereo camera, a video camera, a time of flight sensor, or any combination thereof.
- the deformation of the target tissue region comprises a one-dimensional deformation, a two-dimensional deformation, a three- dimensional deformation, or any combination thereof.
- determining the estimated force applied to the target tissue region is performed by a machine learning algorithm.
- the force is applied by a human operator, and wherein the method further comprises providing a feedback to the operator based on the determined estimated force applied on the target tissue region.
- the feedback comprises a visual feedback, an auditory feedback, a haptic feedback, or any combination thereof.
- the visual feedback comprises a color coded visual feedback, a displayed value, a map, or any combination thereof corresponding to the estimated force.
- a relationship between the estimated force and the feedback is linear, non-linear, or exponential.
- the force is applied by an autonomous or semi-autonomous device
- the method further comprises providing a control feedback to the autonomous or semi- autonomous device based on the force applied by the deformed tissue.
- the autonomous or semi-autonomous device alters its treatment based on the control feedback.
- the method further comprises determining a fluid flow rate within the target tissue based at least on (i) the set of images, (ii) the spatial measurements, (iii) the viscoelastic property of the target tissue region, (iv) the deformation of the target tissue region, or any combination thereof.
- the fluid is blood, sweat, semen, saliva, pus, urine, air, mucus, milk, bile, a hormone, or any combination thereof.
- the fluid flow rate within the target tissue is determined by a machine learning algorithm. In some embodiments, the fluid flow rate is determined by a machine learning algorithm. In some embodiments, the method further comprises determining an identification of the target tissue based at least on (i) the set of images, (ii) the spatial measurements, (iii) the viscoelastic property of the target tissue region, (iv) the deformation of the target tissue region, or any combination thereof. In some embodiments, the identification of the target tissue is determined by a machine learning algorithm. In some embodiments, the identification of the target tissue is an identification that the target tissue is cancerous, benign, malignant, or healthy.
- a computer-implemented system comprising: a digital processing device comprising: at least one processor, an operating system configured to perform executable instructions, a memory, and a computer program including instructions executable by the digital processing device to create an application for determining an estimated force applied on a target tissue region, the application comprising: a module obtaining a set of images of the target tissue region; a module determining a perfusion property, a set of spatial measurement, or both of the target tissue region based at least on the set of images; a module determining a deformation of the target tissue region based at least on the set of spatial measurements; a module determining a viscoelastic property of the target tissue region based at least on the deformation of the target tissue region, the perfusion property of the target tissue region, or both; and a module determining the estimated force applied on the target tissue region based at least on the viscoelastic property of the target tissue region.
- the set of images comprises a laser speckle image, an RGB image, an RGB-Depth image, or any combination thereof.
- the laser speckle image is a subjective laser speckle image, an objective laser speckle image, a near-field laser speckle image, or any combination thereof.
- the set of images is obtained while emitting two or more different wavelengths of light at the target tissue region. In some embodiments, the set of images is obtained while emitting about 10 to about 1,000 different wavelengths of light at the target tissue region. In some embodiments, the set of images of the target issue region and the set spatial measurements of the target tissue region are obtained simultaneously in real time as the target issue region undergoes the deformation.
- the set of images of the target issue region is obtained in-vitro. In some embodiments, the set of images of the target issue region is obtained in-vivo. In some embodiments, at least one of the set of images of the target issue region is obtained while the target tissue region undergoes a known deformation by a pre-determined force. In some embodiments, the target tissue region is a soft tissue region. In some embodiments, determining the mechanical property, the viscoelastic property, or both of the target tissue region is performed by a machine learning algorithm. In some embodiments, the viscoelastic property comprises a viscous property, an elastic property, a fluid mechanics property, or any combination thereof.
- the application further comprises a module obtaining depth measurements from a depth sensor, and wherein the deformation of the target tissue region is further based on the depth measurements.
- the spatial measurements are one-dimensional, two-dimensional, or three-dimensional.
- the depth sensor comprises a stereo camera, a video camera, a time of flight sensor, or any combination thereof.
- the deformation of the target tissue region comprises a one-dimensional deformation, a two-dimensional deformation, a three- dimensional deformation, or any combination thereof.
- determining the estimated force applied to the target tissue region is performed by a machine learning algorithm.
- the force is applied by a human operator, and wherein the application further comprises a module providing a feedback to the operator based on the determined estimated force applied on the target tissue region.
- the feedback comprises a visual feedback, an auditory feedback, a haptic feedback, or any combination thereof.
- the visual feedback comprises a color coded visual feedback, a displayed value, a map, or any combination thereof corresponding to the estimated force.
- a relationship between the estimated force and the feedback is linear, non linear, or exponential.
- the force is applied by an autonomous or semi- autonomous device, and wherein the application further comprises a module providing a control feedback to the autonomous or semi-autonomous device based on the force applied by the deformed tissue.
- the autonomous or semi-autonomous device alters its treatment based on the control feedback.
- the application further comprising a module determining a fluid flow rate within the target tissue based at least on (i) the set of images, (ii) the spatial measurements, (iii) the viscoelastic property of the target tissue region, (iv) the deformation of the target tissue region, or any combination thereof.
- the fluid is blood, sweat, semen, saliva, pus, urine, air, mucus, milk, bile, a hormone, or any combination thereof.
- the fluid flow rate within the target tissue is determined by a machine learning algorithm. In some embodiments, the fluid flow rate is determined by a machine learning algorithm.
- the application further comprising a module determining an identification of the target tissue based at least on (i) the set of images, (ii) the spatial measurements, (iii) the viscoelastic property of the target tissue region, (iv) the deformation of the target tissue region, or any combination thereof.
- the identification of the target tissue is determined by a machine learning algorithm.
- the identification of the target tissue is an identification that the target tissue is cancerous, benign, malignant, or healthy.
- Another aspect provided herein is a non-transitory computer-readable storage media encoded with a computer program including instructions executable by a processor to create an application for determining an estimated force applied on a target tissue region, the application comprising: a module obtaining a set of images of the target tissue region; a module determining a perfusion property, a set of spatial measurement, or both of the target tissue region based at least on the set of images; a module determining a deformation of the target tissue region based at least on the set of spatial measurements; a module determining a viscoelastic property of the target tissue region based at least on the deformation of the target tissue region, the perfusion property of the target tissue region, or both; and a module determining the estimated force applied on the target tissue region based at least on the viscoelastic property of the target tissue region.
- the set of images comprises a laser speckle image, an RGB image, an RGB-Depth image, or any combination thereof.
- the laser speckle image is a subjective laser speckle image, an objective laser speckle image, a near-field laser speckle image, or any combination thereof.
- the set of images is obtained while emitting two or more different wavelengths of light at the target tissue region. In some embodiments, the set of images is obtained while emitting about 10 to about 1,000 different wavelengths of light at the target tissue region. In some embodiments, the set of images of the target issue region and the set spatial measurements of the target tissue region are obtained simultaneously in real time as the target issue region undergoes the deformation.
- the set of images of the target issue region is obtained in-vitro. In some embodiments, the set of images of the target issue region is obtained in-vivo. In some embodiments, at least one of the set of images of the target issue region is obtained while the target tissue region undergoes a known deformation by a pre-determined force. In some embodiments, the target tissue region is a soft tissue region. In some embodiments, determining the mechanical property, the viscoelastic property, or both of the target tissue region is performed by a machine learning algorithm. In some embodiments, the viscoelastic property comprises a viscous property, an elastic property, a fluid mechanics property, or any combination thereof.
- the application further comprises a module obtaining depth measurements from a depth sensor, and wherein the deformation of the target tissue region is further based on the depth measurements.
- the spatial measurements are one-dimensional, two-dimensional, or three-dimensional.
- the depth sensor comprises a stereo camera, a video camera, a time of flight sensor, or any combination thereof.
- the deformation of the target tissue region comprises a one-dimensional deformation, a two-dimensional deformation, a three- dimensional deformation, or any combination thereof.
- determining the estimated force applied to the target tissue region is performed by a machine learning algorithm.
- the force is applied by a human operator, and wherein the application further comprises a module providing a feedback to the operator based on the determined estimated force applied on the target tissue region.
- the feedback comprises a visual feedback, an auditory feedback, a haptic feedback, or any combination thereof.
- the visual feedback comprises a color coded visual feedback, a displayed value, a map, or any combination thereof corresponding to the estimated force.
- a relationship between the estimated force and the feedback is linear, non linear, or exponential.
- the force is applied by an autonomous or semi- autonomous device, and wherein the application further comprises a module providing a control feedback to the autonomous or semi-autonomous device based on the force applied by the deformed tissue.
- the autonomous or semi-autonomous device alters its treatment based on the control feedback.
- the application further comprises a module determining a fluid flow rate within the target tissue based at least on (i) the set of images, (ii) the spatial measurements, (iii) the viscoelastic property of the target tissue region, (iv) the deformation of the target tissue region, or any combination thereof.
- the fluid is blood, sweat, semen, saliva, pus, urine, air, mucus, milk, bile, a hormone, or any combination thereof.
- the fluid flow rate within the target tissue is determined by a machine learning algorithm. In some embodiments, the fluid flow rate is determined by a machine learning algorithm.
- the application further comprising a module determining an identification of the target tissue based at least on (i) the set of images, (ii) the spatial measurements, (iii) the viscoelastic property of the target tissue region, (iv) the deformation of the target tissue region, or any combination thereof.
- the identification of the target tissue is determined by a machine learning algorithm.
- the identification of the target tissue is an identification that the target tissue is cancerous, benign, malignant, or healthy.
- Another aspect provided herein is a computer-implemented method for training a neural network to determine an elastic property of a target issue region, the method comprising: generating a first training set comprising a plurality of sets of set of images, wherein each set of images comprises a first speckle image of the target issue region at rest and a second speckle image of the target issue region being deformed by a known force; training the neural network in a first stage using the first training set; generating a second training set comprising the first training set and the sets of set of images whose elastic property value was incorrectly determined after the first stage of training; and training the neural network in a second stage using the second training set.
- the set of images comprises a subjective set of images, an objective set of images, a near-field set of images, or any combination thereof. In some embodiments, the set of images is obtained while emitting at least 10 different wavelengths of light at the target tissue region. In some embodiments, the set of images is obtained while emitting about 10 to about 1,000 different wavelengths of light at the target tissue region.
- the viscoelastic property comprises a viscous property, an elastic property, a fluid mechanics property, or any combination thereof. In some embodiments, the spatial measurements are one-dimensional, two-dimensional, or three-dimensional.
- Another aspect of the present disclosure provides a non-transitory computer readable medium comprising machine executable code that, upon execution by one or more computer processors, implements any of the methods above or elsewhere herein.
- Another aspect of the present disclosure provides a system comprising one or more computer processors and computer memory coupled thereto.
- the computer memory comprises machine executable code that, upon execution by the one or more computer processors, implements any of the methods above or elsewhere herein.
- FIG. 1 shows a schematic diagram of a method for determining an estimated force, per an embodiment herein;
- FIG. 2 shows a schematic diagram of a method for training a neural network to determine a viscoelastic property of a target issue region, per an embodiment herein;
- FIG. 3 shows a schematic diagram of various light frequencies, per an embodiment herein;
- FIG. 4 shows a schematic diagram of a machine learning algorithm to determine a viscoelastic property of a target issue region, per an embodiment herein;
- FIG. 5A shows an image of a device for obtaining a set of images of the target tissue region, per an embodiment herein;
- FIG. 5B shows an image of a device with a laparoscope for obtaining a set of images of the target tissue region, per an embodiment herein;
- FIG. 6 shows an image of a connectivity device for transferring the set of images of the target tissue region, per an embodiment herein;
- FIG. 7 shows an image of a system for collecting and transferring the set of images of the target tissue region, per an embodiment herein;
- FIG. 8A shows an image of a sample tissue region
- FIG. 8B shows an image of a sample tissue region injected with an
- FIG. 9A shows another image of a target tissue region, per an embodiment herein;
- FIG. 9B shows an image of the perfusion within the target tissue region, per an embodiment herein;
- FIG. 9C shows an image of the target tissue region overlaid with the image of the perfusion within the target tissue region, per an embodiment herein;
- FIG. 10A shows an image of an unablated target tissue region injected with the ICG dye
- FIG. 10B shows an image of an unablated target tissue region injected overlaid with the determined perfusion property, per an embodiment herein;
- FIG. IOC shows an image of an ablated target tissue region injected with the ICG dye
- FIG. 10D shows an image of an ablated target tissue region injected overlaid with the determined perfusion property, per an embodiment herein;
- FIG. 11 shows an exemplary setup to capture a speckle image of a target tissue region undergoing a known deformation by a pre-determined force, per an embodiment herein; and
- FIG. 12 shows a non-limiting example of a computing device; in this case, a device with one or more processors, memory, storage, and a network interface, per an embodiment herein.
- the method comprises: obtaining a set of images of the target tissue region 101; determining a perfusion property, a set of spatial measurement, or both of the target tissue region 102; determining a deformation of the target tissue region 103; determining a viscoelastic property of the target tissue region 104; and determining the estimated force applied on the target tissue region 105.
- the estimated force applied on the target tissue region is determined based at least on the viscoelastic property of the target tissue region.
- the target tissue is a soft tissue.
- the target tissue is an epithelial tissue, connective tissue, muscular tissue, nervous tissue, or any combination thereof.
- the target tissue region is a treatment region receiving treatment by a caregiver.
- the target tissue region has an area of about 2 mm 2 , 5 mm 2 , 10 mm 2 , 20 mm 2 , 50 mm 2 , 100 mm 2 , 200 mm 2 , 500 mm 2 , 1,000 mm 2 , 10,000 mm 2 , 100,000 mm 2 , 1,000,000 mm 2 , or more including increments therein.
- the target tissue is in-vitro. In some embodiments, the target tissue is in-vivo.
- FIGS. 8A and 8B Current methods of determining perfusion in a target tissue, per FIGS. 8A and 8B, typically require the infusion of a fluorescent dye (e.g. an indocyanine green (ICG) dye) into a patient. While key perfusion structures are visible in FIG. 8B, such infusions have several shortcomings. First as the dye requires about 5 minutes to about 24 hours to reach the target tissue, such a procedure must be planned before a surgery of the target tissue, and/or delay the visualization effects. Any additional planning and treatment steps that could go awry should be avoided to ensure a successful surgery. Such a large dye visualization variation among patients further encumbers its use. Further, as clinicians are charged per dosage of the dye, mistimed or untimely injections are costly. Second, the visualization capabilities of the dye dissipate as it flows through the bloodstream, leaving a very narrow opportunity of use. Finally, such dyes are not indicated for all patients based on their biologic interactions.
- a fluorescent dye e.g. an in
- the methods, systems, and media herein do not require the use of a dye or other injected visualization medium. Further the methods, systems, and media herein require little to no planning for use, can be used instantly without any waiting periods, and can be used continually throughout a surgery without inducing extra costs or procedures.
- FIGS. 10A-D the systems, methods, and media herein are more capable at determining areas of perfusion property than the currently available ICG dies.
- visualizations of an unablated target tissue with the ICG dye, per FIG. 10A, and via the instant methods, systems, and media, per FIG. 10B show the same areas of reduced perfusion 100A and 100B, reduced perfusion area lOOC of tissue visualized with the ICG dye, per FIG. IOC, is incapable of detecting areas of reduced perfusion induced by ablation.
- FIG. 10B shows the same areas of reduced perfusion 100A and 100B
- reduced perfusion area lOOC of tissue visualized with the ICG dye, per FIG. IOC is incapable of detecting areas of reduced perfusion induced by ablation.
- the methods, systems, and media herein are capable of detecting areas of reduced perfusion induced by ablation 110 in addition to the remaining areas of reduced perfusion HOD.
- the perfusion property of the target tissue region is determined based at least on the set of images. In some embodiments, the perfusion measures a rate at which a fluid is delivered to tissue, or volume of the fluid per unit time per unit tissue mass in m 3 /(s kg) or ml/min/g. In some embodiments, the fluid is blood, sweat, semen, saliva, pus, urine, air, mucus, milk, bile, a hormone, or any combination thereof. In some embodiments, the perfusion property is further determined by measurements collected by an oximeter, a pulse rate monitor, or any combination thereof. In some embodiments, the perfusion property is further determined based on predetermined perfusion properties of an organ or tissue. FIGS.
- FIG. 9A shows an exemplary image of a target tissue region.
- FIG. 9B shows an exemplary image of the perfusion of the target tissue region.
- FIG. 9C shows an exemplary image of the target tissue region overlaid with the image of the perfusion of the target tissue region.
- the set of spatial measurements of the target tissue region is determined based at least on the set of images. In some embodiments, the deformation of the target tissue region is determined based at least on the set of spatial measurements. In some embodiments, the images of the target tissue region comprise two-dimensional images of the target tissue region, wherein the set of spatial measurements of the target tissue region is determined based on the two-dimensional images of the target tissue region. In some embodiments, the images of the target tissue region comprise three-dimensional images of the target tissue region, wherein the set of spatial measurements of the target tissue region is determined based on the three-dimensional images of the target tissue region. In some embodiments, the set of spatial measurements of the target tissue region are two-dimensional. In some embodiments, the set of spatial measurements of the target tissue region are two-dimensional, wherein one dimension is normal to the target tissue region. In some embodiments, the set of spatial measurements of the target tissue region are three-dimensional.
- the viscoelastic property of the target tissue region is determined based at least on the deformation of the target tissue region, the perfusion property of the target tissue region, or both.
- the viscoelastic property comprises a viscosity property, an elastic property, a fluid mechanics property, or any combination thereof.
- the viscoelastic property comprises a stiffness.
- the viscosity property correlates to a rate at which the target tissue deforms under force.
- the elastic property correlates to the deformation distance under force.
- the viscosity property is a kinematic viscosity, a dynamic viscosity, or both.
- the fluid mechanics property is a flow resistance, a pulse rate, a fluid pressure, a fluid volume, a fluid temperature, a fluid density, or any combination thereof.
- FIGS. 5A and 5B show images of a device for obtaining a set of images of the target tissue region, without and with a laparoscope, respectively.
- FIG. 6 shows an image of a connectivity device for transferring the set of images of the target tissue region.
- FIG. 7 shows an image of a system for collecting and transferring the set of images of the target tissue region [0046]
- the set of images comprises a laser speckle image, a Red-Green- Blue (RGB) image, an RGB-Depth image, or any combination thereof.
- the set of images comprises a laser speckle video, a Red-Green-Blue (RGB) video, an RGB- Depth video, or any combination thereof.
- the RGB-Depth image comprises an RGB image overlaid with a depth measurement.
- the laser speckle image is a subjective laser speckle image, an objective laser speckle image, a near-field laser speckle image, or any combination thereof.
- a subjective laser speckle image is captured while the sample is directly illuminated the with a coherent light (e.g. a laser beam).
- the subjective laser speckle image depends on the viewing system parameters, such as, for example: the size of the lens aperture, and the position of the imaging system.
- a subjective laser speckle image is captured while the sample is indirectly illuminated the with a coherent light (e.g. a laser beam).
- the laser speckle image is captured by a camera.
- the set of images is obtained while emitting two or more different wavelengths of light at the target tissue region. In some embodiments, the set of images is obtained while emitting about 10 to about 1,000 different wavelengths of light at the target tissue region. In some embodiments, per FIG. 3, the set of images is obtained while emitting a hyperspectral combination of wavelengths 301, a laser wavelength 302, and a near-infrared wavelength 303. In some embodiments, the set of images of the target issue region and the set spatial measurements of the target tissue region are obtained simultaneously in real time. In some embodiments, the set of images of the target issue region and the set spatial measurements of the target tissue region are obtained simultaneously in real time as the target issue region undergoes the deformation.
- the set of images of the target issue region is obtained in-vitro. In some embodiments, the set of images of the target issue region is obtained in-vivo. In some embodiments, at least one of the set of images of the target issue region is obtained while the target tissue region undergoes a known deformation by a pre-determined force. In some embodiments, a first image of the set of images of the target issue region is obtained while the target tissue region undergoes a known deformation by a pre-determined force.
- FIG. 11 shows an exemplary setup to capture a speckle image of the target issue region 1101 while the target tissue region 1101 undergoes a known deformation by a pre-determined force 1103.
- a thread 1102 is attached to the target tissue region 1101 imparting a known pre-determined force 1103 thereon, while a speckle image is captured by an image capturing device 1104.
- the thread 1102 imparts a normal tensile pre determined force 1103 to the target tissue region 1101 via the thread 1102. Additionally or alternatively, the thread 1102 imparts a normal compressive, or a shear pre-determined force 1103 to the target tissue region 1101.
- the set of images are all captured with the same orientation between the image capturing device and the target tissue. In some embodiments, at least a portion of the set of images are all captured with the same orientation between the image capturing device and the target tissue.
- the method further comprises obtaining depth measurements from a depth sensor.
- the depth sensor is a stereo triangulation sensor, a structured light sensor, a video camera, a time of flight sensor, an interferometer, a coded aperture, or any combination thereof.
- the deformation of the target tissue region is further based on the depth measurements.
- the spatial measurements are one-dimensional, two-dimensional, or three-dimensional.
- the deformation of the target tissue region comprises a one-dimensional deformation, a two-dimensional deformation, a three-dimensional deformation, or any combination thereof.
- the force is applied by a human operator.
- the method further comprises providing a feedback to the operator.
- the method further comprises providing a feedback to the operator based on the determined estimated force applied on the target tissue region.
- the feedback comprises a visual feedback, an auditory feedback, a haptic feedback, or any combination thereof.
- the visual feedback comprises a color coded visual feedback, a displayed value, a map, or any combination thereof corresponding to the estimated force.
- a relationship between the estimated force and the feedback is linear, non linear, or exponential.
- the force is applied by an autonomous or semi-autonomous device.
- the method further comprises providing a control feedback to the autonomous or semi-autonomous device based on the force applied by the deformed tissue.
- the autonomous or semi-autonomous device alters its treatment based on the control feedback.
- the method further comprises determining a fluid flow rate within the target tissue.
- the flow rate is based at least on (i) the set of images, (ii) the spatial measurements, (iii) the viscoelastic property of the target tissue region, (iv) the deformation of the target tissue region, or any combination thereof.
- the fluid is blood, sweat, semen, saliva, pus, urine, air, mucus, milk, bile, a hormone, or any combination thereof.
- the fluid flow rate within the target tissue is determined by a machine learning algorithm. In some embodiments, the fluid flow rate is determined by a machine learning algorithm.
- the method further comprises determining an identification of the target tissue based at least on (i) the set of images, (ii) the spatial measurements, (iii) the viscoelastic property of the target tissue region, (iv) the deformation of the target tissue region, or any combination thereof.
- the identification of the target tissue is determined by a machine learning algorithm.
- the identification of the target tissue is an identification that the target tissue is cancerous, benign, malignant, or healthy. Machine Learning
- determining the mechanical property, the viscoelastic property, or both of the target tissue region is performed by a machine learning algorithm. In some embodiments, determining the estimated force applied to the target tissue region is performed by a machine learning algorithm. In some embodiments, the machine learning algorithm employs a neural network.
- Examples of the machine learning algorithms that can be used with the embodiments herein may comprise a regression-based learning algorithm, linear or non-linear algorithms, feed-forward neural network, generative adversarial network (GAN), or deep residual networks.
- the machine learning algorithm may include, for example, an unsupervised learning classifier, a supervised learning classifier, or a combination thereof.
- An unsupervised learning classifier may include, for example, clustering, hierarchical clustering, k-means, mixture models, DBSCAN, OPTICS algorithm, anomaly detection, local outlier factor, neural networks, autoencoders, deep belief nets, hebbian learning, generative adversarial networks, self-organizing map, expectation- maximization algorithm (EM), method of moments, blind signal separation techniques, principal component analysis, independent component analysis, non-negative matrix factorization, singular value decomposition, or a combination thereof.
- a supervised learning classifier may include, for example, support vector machines, linear regression, logistic regression, linear discriminant analysis, decision trees, k-nearest neighbor algorithm, neural networks, similarity learning, or a combination thereof.
- the machine learning algorithm may comprise a deep learning neural network.
- the deep learning neural network may comprise a convolutional neural network (CNN).
- the CNN may include, for example, U-Net, ImageNet, LeNet-5, AlexNet, ZFNet, GoogleNet, VGGNet, ResNetl8 orResNet, etc.
- FIG. 4 shows an exemplary schematic flowchart of a machine learning algorithm for determining the estimated force applied to the target tissue region.
- the exemplary algorithm comprises: receiving a first input speckle (xO) 401A and a second input speckle (xt) 401B; determining a hidden abstract representation of the first input speckle (hO) 403A and second input speckle (h_t) 403B via an encoder 402, comparing the abstract representations of the first (hO) and second input speckles (h_t) 404; and determining an output force 405.
- At least one of the first input speckle (hO) 403A and the second input speckle (h_t) 403B are captured while a predetermined force is applied to the target tissue region.
- a predetermined force is applied to the target tissue region.
- the machine learning algorithm is a supervised machine learning algorithm.
- the machine learning algorithms utilized therein employ one or more forms of labels including but not limited to human annotated labels and semi-supervised labels.
- the human annotated labels can be provided by a hand-crafted heuristic.
- the hand-crafted heuristic can comprise examining differences between images of the target tissue region, spatial measurements, or both.
- the semi-supervised labels can be determined using a clustering technique to find images of the target tissue region, spatial measurements, or both similar to those flagged by previous human annotated labels and previous semi-supervised labels.
- the semi-supervised labels can employ a XGBoost, a neural network, or both.
- the distant supervision method can create a large training set seeded by a small hand- annotated training set.
- the distant supervision method can comprise positive-unlabeled learning with the training set as the ‘positive’ class.
- the distant supervision method can employ a logistic regression model, a recurrent neural network, or both.
- the recurrent neural network can be advantageous for Natural Language Processing (NLP) machine learning.
- NLP Natural Language Processing
- Examples of machine learning algorithms can include a support vector machine (SVM), a naive Bayes classification, a random forest, a neural network, deep learning, or other supervised learning algorithm or unsupervised learning algorithm for classification and regression.
- SVM support vector machine
- the machine learning algorithms can be trained using one or more training datasets.
- the machine learning algorithm utilizes regression modeling, wherein relationships between predictor variables and dependent variables are determined and weighted.
- the viscoelastic property can be a dependent variable and is derived from the images of the target tissue region, spatial measurements, or both.
- a machine learning algorithm is used to select catalogue images and recommend project scope.
- any number of Ai and Xi variable can be included in the model.
- Xi is the number of images
- X 2 is the number of spatial measurement
- X 3 is the viscoelastic property of the target tissue region.
- the programming language “R” is used to run the model.
- training comprises multiple steps. In a first step, an initial model is constructed by assigning probability weights to predictor variables. In a second step, the initial model is used to “recommend” the viscoelastic property of the target tissue region.
- the validation module accepts verified data regarding the viscoelastic property of the target tissue region and feeds back the verified data to the renovation probability calculation.
- At least one of the first step, the second step, and the third step can repeat one or more times continuously or at set intervals.
- Another aspect provided herein is a computer-implemented method for training a neural network to determine an elastic property of a target issue region.
- the method comprises: generating a first training set 201; training the neural network in a first stage using the first training set 202; generating a second training set 203; and training the neural network in a second stage using the second training set 204.
- the first training set comprising a plurality of sets of set of images.
- each set of images comprises a first speckle image of the target issue region at rest and a second speckle image of the target issue region.
- the second speckle image is captured while the target issue region is being deformed.
- the second speckle image is captured while the target issue region is being deformed by a known force.
- the second training set comprising the first training set and the sets of set of images whose elastic property value was incorrectly determined after the first stage of training.
- the set of images comprises a subjective set of images, an objective set of images, a near-field set of images, or any combination thereof.
- the set of images is obtained while emitting at least 10 different wavelengths of light at the target tissue region.
- the set of images is obtained while emitting about 10 to about 1,000 different wavelengths of light at the target tissue region.
- the viscoelastic property comprises a viscous property, an elastic property, a fluid mechanics property, or any combination thereof.
- the spatial measurements are one-dimensional, two-dimensional, or three-dimensional.
- the present disclosure provides a method of tracking tissue deformations.
- the method may comprise: (a) obtaining a scalar optical flow reading, wherein the scalar optical flow reading corresponds to one or more laser speckle signals; (b) using said scalar optical flow reading to determine a pixel-wise motion magnitude estimate for a tissue region; and (c) integrating said pixel -wise motion magnitude estimate over time and space to track a deformation of the tissue region.
- the one or more laser speckle signals may be associated with, based on, and/or derived from the deformation of the tissue region.
- the one or more laser speckle signals may be obtained during a deformation of the tissue region.
- the pixel-wise motion magnitude estimate may comprise a directionless motion estimate.
- the method may further comprise combining (i) the pixel-wise motion estimate with (ii) depth and/or RGB-D data of the tissue region to generate a pixel-wise displacement map.
- the pixel-wise displacement map may comprise a visual or data-based representation of a deformation of a tissue region at one or more pixels (or per pixel of an image of the tissue region).
- the term “about” refers to an amount that is near the stated amount by 10%, 5%, or 1%, including increments therein.
- the term “about” in reference to a percentage refers to an amount that is greater or less the stated percentage by 10%, 5%, or 1%, including increments therein.
- each of the expressions “at least one of A, B and C”, “at least one of A, B, or C”, “one or more of A, B, and C”, “one or more of A, B, or C” and “A, B, and/or C” means A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B and C together.
- perfusion refers to is a measurement of the passage of fluid through an organ or a tissue. In some embodiments, perfusion is measured as the rate at which blood is delivered to tissue, or volume of blood per unit time (blood flow) per unit tissue mass. In some embodiments, perfusion is measured in m 3 /(s kg) or ml/min/g.
- the term “speckle image” refers to a pattern is produced by the mutual interference of a set of incoherent waves.
- the waves have the same frequency, having different phases and amplitudes, which add together to give a resultant wave whose amplitude varies randomly.
- FIG. 12 a block diagram is shown depicting an exemplary machine that includes a computer system 1200 (e.g., a processing or computing system) within which a set of instructions can execute for causing a device to perform or execute any one or more of the aspects and/or methodologies for static code scheduling of the present disclosure.
- a computer system 1200 e.g., a processing or computing system
- the components in FIG. 12 are examples only and do not limit the scope of use or functionality of any hardware, software, embedded logic component, or a combination of two or more such components implementing particular embodiments.
- Computer system 1200 may include one or more processors 1201, a memory 1203, and a storage 1208 that communicate with each other, and with other components, via a bus 1240.
- the bus 1240 may also link a display 1232, one or more input devices 1233 (which may, for example, include a keypad, a keyboard, a mouse, a stylus, etc.), one or more output devices 1234, one or more storage devices 1235, and various tangible storage media 1236. All of these elements may interface directly or via one or more interfaces or adaptors to the bus 1240.
- the various tangible storage media 1236 can interface with the bus 1240 via storage medium interface 1226.
- Computer system 1200 may have any suitable physical form, including but not limited to one or more integrated circuits (ICs), printed circuit boards (PCBs), mobile handheld devices (such as mobile telephones or PDAs), laptop or notebook computers, distributed computer systems, computing grids, or servers.
- ICs integrated circuits
- PCBs printed circuit boards
- mobile handheld devices such as mobile telephones or PDAs
- laptop or notebook computers distributed computer systems, computing grids, or servers.
- Computer system 1200 includes one or more processor(s) 1201 (e.g., central processing units (CPUs) or general purpose graphics processing units (GPGPUs)) that carry out functions.
- processor(s) 1201 optionally contains a cache memory unit 1202 for temporary local storage of instructions, data, or computer addresses.
- Processor(s) 1201 are configured to assist in execution of computer readable instructions.
- Computer system 1200 may provide functionality for the components depicted in FIG. 12 as a result of the processor(s) 1201 executing non-transitory, processor-executable instructions embodied in one or more tangible computer-readable storage media, such as memory 1203, storage 1208, storage devices 1235, and/or storage medium 1236.
- the computer-readable media may store software that implements particular embodiments, and processor(s) 1201 may execute the software.
- Memory 1203 may read the software from one or more other computer-readable media (such as mass storage device(s) 1235, 1236) or from one or more other sources through a suitable interface, such as network interface 1220.
- the software may cause processor(s) 1201 to carry out one or more processes or one or more steps of one or more processes described or illustrated herein. Carrying out such processes or steps may include defining data structures stored in memory 1203 and modifying the data structures as directed by the software.
- the memory 1203 may include various components (e.g., machine readable media) including, but not limited to, a random access memory component (e.g., RAM 1204) (e.g., static RAM (SRAM), dynamic RAM (DRAM), ferroelectric random access memory (FRAM), phase- change random access memory (PRAM), etc.), a read-only memory component (e.g., ROM 1205), and any combinations thereof.
- ROM 1205 may act to communicate data and instructions unidirectionally to processor(s) 1201, and RAM 1204 may act to communicate data and instructions bidirectionally with processor(s) 1201.
- ROM 1205 and RAM 1204 may include any suitable tangible computer-readable media described below.
- a basic input/output system 1206 (BIOS), including basic routines that help to transfer information between elements within computer system 1200, such as during start-up, may be stored in the memory 1203.
- BIOS basic input/output system 1206
- Fixed storage 1208 is connected bidirectionally to processor(s) 1201, optionally through storage control unit 1207.
- Fixed storage 1208 provides additional data storage capacity and may also include any suitable tangible computer-readable media described herein.
- Storage 1208 may be used to store operating system 1209, executable(s) 1210, data 1211, applications 1212 (application programs), and the like.
- Storage 1208 can also include an optical disk drive, a solid- state memory device (e.g., flash-based systems), or a combination of any of the above.
- Information in storage 1208 may, in appropriate cases, be incorporated as virtual memory in memory 1203.
- storage device(s) 1235 may be removably interfaced with computer system 1200 (e.g., via an external port connector (not shown)) via a storage device interface 1225.
- storage device(s) 1235 and an associated machine-readable medium may provide non-volatile and/or volatile storage of machine-readable instructions, data structures, program modules, and/or other data for the computer system 1200.
- software may reside, completely or partially, within a machine-readable medium on storage device(s) 1235.
- software may reside, completely or partially, within processor(s)
- Bus 1240 connects a wide variety of subsystems.
- reference to a bus may encompass one or more digital signal lines serving a common function, where appropriate.
- Bus 1240 may be any of several types of bus structures including, but not limited to, a memory bus, a memory controller, a peripheral bus, a local bus, and any combinations thereof, using any of a variety of bus architectures.
- such architectures include an Industry Standard Architecture (ISA) bus, an Enhanced ISA (EISA) bus, a Micro Channel Architecture (MCA) bus, a Video Electronics Standards Association local bus (VLB), a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, an Accelerated Graphics Port (AGP) bus, HyperTransport (HTX) bus, serial advanced technology attachment (SATA) bus, and any combinations thereof.
- ISA Industry Standard Architecture
- EISA Enhanced ISA
- MCA Micro Channel Architecture
- VLB Video Electronics Standards Association local bus
- PCI Peripheral Component Interconnect
- PCI-X PCI-Express
- AGP Accelerated Graphics Port
- HTTP HyperTransport
- SATA serial advanced technology attachment
- Computer system 1200 may also include an input device 1233.
- a user of computer system 1200 may enter commands and/or other information into computer system 1200 via input device(s) 1233.
- Examples of an input device(s) 1233 include, but are not limited to, an alpha-numeric input device (e.g., a keyboard), a pointing device (e.g., a mouse or touchpad), a touchpad, a touch screen, a multi -touch screen, a joystick, a stylus, a gamepad, an audio input device (e.g., a microphone, a voice response system, etc.), an optical scanner, a video or still image capture device (e.g., a camera), and any combinations thereof.
- an alpha-numeric input device e.g., a keyboard
- a pointing device e.g., a mouse or touchpad
- a touchpad e.g., a touch screen
- a multi -touch screen e.g.
- the input device is a Kinect, Leap Motion, or the like.
- Input device(s) 1233 may be interfaced to bus 1240 via any of a variety of input interfaces 1223 (e.g., input interface 1223) including, but not limited to, serial, parallel, game port, USB, FIREWIRE, THUNDERBOLT, or any combination of the above.
- computer system 1200 when computer system 1200 is connected to network 1230, computer system 1200 may communicate with other devices, specifically mobile devices and enterprise systems, distributed computing systems, cloud storage systems, cloud computing systems, and the like, connected to network 1230. Communications to and from computer system 1200 may be sent through network interface 1220.
- network interface 1220 may receive incoming communications (such as requests or responses from other devices) in the form of one or more packets (such as Internet Protocol (IP) packets) from network 1230, and computer system 1200 may store the incoming communications in memory 1203 for processing.
- Computer system 1200 may similarly store outgoing communications (such as requests or responses to other devices) in the form of one or more packets in memory 1203 and communicated to network 1230 from network interface 1220.
- Processor(s) 1201 may access these communication packets stored in memory 1203 for processing.
- Examples of the network interface 1220 include, but are not limited to, a network interface card, a modem, and any combination thereof.
- Examples of a network 1230 or network segment 1230 include, but are not limited to, a distributed computing system, a cloud computing system, a wide area network (WAN) (e.g., the Internet, an enterprise network), a local area network (LAN) (e.g., a network associated with an office, a building, a campus or other relatively small geographic space), a telephone network, a direct connection between two computing devices, a peer-to-peer network, and any combinations thereof.
- a network, such as network 1230 may employ a wired and/or a wireless mode of communication. In general, any network topology may be used.
- Information and data can be displayed through a display 1232.
- a display 1232 include, but are not limited to, a cathode ray tube (CRT), a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT-LCD), an organic liquid crystal display (OLED) such as a passive-matrix OLED (PMOLED) or active-matrix OLED (AMOLED) display, a plasma display, and any combinations thereof.
- the display 1232 can interface to the processor(s) 1201, memory 1203, and fixed storage 1208, as well as other devices, such as input device(s) 1233, via the bus 1240.
- the display 1232 is linked to the bus 1240 via a video interface 1222, and transport of data between the display 1232 and the bus 1240 can be controlled via the graphics control 1221.
- the display is a video projector.
- the display is a head-mounted display (HMD) such as a VR headset.
- suitable VR headsets include, by way of non-limiting examples, HTC Vive, Oculus Rift, Samsung Gear VR, Microsoft HoloLens, Razer OSVR, FOVE VR, Zeiss VR One, Avegant Glyph, Freefly VR headset, and the like.
- the display is a combination of devices such as those disclosed herein.
- computer system 1200 may include one or more other peripheral output devices 1234 including, but not limited to, an audio speaker, a printer, a storage device, and any combinations thereof.
- peripheral output devices may be connected to the bus 1240 via an output interface 1224.
- Examples of an output interface 1224 include, but are not limited to, a serial port, a parallel connection, a USB port, a FIREWIRE port, a THUNDERBOLT port, and any combinations thereof.
- computer system 1200 may provide functionality as a result of logic hardwired or otherwise embodied in a circuit, which may operate in place of or together with software to execute one or more processes or one or more steps of one or more processes described or illustrated herein.
- Reference to software in this disclosure may encompass logic, and reference to logic may encompass software.
- reference to a computer-readable medium may encompass a circuit (such as an IC) storing software for execution, a circuit embodying logic for execution, or both, where appropriate.
- DSP digital signal processor
- ASIC application specific integrated circuit
- FPGA field programmable gate array
- a general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine.
- a processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.
- a software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
- An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium.
- the storage medium may be integral to the processor.
- the processor and the storage medium may reside in an ASIC.
- the ASIC may reside in a user terminal.
- the processor and the storage medium may reside as discrete components in a user terminal.
- suitable computing devices include, by way of non-limiting examples, server computers, desktop computers, laptop computers, notebook computers, sub-notebook computers, netbook computers, netpad computers, set-top computers, media streaming devices, handheld computers, Internet appliances, mobile smartphones, tablet computers, personal digital assistants, video game consoles, and vehicles.
- server computers desktop computers, laptop computers, notebook computers, sub-notebook computers, netbook computers, netpad computers, set-top computers, media streaming devices, handheld computers, Internet appliances, mobile smartphones, tablet computers, personal digital assistants, video game consoles, and vehicles.
- Suitable tablet computers include those with booklet, slate, and convertible configurations, known to those of skill in the art.
- the computing device includes an operating system configured to perform executable instructions.
- the operating system is, for example, software, including programs and data, which manages the device’s hardware and provides services for execution of applications.
- suitable server operating systems include, by way of non-limiting examples, FreeBSD, OpenBSD, NetBSD®, Linux, Apple® Mac OS X Server®, Oracle® Solaris®, Windows Server®, and Novell® NetWare®.
- suitable personal computer operating systems include, by way of non limiting examples, Microsoft® Windows®, Apple® Mac OS X®, UNIX®, and UNIX-like operating systems such as GNU/Linux®.
- the operating system is provided by cloud computing.
- suitable mobile smartphone operating systems include, by way of non-limiting examples, Nokia® Symbian®
- suitable media streaming device operating systems include, by way of non-limiting examples, Apple TV®, Roku®, Boxee®, Google TV®, Google Chromecast®, Amazon Fire®, and Samsung® HomeSync®.
- suitable video game console operating systems include, by way of non limiting examples, Sony® PS3®, Sony® PS4®, Microsoft® Xbox 360®, Microsoft Xbox One, Nintendo® Wii®, Nintendo® Wii U®, and Ouya®.
- the platforms, systems, media, and methods disclosed herein include one or more non-transitory computer readable storage media encoded with a program including instructions executable by the operating system of an optionally networked computing device.
- a computer readable storage medium is a tangible component of a computing device.
- a computer readable storage medium is optionally removable from a computing device.
- a computer readable storage medium includes, by way of non-limiting examples, CD-ROMs, DVDs, flash memory devices, solid state memory, magnetic disk drives, magnetic tape drives, optical disk drives, distributed computing systems including cloud computing systems and services, and the like.
- the program and instructions are permanently, substantially permanently, semi permanently, or non-transitorily encoded on the media.
- the platforms, systems, media, and methods disclosed herein include at least one computer program, or use of the same.
- a computer program includes a sequence of instructions, executable by one or more processor(s) of the computing device’s CPU, written to perform a specified task.
- Computer readable instructions may be implemented as program modules, such as functions, objects, Application Programming Interfaces (APIs), computing data structures, and the like, that perform particular tasks or implement particular abstract data types. In light of the disclosure provided herein, those of skill in the art will recognize that a computer program may be written in various versions of various languages.
- the functionality of the computer readable instructions may be combined or distributed as desired in various environments.
- a computer program comprises one sequence of instructions.
- a computer program comprises a plurality of sequences of instructions. In some embodiments, a computer program is provided from one location. In other embodiments, a computer program is provided from a plurality of locations. In various embodiments, a computer program includes one or more software modules. In various embodiments, a computer program includes, in part or in whole, one or more web applications, one or more mobile applications, one or more standalone applications, one or more web browser plug-ins, extensions, add-ins, or add-ons, or combinations thereof.
- the platforms, systems, media, and methods disclosed herein include software, server, and/or database modules, or use of the same.
- software modules are created by techniques known to those of skill in the art using machines, software, and languages known to the art.
- the software modules disclosed herein are implemented in a multitude of ways.
- a software module comprises a file, a section of code, a programming object, a programming structure, or combinations thereof.
- a software module comprises a plurality of files, a plurality of sections of code, a plurality of programming objects, a plurality of programming structures, or combinations thereof.
- the one or more software modules comprise, by way of non-limiting examples, a web application, a mobile application, and a standalone application.
- software modules are in one computer program or application. In other embodiments, software modules are in more than one computer program or application. In some embodiments, software modules are hosted on one machine. In other embodiments, software modules are hosted on more than one machine. In further embodiments, software modules are hosted on a distributed computing platform such as a cloud computing platform. In some embodiments, software modules are hosted on one or more machines in one location. In other embodiments, software modules are hosted on one or more machines in more than one location. Databases
- the platforms, systems, media, and methods disclosed herein include one or more databases, or use of the same.
- suitable databases include, by way of non-limiting examples, relational databases, non-relational databases, object oriented databases, object databases, entity- relationship model databases, associative databases, and XML databases. Further non-limiting examples include SQL, PostgreSQL, MySQL, Oracle, DB2, and Sybase.
- a database is internet-based.
- a database is web-based.
- a database is cloud computing-based.
- a database is a distributed database.
- a database is based on one or more local computer storage devices.
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Medical Informatics (AREA)
- Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Molecular Biology (AREA)
- Pathology (AREA)
- Animal Behavior & Ethology (AREA)
- Biophysics (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Artificial Intelligence (AREA)
- Optics & Photonics (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physiology (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Theoretical Computer Science (AREA)
- Cardiology (AREA)
- Hematology (AREA)
- Quality & Reliability (AREA)
- Psychiatry (AREA)
- Mathematical Physics (AREA)
- Fuzzy Systems (AREA)
- Multimedia (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
- Lasers (AREA)
- Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
- Force Measurement Appropriate To Specific Purposes (AREA)
- Laser Surgery Devices (AREA)
Abstract
Description
Claims
Priority Applications (7)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
IL294553A IL294553A (en) | 2020-01-08 | 2021-01-07 | Laser speckle force feedback estimation |
EP21738084.9A EP4087474A4 (en) | 2020-01-08 | 2021-01-07 | Laser speckle force feedback estimation |
KR1020227027278A KR20220125804A (en) | 2020-01-08 | 2021-01-07 | Laser speckle force feedback estimation |
JP2022541961A JP2023509772A (en) | 2020-01-08 | 2021-01-07 | Estimation of laser speckle force feedback |
CN202180019070.XA CN115243610A (en) | 2020-01-08 | 2021-01-07 | Laser speckle force feedback estimation |
CA3164149A CA3164149A1 (en) | 2020-01-08 | 2021-01-07 | Laser speckle force feedback estimation |
US17/810,988 US20220409065A1 (en) | 2020-01-08 | 2022-07-06 | Laser speckle force feedback estimation |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US202062958501P | 2020-01-08 | 2020-01-08 | |
US62/958,501 | 2020-01-08 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/810,988 Continuation US20220409065A1 (en) | 2020-01-08 | 2022-07-06 | Laser speckle force feedback estimation |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021142138A1 true WO2021142138A1 (en) | 2021-07-15 |
Family
ID=76788882
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2021/012524 WO2021142138A1 (en) | 2020-01-08 | 2021-01-07 | Laser speckle force feedback estimation |
Country Status (8)
Country | Link |
---|---|
US (1) | US20220409065A1 (en) |
EP (1) | EP4087474A4 (en) |
JP (1) | JP2023509772A (en) |
KR (1) | KR20220125804A (en) |
CN (1) | CN115243610A (en) |
CA (1) | CA3164149A1 (en) |
IL (1) | IL294553A (en) |
WO (1) | WO2021142138A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2023049401A1 (en) * | 2021-09-24 | 2023-03-30 | Activ Surgical, Inc. | Systems and methods for perfusion quantification |
WO2024030827A3 (en) * | 2022-08-02 | 2024-04-04 | The Regents Of The University Of California | Systems and methods of non-contact force sensing |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110112549A1 (en) * | 2008-05-28 | 2011-05-12 | Zipi Neubach | Ultrasound guided robot for flexible needle steering |
US20110172565A1 (en) * | 2008-05-16 | 2011-07-14 | Drexel University | System and method for evaluating tissue |
US20120265061A1 (en) * | 2011-04-13 | 2012-10-18 | St. Jude Medical, Inc. | High speed elastographic property mapping of lumens utilizing micropalpation delivered from an oct-equipped catheter tip |
US20130237820A1 (en) * | 2010-06-01 | 2013-09-12 | The Trustees Of Columbia University In The City Of New York | Devices, methods, and systems for measuring elastic properties of biological tissues |
US20170181626A1 (en) * | 2015-12-23 | 2017-06-29 | Industrial Technology Research Institute | Introcular pressure detecting device and detecting method thereof |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2017139774A1 (en) * | 2016-02-12 | 2017-08-17 | The General Hospital Corporation | Laser speckle micro-rheology in characterization of biomechanical properties of tissues |
CN108613979B (en) * | 2018-03-12 | 2020-08-11 | 华中科技大学鄂州工业技术研究院 | Laser speckle image processing device and method for viscoelasticity quantitative detection |
-
2021
- 2021-01-07 IL IL294553A patent/IL294553A/en unknown
- 2021-01-07 CN CN202180019070.XA patent/CN115243610A/en active Pending
- 2021-01-07 EP EP21738084.9A patent/EP4087474A4/en active Pending
- 2021-01-07 JP JP2022541961A patent/JP2023509772A/en active Pending
- 2021-01-07 KR KR1020227027278A patent/KR20220125804A/en unknown
- 2021-01-07 WO PCT/US2021/012524 patent/WO2021142138A1/en unknown
- 2021-01-07 CA CA3164149A patent/CA3164149A1/en active Pending
-
2022
- 2022-07-06 US US17/810,988 patent/US20220409065A1/en active Pending
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110172565A1 (en) * | 2008-05-16 | 2011-07-14 | Drexel University | System and method for evaluating tissue |
US20110112549A1 (en) * | 2008-05-28 | 2011-05-12 | Zipi Neubach | Ultrasound guided robot for flexible needle steering |
US20130237820A1 (en) * | 2010-06-01 | 2013-09-12 | The Trustees Of Columbia University In The City Of New York | Devices, methods, and systems for measuring elastic properties of biological tissues |
US20120265061A1 (en) * | 2011-04-13 | 2012-10-18 | St. Jude Medical, Inc. | High speed elastographic property mapping of lumens utilizing micropalpation delivered from an oct-equipped catheter tip |
US20170181626A1 (en) * | 2015-12-23 | 2017-06-29 | Industrial Technology Research Institute | Introcular pressure detecting device and detecting method thereof |
Non-Patent Citations (1)
Title |
---|
See also references of EP4087474A4 * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2023049401A1 (en) * | 2021-09-24 | 2023-03-30 | Activ Surgical, Inc. | Systems and methods for perfusion quantification |
WO2024030827A3 (en) * | 2022-08-02 | 2024-04-04 | The Regents Of The University Of California | Systems and methods of non-contact force sensing |
Also Published As
Publication number | Publication date |
---|---|
CN115243610A (en) | 2022-10-25 |
KR20220125804A (en) | 2022-09-14 |
JP2023509772A (en) | 2023-03-09 |
EP4087474A4 (en) | 2024-03-06 |
US20220409065A1 (en) | 2022-12-29 |
CA3164149A1 (en) | 2021-07-15 |
IL294553A (en) | 2022-09-01 |
EP4087474A1 (en) | 2022-11-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Eckert et al. | Augmented reality in medicine: systematic and bibliographic review | |
US20220409065A1 (en) | Laser speckle force feedback estimation | |
Niehorster et al. | The accuracy and precision of position and orientation tracking in the HTC vive virtual reality system for scientific research | |
US10902034B2 (en) | Method for populating a map with a plurality of avatars through the use of a mobile technology platform | |
US20210110570A1 (en) | System for guiding medically invasive devices relative to anatomical structures via image processing | |
Chen et al. | Recent developments and future challenges in medical mixed reality | |
Pepe et al. | A marker-less registration approach for mixed reality–aided maxillofacial surgery: a pilot evaluation | |
TWI781226B (en) | Methods, systems, and media for detecting spoofing in mobile authentication | |
WO2018187748A1 (en) | Systems and methods for mixed reality medical training | |
US20200388177A1 (en) | Simulated reality based confidence assessment | |
US20230095492A1 (en) | Methods and systems using fractional rank precision and mean average precision as test-retest reliability measures | |
US11507248B2 (en) | Methods, systems, and media for anti-spoofing using eye-tracking | |
Gasmi et al. | Augmented reality, virtual reality and new age technologies demand escalates amid COVID-19 | |
Ganapathy | Metaverse and healthcare: A clinician's perspective | |
US20160358040A1 (en) | System and methods for trajectory pattern recognition | |
US20230154092A1 (en) | Kinematic interaction system with improved pose tracking | |
US20220293014A1 (en) | Virtual reality medical simulation | |
WO2021206995A1 (en) | System for delivering targeted content with care preference suggestions | |
CA3154451A1 (en) | Systems and methods for digital vaccine | |
Dominic et al. | Combining predictive analytics and artificial intelligence with human intelligence in iot-based image-guided surgery | |
US20230120871A1 (en) | System for delivering targeted content to unique user persona groups | |
US20240130696A1 (en) | Enhanced invasive imaging systems and methods | |
US20240169711A1 (en) | Multi-modal understanding of emotions in video content | |
KASOWSKI et al. | Furthering Visual Accessibility with Extended Reality (XR): A Systematic | |
Abate et al. | Integrating Gaze Tracking with Augmented Reality on Mobile Devices: A Framework for Enhanced User Interaction |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21738084 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2022541961 Country of ref document: JP Kind code of ref document: A Ref document number: 3164149 Country of ref document: CA |
|
ENP | Entry into the national phase |
Ref document number: 20227027278 Country of ref document: KR Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2021738084 Country of ref document: EP Effective date: 20220808 |