CN114554939B - 用于vats和微创手术中的术中虚拟注释的交互式内窥镜检查 - Google Patents

用于vats和微创手术中的术中虚拟注释的交互式内窥镜检查

Info

Publication number
CN114554939B
CN114554939B CN202080071517.3A CN202080071517A CN114554939B CN 114554939 B CN114554939 B CN 114554939B CN 202080071517 A CN202080071517 A CN 202080071517A CN 114554939 B CN114554939 B CN 114554939B
Authority
CN
China
Prior art keywords
annotation
interventional
processor
software instructions
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202080071517.3A
Other languages
English (en)
Chinese (zh)
Other versions
CN114554939A (zh
Inventor
P·西恩帕波
T·M·比德隆
A·陈
P·瓦格达尔吉
W·麦克纳马拉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Koninklijke Philips NV
Original Assignee
Koninklijke Philips NV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips NV filed Critical Koninklijke Philips NV
Publication of CN114554939A publication Critical patent/CN114554939A/zh
Application granted granted Critical
Publication of CN114554939B publication Critical patent/CN114554939B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000094—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope extracting biological structures
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000096—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope using artificial intelligence
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00043—Operational features of endoscopes provided with output arrangements
    • A61B1/00045—Display arrangement
    • A61B1/0005—Display arrangement combining images e.g. side-by-side, superimposed or tiled
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/313—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for introducing through surgical openings, e.g. laparoscopes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10068—Endoscopic image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20164—Salient point detection; Corner detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30061—Lung
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30061—Lung
    • G06T2207/30064—Lung nodule
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images
    • G06V2201/031—Recognition of patterns in medical or anatomical images of internal organs
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images
    • G06V2201/034—Recognition of patterns in medical or anatomical images of medical instruments

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Surgery (AREA)
  • General Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Biomedical Technology (AREA)
  • Veterinary Medicine (AREA)
  • Pathology (AREA)
  • Biophysics (AREA)
  • Optics & Photonics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Quality & Reliability (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Endoscopes (AREA)
  • Image Analysis (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
CN202080071517.3A 2019-09-12 2020-09-11 用于vats和微创手术中的术中虚拟注释的交互式内窥镜检查 Active CN114554939B (zh)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201962899365P 2019-09-12 2019-09-12
US62/899,365 2019-09-12
PCT/EP2020/075422 WO2021048326A1 (en) 2019-09-12 2020-09-11 Interactive endoscopy for intraoperative virtual annotation in vats and minimally invasive surgery

Publications (2)

Publication Number Publication Date
CN114554939A CN114554939A (zh) 2022-05-27
CN114554939B true CN114554939B (zh) 2026-03-31

Family

ID=72473566

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202080071517.3A Active CN114554939B (zh) 2019-09-12 2020-09-11 用于vats和微创手术中的术中虚拟注释的交互式内窥镜检查

Country Status (5)

Country Link
US (1) US12347216B2 (https=)
EP (1) EP4028988A1 (https=)
JP (1) JP7577737B2 (https=)
CN (1) CN114554939B (https=)
WO (1) WO2021048326A1 (https=)

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230409749A1 (en) * 2020-11-20 2023-12-21 SST Canada Inc. Systems and methods for surgical video de-identification
US12361557B2 (en) * 2020-12-21 2025-07-15 Medtronic Navigation, Inc. Systems and methods for monitoring one or more anatomical elements
WO2022177823A1 (en) * 2021-02-17 2022-08-25 PacificMD Biotech, LLC System and method for endoscopic imaging and analyses
JP2022149625A (ja) * 2021-03-25 2022-10-07 キヤノンメディカルシステムズ株式会社 医用画像診断システムおよび画像処理プログラム
US11957302B2 (en) 2021-05-24 2024-04-16 Verily Life Sciences Llc User-interface for visualization of endoscopy procedures
US12057219B2 (en) 2021-07-22 2024-08-06 Cilag Gmbh International Surgical data processing and metadata annotation
US11783938B2 (en) 2021-07-22 2023-10-10 Cilag Gmbh International Integrated hub systems control interfaces and connections
US12592312B2 (en) 2021-07-22 2026-03-31 Cilag Gmbh International Surgical data system and control
US11601232B2 (en) * 2021-07-22 2023-03-07 Cilag Gmbh International Redundant communication channels and processing of imaging feeds
US12205712B2 (en) 2021-07-22 2025-01-21 Cilag Gmbh International Method of surgical system power management, communication, processing, storage and display
US12068068B2 (en) 2021-07-22 2024-08-20 Cilag Gmbh International Cooperative composite video streams layered onto the surgical site and instruments
US12046358B2 (en) 2021-07-22 2024-07-23 Cilag Gmbh International Configuration of the display settings and displayed information based on the recognition of the user(s) and awareness of procedure, location or usage
TR2022020864A2 (tr) * 2022-12-28 2023-01-23 Smart Alfa Teknoloji Sanayi Ve Ticaret Anonim Sirketi Tibbi̇ amaçli anatomi̇k görüntü alinmasi sirasinda hasta hareketleri̇ni̇n tespi̇ti̇ ve yöneti̇lmesi̇ amaciyla kullanilabi̇len bi̇r usul ve si̇stem
CN116562392B (zh) * 2023-04-17 2025-12-05 同济大学 血管介入手术动作感知与学习系统、方法及存储介质
CN119856978B (zh) * 2024-12-25 2025-09-02 中国科学院自动化研究所 肘关节缝合锚钉置入方向的确定方法、装置、电子设备、存储介质和程序产品

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8443279B1 (en) 2004-10-13 2013-05-14 Stryker Corporation Voice-responsive annotation of video generated by an endoscopic camera
KR101759534B1 (ko) * 2009-10-30 2017-07-19 더 존스 홉킨스 유니버시티 외과 수술 시 임상적으로 중요한 해부상의 랜드마크에 대한 시각 추적 및 주석 달기
US10631712B2 (en) 2011-02-10 2020-04-28 Karl Storz Imaging, Inc. Surgeon's aid for medical display
US9123155B2 (en) * 2011-08-09 2015-09-01 Covidien Lp Apparatus and method for using augmented reality vision system in surgical procedures
EP3166468A4 (en) * 2014-07-10 2018-03-07 M.S.T. Medical Surgery Technologies Ltd. Improved interface for laparoscopic surgeries - movement gestures
CN107004043B (zh) 2014-11-07 2022-07-29 皇家飞利浦有限公司 优化检测和标记感兴趣解剖结构的系统和方法
US10169535B2 (en) 2015-01-16 2019-01-01 The University Of Maryland, Baltimore County Annotation of endoscopic video using gesture and voice commands
JP6348078B2 (ja) 2015-03-06 2018-06-27 富士フイルム株式会社 分岐構造判定装置、分岐構造判定装置の作動方法および分岐構造判定プログラム
CN110050281B (zh) * 2016-12-08 2023-06-20 皇家飞利浦有限公司 学习图像中的对象的注释
US11216673B2 (en) * 2017-04-04 2022-01-04 Robert Bosch Gmbh Direct vehicle detection as 3D bounding boxes using neural network image processing
US20190069957A1 (en) * 2017-09-06 2019-03-07 Verily Life Sciences Llc Surgical recognition system
US11266468B2 (en) * 2017-12-28 2022-03-08 Cilag Gmbh International Cooperative utilization of data derived from secondary sources by intelligent surgical hubs
US10789456B2 (en) * 2017-12-28 2020-09-29 Adobe Inc. Facial expression recognition utilizing unsupervised learning
US10864043B2 (en) * 2018-02-09 2020-12-15 Kathleen M Douglas Interactive placement of a 3D digital representation of a surgical device or anatomic feature into a 3D radiologic image for pre-operative planning
US10957108B2 (en) * 2019-04-15 2021-03-23 Shutterstock, Inc. Augmented reality image retrieval systems and methods

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Augmented reality and cone beam CT guidance for transoral robotic surgery;Wen P. Liu;J Robot Surg;20150930;第1-16页 *

Also Published As

Publication number Publication date
CN114554939A (zh) 2022-05-27
US12347216B2 (en) 2025-07-01
WO2021048326A1 (en) 2021-03-18
JP2022548237A (ja) 2022-11-17
JP7577737B2 (ja) 2024-11-05
US20220358773A1 (en) 2022-11-10
EP4028988A1 (en) 2022-07-20

Similar Documents

Publication Publication Date Title
CN114554939B (zh) 用于vats和微创手术中的术中虚拟注释的交互式内窥镜检查
Collins et al. Augmented reality guided laparoscopic surgery of the uterus
Vercauteren et al. Cai4cai: the rise of contextual artificial intelligence in computer-assisted interventions
US20250151976A1 (en) Systems and methods for registering imaging data from different imaging modalities based on subsurface image scanning
US12040071B2 (en) Method and program for providing feedback on surgical outcome
JP6348130B2 (ja) デュアルデータ同期を用いた解剖学的部位の位置の再特定
US20230190136A1 (en) Systems and methods for computer-assisted shape measurements in video
US12051156B2 (en) System and method for linking a segmentation graph to volumetric data
CN112654324A (zh) 用于在外科手术期间提供协助的系统和方法
US20130296682A1 (en) Integrating pre-surgical and surgical images
JP2020532347A (ja) 外科用認識システム
US11847730B2 (en) Orientation detection in fluoroscopic images
JP7810655B2 (ja) ビデオにおけるコンピュータ支援の標識または基準配置のシステムおよび方法
CN116883281A (zh) 一种用于图像引导放疗的影像呈像画质增强系统和方法
JP2021045340A (ja) 鏡視下手術支援装置、鏡視下手術支援方法、及びプログラム
CN114334096A (zh) 基于医学影像的术中辅助显示方法、装置及存储介质
US12475578B2 (en) Systems and methods for registering visual representations of a surgical space
Kim et al. A learning-based, region of interest-tracking algorithm for catheter detection in echocardiography
KR20190133424A (ko) 수술결과에 대한 피드백 제공방법 및 프로그램
WO2026024546A1 (en) Methods and systems for preoperative visualization of resection areas in thoracic surgeries using laparoscopic and endoluminal robotic systems
CN120813975A (zh) 用于对分支解剖结构进行分类的系统和方法
Zhang Deep-Learning-Enhanced Atlas-Based Preoperative and Intraoperative Registration for Cochlear Implant Surgery Navigation
CN115990059A (zh) 操作用于图像引导手术过程的医疗导航系统的方法
Reiter Assistive visual tools for surgery
Banerjee Fast 4D Ultrasound Registration for Image Guided Liver Interventions

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant