JP7577737B2 - Vats及び低侵襲手術での術中仮想アノテーションのためのインタラクティブ内視鏡検査 - Google Patents

Vats及び低侵襲手術での術中仮想アノテーションのためのインタラクティブ内視鏡検査 Download PDF

Info

Publication number
JP7577737B2
JP7577737B2 JP2022516050A JP2022516050A JP7577737B2 JP 7577737 B2 JP7577737 B2 JP 7577737B2 JP 2022516050 A JP2022516050 A JP 2022516050A JP 2022516050 A JP2022516050 A JP 2022516050A JP 7577737 B2 JP7577737 B2 JP 7577737B2
Authority
JP
Japan
Prior art keywords
annotation
interventional
image
software instructions
controller
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2022516050A
Other languages
English (en)
Japanese (ja)
Other versions
JP2022548237A5 (https=
JP2022548237A (ja
Inventor
ポール ティエンファラパ
トーレ ミシェル バイドロン
アルヴィン チェン
プラサド ヴァグダルギ
ウィリアム マクナマラ
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Koninklijke Philips NV
Original Assignee
Koninklijke Philips NV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips NV filed Critical Koninklijke Philips NV
Publication of JP2022548237A publication Critical patent/JP2022548237A/ja
Publication of JP2022548237A5 publication Critical patent/JP2022548237A5/ja
Application granted granted Critical
Publication of JP7577737B2 publication Critical patent/JP7577737B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/70Labelling scene content, e.g. deriving syntactic or semantic representations
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002Operational features of endoscopes
    • A61B1/00004Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002Operational features of endoscopes
    • A61B1/00004Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000094Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope extracting biological structures
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002Operational features of endoscopes
    • A61B1/00004Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000096Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope using artificial intelligence
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002Operational features of endoscopes
    • A61B1/00043Operational features of endoscopes provided with output arrangements
    • A61B1/00045Display arrangement
    • A61B1/0005Display arrangement combining images e.g. side-by-side, superimposed or tiled
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/313Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for introducing through surgical openings, e.g. laparoscopes
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00Two-dimensional [2D] image generation
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion
    • G06T7/246Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/30Determination of transform parameters for the alignment of images, i.e. image registration
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • G06T7/73Determining position or orientation of objects or cameras using feature-based methods
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/44Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10068Endoscopic image
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20112Image segmentation details
    • G06T2207/20164Salient point detection; Corner detection
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30061Lung
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30061Lung
    • G06T2207/30064Lung nodule
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30096Tumor; Lesion
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03Recognition of patterns in medical or anatomical images
    • G06V2201/031Recognition of patterns in medical or anatomical images of internal organs
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03Recognition of patterns in medical or anatomical images
    • G06V2201/034Recognition of patterns in medical or anatomical images of medical instruments

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Surgery (AREA)
  • General Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Biomedical Technology (AREA)
  • Veterinary Medicine (AREA)
  • Pathology (AREA)
  • Biophysics (AREA)
  • Optics & Photonics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Quality & Reliability (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Endoscopes (AREA)
  • Image Analysis (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
JP2022516050A 2019-09-12 2020-09-11 Vats及び低侵襲手術での術中仮想アノテーションのためのインタラクティブ内視鏡検査 Active JP7577737B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201962899365P 2019-09-12 2019-09-12
US62/899,365 2019-09-12
PCT/EP2020/075422 WO2021048326A1 (en) 2019-09-12 2020-09-11 Interactive endoscopy for intraoperative virtual annotation in vats and minimally invasive surgery

Publications (3)

Publication Number Publication Date
JP2022548237A JP2022548237A (ja) 2022-11-17
JP2022548237A5 JP2022548237A5 (https=) 2023-09-15
JP7577737B2 true JP7577737B2 (ja) 2024-11-05

Family

ID=72473566

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2022516050A Active JP7577737B2 (ja) 2019-09-12 2020-09-11 Vats及び低侵襲手術での術中仮想アノテーションのためのインタラクティブ内視鏡検査

Country Status (5)

Country Link
US (1) US12347216B2 (https=)
EP (1) EP4028988A1 (https=)
JP (1) JP7577737B2 (https=)
CN (1) CN114554939B (https=)
WO (1) WO2021048326A1 (https=)

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230409749A1 (en) * 2020-11-20 2023-12-21 SST Canada Inc. Systems and methods for surgical video de-identification
US12361557B2 (en) * 2020-12-21 2025-07-15 Medtronic Navigation, Inc. Systems and methods for monitoring one or more anatomical elements
WO2022177823A1 (en) * 2021-02-17 2022-08-25 PacificMD Biotech, LLC System and method for endoscopic imaging and analyses
JP2022149625A (ja) * 2021-03-25 2022-10-07 キヤノンメディカルシステムズ株式会社 医用画像診断システムおよび画像処理プログラム
US11957302B2 (en) 2021-05-24 2024-04-16 Verily Life Sciences Llc User-interface for visualization of endoscopy procedures
US12057219B2 (en) 2021-07-22 2024-08-06 Cilag Gmbh International Surgical data processing and metadata annotation
US11783938B2 (en) 2021-07-22 2023-10-10 Cilag Gmbh International Integrated hub systems control interfaces and connections
US12592312B2 (en) 2021-07-22 2026-03-31 Cilag Gmbh International Surgical data system and control
US11601232B2 (en) * 2021-07-22 2023-03-07 Cilag Gmbh International Redundant communication channels and processing of imaging feeds
US12205712B2 (en) 2021-07-22 2025-01-21 Cilag Gmbh International Method of surgical system power management, communication, processing, storage and display
US12068068B2 (en) 2021-07-22 2024-08-20 Cilag Gmbh International Cooperative composite video streams layered onto the surgical site and instruments
US12046358B2 (en) 2021-07-22 2024-07-23 Cilag Gmbh International Configuration of the display settings and displayed information based on the recognition of the user(s) and awareness of procedure, location or usage
TR2022020864A2 (tr) * 2022-12-28 2023-01-23 Smart Alfa Teknoloji Sanayi Ve Ticaret Anonim Sirketi Tibbi̇ amaçli anatomi̇k görüntü alinmasi sirasinda hasta hareketleri̇ni̇n tespi̇ti̇ ve yöneti̇lmesi̇ amaciyla kullanilabi̇len bi̇r usul ve si̇stem
CN116562392B (zh) * 2023-04-17 2025-12-05 同济大学 血管介入手术动作感知与学习系统、方法及存储介质
CN119856978B (zh) * 2024-12-25 2025-09-02 中国科学院自动化研究所 肘关节缝合锚钉置入方向的确定方法、装置、电子设备、存储介质和程序产品

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016163609A (ja) 2015-03-06 2016-09-08 富士フイルム株式会社 分岐構造判定装置、方法およびプログラム
JP2017534117A (ja) 2014-11-07 2017-11-16 コーニンクレッカ フィリップス エヌ ヴェKoninklijke Philips N.V. 最適化された解剖学的関心構造ラベリング
US20190069957A1 (en) 2017-09-06 2019-03-07 Verily Life Sciences Llc Surgical recognition system

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8443279B1 (en) 2004-10-13 2013-05-14 Stryker Corporation Voice-responsive annotation of video generated by an endoscopic camera
KR101759534B1 (ko) * 2009-10-30 2017-07-19 더 존스 홉킨스 유니버시티 외과 수술 시 임상적으로 중요한 해부상의 랜드마크에 대한 시각 추적 및 주석 달기
US10631712B2 (en) 2011-02-10 2020-04-28 Karl Storz Imaging, Inc. Surgeon's aid for medical display
US9123155B2 (en) * 2011-08-09 2015-09-01 Covidien Lp Apparatus and method for using augmented reality vision system in surgical procedures
EP3166468A4 (en) * 2014-07-10 2018-03-07 M.S.T. Medical Surgery Technologies Ltd. Improved interface for laparoscopic surgeries - movement gestures
US10169535B2 (en) 2015-01-16 2019-01-01 The University Of Maryland, Baltimore County Annotation of endoscopic video using gesture and voice commands
CN110050281B (zh) * 2016-12-08 2023-06-20 皇家飞利浦有限公司 学习图像中的对象的注释
US11216673B2 (en) * 2017-04-04 2022-01-04 Robert Bosch Gmbh Direct vehicle detection as 3D bounding boxes using neural network image processing
US11266468B2 (en) * 2017-12-28 2022-03-08 Cilag Gmbh International Cooperative utilization of data derived from secondary sources by intelligent surgical hubs
US10789456B2 (en) * 2017-12-28 2020-09-29 Adobe Inc. Facial expression recognition utilizing unsupervised learning
US10864043B2 (en) * 2018-02-09 2020-12-15 Kathleen M Douglas Interactive placement of a 3D digital representation of a surgical device or anatomic feature into a 3D radiologic image for pre-operative planning
US10957108B2 (en) * 2019-04-15 2021-03-23 Shutterstock, Inc. Augmented reality image retrieval systems and methods

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017534117A (ja) 2014-11-07 2017-11-16 コーニンクレッカ フィリップス エヌ ヴェKoninklijke Philips N.V. 最適化された解剖学的関心構造ラベリング
JP2016163609A (ja) 2015-03-06 2016-09-08 富士フイルム株式会社 分岐構造判定装置、方法およびプログラム
US20190069957A1 (en) 2017-09-06 2019-03-07 Verily Life Sciences Llc Surgical recognition system

Also Published As

Publication number Publication date
CN114554939A (zh) 2022-05-27
US12347216B2 (en) 2025-07-01
WO2021048326A1 (en) 2021-03-18
CN114554939B (zh) 2026-03-31
JP2022548237A (ja) 2022-11-17
US20220358773A1 (en) 2022-11-10
EP4028988A1 (en) 2022-07-20

Similar Documents

Publication Publication Date Title
JP7577737B2 (ja) Vats及び低侵襲手術での術中仮想アノテーションのためのインタラクティブ内視鏡検査
US12283196B2 (en) Surgical simulator providing labeled data
Vercauteren et al. Cai4cai: the rise of contextual artificial intelligence in computer-assisted interventions
JP6931121B2 (ja) 外科用認識システム
KR20190100011A (ko) 수술영상을 이용한 수술정보 제공 방법 및 장치
JP2023528655A (ja) 医療データを処理するためのシステムおよび方法
US12051156B2 (en) System and method for linking a segmentation graph to volumetric data
WO2024186811A1 (en) Machine learning for object tracking
US20220293245A1 (en) Voice controlled medical diagnosis system
CN106164981A (zh) 外科手术中为手术器械插入显示计时信号的方法和系统
CN116883281A (zh) 一种用于图像引导放疗的影像呈像画质增强系统和方法
JP7444569B2 (ja) 鏡視下手術支援装置、鏡視下手術支援方法、及びプログラム
KR20190133423A (ko) 수술 시뮬레이션 정보 생성방법 및 프로그램
Ueki et al. Developing an artificial intelligence model for phase recognition in robot‐assisted radical prostatectomy
KR20190133424A (ko) 수술결과에 대한 피드백 제공방법 및 프로그램
Zheng et al. Fundamentals of digital surgery
CN113610840B (zh) 图像处理方法、装置、存储介质及电子设备
CN120813975A (zh) 用于对分支解剖结构进行分类的系统和方法
US20260024243A1 (en) User interface for structures detected in surgical procedures
WO2026024546A1 (en) Methods and systems for preoperative visualization of resection areas in thoracic surgeries using laparoscopic and endoluminal robotic systems
Zhang Deep-Learning-Enhanced Atlas-Based Preoperative and Intraoperative Registration for Cochlear Implant Surgery Navigation
Kirmizibayrak Interactive volume visualization and editing methods for surgical applications
WO2024185468A1 (ja) 医療支援装置、内視鏡システム、医療支援方法、及びプログラム
CN119214792A (zh) 一种手术机器人介入手术系统和方法
Foo A framework for tumor segmentation and interactive immersive visualization of medical image data for surgical planning

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20230907

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20230907

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20240313

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20240326

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240620

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20240925

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20241023

R150 Certificate of patent or registration of utility model

Ref document number: 7577737

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150