AU2020362217B2 - Systems and methods for changing the direction of view during video guided clinical procedures using real-time image processing - Google Patents

Systems and methods for changing the direction of view during video guided clinical procedures using real-time image processing Download PDF

Info

Publication number
AU2020362217B2
AU2020362217B2 AU2020362217A AU2020362217A AU2020362217B2 AU 2020362217 B2 AU2020362217 B2 AU 2020362217B2 AU 2020362217 A AU2020362217 A AU 2020362217A AU 2020362217 A AU2020362217 A AU 2020362217A AU 2020362217 B2 AU2020362217 B2 AU 2020362217B2
Authority
AU
Australia
Prior art keywords
camera
image
source
target
point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
AU2020362217A
Other languages
English (en)
Other versions
AU2020362217A1 (en
Inventor
Michel Goncalves ALMEIDA ANTUNES
João Pedro DE ALMEIDA BARRETO
Carolina DOS SANTOS RAPOSO
Rui Jorge Melo Teixeira
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
S&n Orion Prime SA
Original Assignee
S&n Orion Prime S A
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by S&n Orion Prime S A filed Critical S&n Orion Prime S A
Publication of AU2020362217A1 publication Critical patent/AU2020362217A1/en
Application granted granted Critical
Publication of AU2020362217B2 publication Critical patent/AU2020362217B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00163—Optical arrangements
    • A61B1/00174—Optical arrangements characterised by the viewing angles
    • A61B1/00179—Optical arrangements characterised by the viewing angles for off-axis viewing
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000094—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope extracting biological structures
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000096—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope using artificial intelligence
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00043—Operational features of endoscopes provided with output arrangements
    • A61B1/00045—Display arrangement
    • A61B1/0005—Display arrangement combining images e.g. side-by-side, superimposed or tiled
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00163—Optical arrangements
    • A61B1/00174—Optical arrangements characterised by the viewing angles
    • A61B1/00183—Optical arrangements characterised by the viewing angles for variable viewing angles
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00163—Optical arrangements
    • A61B1/00188—Optical arrangements with focusing or zooming features
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B23/00—Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices
    • G02B23/24—Instruments or systems for viewing the inside of hollow bodies, e.g. fibrescopes
    • G02B23/2476—Non-optical details, e.g. housings, mountings, supports
    • G02B23/2484—Arrangements in relation to a camera or imaging device
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/10—Geometric effects
    • G06T15/20—Perspective computation
    • G06T15/205—Image-based rendering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/18—Image warping, e.g. rearranging pixels individually
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10068—Endoscopic image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/41—Medical

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Surgery (AREA)
  • Medical Informatics (AREA)
  • Optics & Photonics (AREA)
  • Biophysics (AREA)
  • Public Health (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Veterinary Medicine (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Molecular Biology (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Geometry (AREA)
  • Computer Graphics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Multimedia (AREA)
  • Astronomy & Astrophysics (AREA)
  • Endoscopes (AREA)
  • Instruments For Viewing The Inside Of Hollow Bodies (AREA)
  • Image Processing (AREA)
AU2020362217A 2019-10-07 2020-10-07 Systems and methods for changing the direction of view during video guided clinical procedures using real-time image processing Active AU2020362217B2 (en)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US201962911986P 2019-10-07 2019-10-07
US201962911950P 2019-10-07 2019-10-07
US62/911,986 2019-10-07
US62/911,950 2019-10-07
PCT/US2020/054640 WO2021071991A1 (en) 2019-10-07 2020-10-07 Systems and methods for changing the direction of view during video guided clinical procedures using real-time image processing

Publications (2)

Publication Number Publication Date
AU2020362217A1 AU2020362217A1 (en) 2022-04-07
AU2020362217B2 true AU2020362217B2 (en) 2025-05-29

Family

ID=75436886

Family Applications (3)

Application Number Title Priority Date Filing Date
AU2020362217A Active AU2020362217B2 (en) 2019-10-07 2020-10-07 Systems and methods for changing the direction of view during video guided clinical procedures using real-time image processing
AU2020362216A Active AU2020362216B2 (en) 2019-10-07 2020-10-07 Systems and methods for characterization of an endoscope and automatic calibration of an endoscopic camera system
AU2025238014A Pending AU2025238014A1 (en) 2019-10-07 2025-09-25 Systems and methods for characterization of an endoscope and automatic calibration of an endoscopic camera system

Family Applications After (2)

Application Number Title Priority Date Filing Date
AU2020362216A Active AU2020362216B2 (en) 2019-10-07 2020-10-07 Systems and methods for characterization of an endoscope and automatic calibration of an endoscopic camera system
AU2025238014A Pending AU2025238014A1 (en) 2019-10-07 2025-09-25 Systems and methods for characterization of an endoscope and automatic calibration of an endoscopic camera system

Country Status (6)

Country Link
US (4) US12178401B2 (enExample)
EP (2) EP4041048B1 (enExample)
JP (2) JP7727622B2 (enExample)
CN (2) CN114630611B (enExample)
AU (3) AU2020362217B2 (enExample)
WO (2) WO2021071988A1 (enExample)

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12544180B2 (en) 2018-10-26 2026-02-10 Intuitive Surgical Operations, Inc. Mixed reality systems and methods for indicating an extent of a field of view of an imaging device
US12008721B2 (en) * 2018-10-26 2024-06-11 Intuitive Surgical Operations, Inc. Mixed reality systems and methods for indicating an extent of a field of view of an imaging device
AU2020362217B2 (en) 2019-10-07 2025-05-29 S&N Orion Prime, S.A. Systems and methods for changing the direction of view during video guided clinical procedures using real-time image processing
WO2021168408A1 (en) 2020-02-20 2021-08-26 Smith & Nephew, Inc. Methods for arthroscopic video analysis and devices therefor
WO2021203082A1 (en) 2020-04-03 2021-10-07 Smith & Nephew, Inc. User interface for digital markers in arthroscopy
WO2021203077A1 (en) 2020-04-03 2021-10-07 Smith & Nephew, Inc. Methods for arthroscopic surgery video segmentation and devices therefor
US12236624B2 (en) 2020-09-23 2025-02-25 Smith & Nephew, Inc. Optical caliper for 3-D endoscopic imaging and measurement
CN114913236A (zh) * 2021-02-09 2022-08-16 深圳市汇顶科技股份有限公司 相机标定方法、装置及电子设备
WO2022176943A1 (ja) * 2021-02-22 2022-08-25 パナソニックＩｐマネジメント株式会社 口腔内カメラシステム及び画像表示方法
WO2022251814A2 (en) 2021-05-24 2022-12-01 Stryker Corporation Systems and methods for generating three-dimensional measurements using endoscopic video data
CN115393221B (zh) * 2022-08-29 2023-06-20 眉山市人民医院 适用于输尿管软镜末端的探镜角度校准方法、终端及介质
CN115590456B (zh) * 2022-09-29 2026-01-02 上海交通大学 转动式实时全景耳科用内窥镜
US12591997B2 (en) * 2023-01-25 2026-03-31 Auros Technology, Inc. Arrangement device and method
CN117392243B (zh) * 2023-12-13 2024-02-06 山东科技大学 基于图像处理的编码器安装位置检测方法及系统
CN117796745B (zh) * 2024-02-29 2024-05-03 四川大学 一种估计消化内镜镜头进退距离的方法
CN118747881B (zh) * 2024-08-30 2025-01-28 纽劢科技(上海)有限公司 一种基于全fov标注方式的真值评测方法及装置
FR3166464A1 (fr) * 2024-09-19 2026-03-20 Surgar Système et procédé d’étalonnage dynamique d’une caméra endoscopique

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140285676A1 (en) * 2011-07-25 2014-09-25 Universidade De Coimbra Method and apparatus for automatic camera calibration using one or more images of a checkerboard pattern
US20160048953A1 (en) * 2009-03-31 2016-02-18 Intuitive Surgical Operations, Inc Targets, fixtures, and workflows for calibrating an endoscopic camera

Family Cites Families (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5313306A (en) 1991-05-13 1994-05-17 Telerobotics International, Inc. Omniview motionless camera endoscopy system
US5417210A (en) * 1992-05-27 1995-05-23 International Business Machines Corporation System and method for augmentation of endoscopic surgery
US7344494B2 (en) 2004-02-09 2008-03-18 Karl Storz Development Corp. Endoscope with variable direction of view module
US9615772B2 (en) * 2004-02-20 2017-04-11 Karl Storz Imaging, Inc. Global endoscopic viewing indicator
JP4631048B2 (ja) 2005-02-14 2011-02-16 国立大学法人岩手大学 撮像装置及び撮像系パラメータの校正方法
US9526587B2 (en) * 2008-12-31 2016-12-27 Intuitive Surgical Operations, Inc. Fiducial marker design and detection for locating surgical instrument in images
US20070161854A1 (en) * 2005-10-26 2007-07-12 Moshe Alamaro System and method for endoscopic measurement and mapping of internal organs, tumors and other objects
GB0605817D0 (en) 2006-03-23 2006-05-03 Imp Innovations Ltd Reconstruction of anterior cruciate ligaments
US20070236514A1 (en) * 2006-03-29 2007-10-11 Bracco Imaging Spa Methods and Apparatuses for Stereoscopic Image Guided Surgical Navigation
US8052598B2 (en) * 2006-10-12 2011-11-08 General Electric Company Systems and methods for calibrating an endoscope
US7892165B2 (en) 2006-10-23 2011-02-22 Hoya Corporation Camera calibration for endoscope navigation system
US8864652B2 (en) 2008-06-27 2014-10-21 Intuitive Surgical Operations, Inc. Medical robotic system providing computer generated auxiliary views of a camera instrument for controlling the positioning and orienting of its tip
US10092169B2 (en) 2008-07-08 2018-10-09 Karl Storz Imaging, Inc. Solid state variable direction of view endoscope
DE102010040992A1 (de) 2010-09-17 2012-03-22 Henke-Sass, Wolf Gmbh Endoskop mit variabler Blickrichtung
SG11201501528PA (en) 2012-09-12 2015-03-30 Univ Singapore Device for inserting an implant and method of using the same
US20150238276A1 (en) * 2012-09-30 2015-08-27 M.S.T. Medical Surgery Technologies Ltd. Device and method for assisting laparoscopic surgery - directing and maneuvering articulating tool
CN104936566B (zh) 2012-09-30 2018-04-13 新加坡国立大学 生物可吸收的药剂洗脱通气管
DE102012111290A1 (de) 2012-11-22 2014-05-22 Karl Storz Gmbh & Co. Kg Ein Endoskop mit einstellbarer Blickrichtung
DE102013220945A1 (de) 2013-10-16 2015-04-30 Olympus Winter & Ibe Gmbh Endoskop mit einstellbarer Blickrichtung
WO2015064521A1 (ja) * 2013-10-30 2015-05-07 オリンパスメディカルシステムズ株式会社 撮像装置
WO2015151098A2 (en) * 2014-04-02 2015-10-08 M.S.T. Medical Surgery Technologies Ltd. An articulated structured light based-laparoscope
EP3273854B1 (en) 2015-03-26 2021-09-22 Universidade de Coimbra Systems for computer-aided surgery using intra-operative video acquired by a free moving camera
WO2017027638A1 (en) * 2015-08-10 2017-02-16 The Board Of Trustees Of The Leland Stanford Junior University 3d reconstruction and registration of endoscopic data
EP3130276B8 (en) 2015-08-12 2020-02-26 TransEnterix Europe Sàrl Endoscope with wide angle lens and adjustable view
US10176554B2 (en) 2015-10-05 2019-01-08 Google Llc Camera calibration using synthetic images
WO2017104427A1 (ja) * 2015-12-17 2017-06-22 オリンパス株式会社 内視鏡用光学アダプタ
US11344180B2 (en) * 2017-06-15 2022-05-31 Children's National Medical Center System, apparatus, and method for calibrating oblique-viewing rigid endoscope
EP3681450B1 (en) 2017-09-13 2024-09-04 National University of Singapore Device for incision and insertion of a ventilation tube
CN108938086A (zh) * 2018-06-08 2018-12-07 艾瑞迈迪医疗科技（北京）有限公司 内窥镜畸变矫正方法及手术导航设备
EP4647034A3 (en) * 2018-08-24 2025-12-10 Intuitive Surgical Operations, Inc. Off-camera calibration parameters for an image capture device
CN110335318B (zh) * 2019-04-28 2022-02-11 安翰科技(武汉)股份有限公司 一种基于摄像系统的消化道内物体测量方法
AU2020362217B2 (en) 2019-10-07 2025-05-29 S&N Orion Prime, S.A. Systems and methods for changing the direction of view during video guided clinical procedures using real-time image processing
WO2021168408A1 (en) 2020-02-20 2021-08-26 Smith & Nephew, Inc. Methods for arthroscopic video analysis and devices therefor
WO2021203082A1 (en) 2020-04-03 2021-10-07 Smith & Nephew, Inc. User interface for digital markers in arthroscopy
WO2021203077A1 (en) 2020-04-03 2021-10-07 Smith & Nephew, Inc. Methods for arthroscopic surgery video segmentation and devices therefor
US12343218B2 (en) 2020-06-18 2025-07-01 Smith & Nephew, Inc. Methods for autoregistration of arthroscopic video images to preoperative models and devices thereof
WO2022006248A1 (en) 2020-07-02 2022-01-06 Smith & Nephew, Inc. Optical tracking of objects in arthroscopic surgery
WO2022006041A1 (en) 2020-07-02 2022-01-06 Smith & Nephew, Inc. Methods and systems for treating femoroacetabular impingement
US12502222B2 (en) 2021-01-25 2025-12-23 Smith & Nephew, Inc. Systems and methods for fusing arthroscopic video data

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160048953A1 (en) * 2009-03-31 2016-02-18 Intuitive Surgical Operations, Inc Targets, fixtures, and workflows for calibrating an endoscopic camera
US20140285676A1 (en) * 2011-07-25 2014-09-25 Universidade De Coimbra Method and apparatus for automatic camera calibration using one or more images of a checkerboard pattern

Also Published As

Publication number Publication date
JP7727622B2 (ja) 2025-08-21
US20250089999A1 (en) 2025-03-20
EP4041048B1 (en) 2025-01-01
WO2021071991A1 (en) 2021-04-15
CN114727744A (zh) 2022-07-08
US20220383588A1 (en) 2022-12-01
EP4041048A4 (en) 2023-11-15
JP2022550913A (ja) 2022-12-05
US20220392110A1 (en) 2022-12-08
EP4041049A1 (en) 2022-08-17
EP4041048A1 (en) 2022-08-17
AU2020362216B2 (en) 2025-06-26
CN114727744B (zh) 2026-01-30
US12605049B2 (en) 2026-04-21
CN114630611A (zh) 2022-06-14
WO2021071988A1 (en) 2021-04-15
US12178401B2 (en) 2024-12-31
EP4041049A4 (en) 2024-01-10
US20250134360A1 (en) 2025-05-01
AU2025238014A1 (en) 2025-10-16
JP7604473B2 (ja) 2024-12-23
US12220104B2 (en) 2025-02-11
JP2023504344A (ja) 2023-02-03
CN114630611B (zh) 2026-03-31
AU2020362217A1 (en) 2022-04-07
AU2020362216A1 (en) 2022-04-14

Similar Documents

Publication Publication Date Title
AU2020362217B2 (en) Systems and methods for changing the direction of view during video guided clinical procedures using real-time image processing
JP7280188B2 (ja) 医療用拡大高解像度撮像方法及び撮像システム
JP7460631B2 (ja) デュアル画像センサを有する内視鏡
US12290235B2 (en) Wireless laparoscopic device with gimballed camera
US8911358B2 (en) Endoscopic vision system
EP2143374B1 (en) Solid state variable direction of view endoscope
US9289267B2 (en) Method and apparatus for minimally invasive surgery using endoscopes
CN114051387A (zh) 医学观察系统、控制装置和控制方法
JP7444065B2 (ja) 医療用観察システム、医療用観察装置及び医療用観察方法
EP4274506A1 (en) Registration degradation correction for surgical navigation procedures
US20140336461A1 (en) Surgical structured light system
JPWO2016199273A1 (ja) 内視鏡装置及び内視鏡装置の作動方法
JP2008532602A (ja) 外科手術ナビゲーションと顕微鏡による可視化の方法と装置
WO2025088159A1 (en) Endoscopic systems and methods for imaging an object
EP2031559A1 (en) Augmented visualization in two-dimensional images
JP2014175965A (ja) 手術用カメラ
CN115623163B (zh) 二维三维图像的采集与融合显示系统及方法
US20250255679A1 (en) Processing surgical data
GB2622893A (en) Processing surgical data

Legal Events

Date Code Title Description
FGA Letters patent sealed or granted (standard patent)