CN114025701B - 手术工具尖端和朝向确定 - Google Patents

手术工具尖端和朝向确定 Download PDF

Info

Publication number
CN114025701B
CN114025701B CN202080034221.4A CN202080034221A CN114025701B CN 114025701 B CN114025701 B CN 114025701B CN 202080034221 A CN202080034221 A CN 202080034221A CN 114025701 B CN114025701 B CN 114025701B
Authority
CN
China
Prior art keywords
surgical tool
tip
keypoints
keypoint
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202080034221.4A
Other languages
English (en)
Chinese (zh)
Other versions
CN114025701A (zh
Inventor
X·陈
K-K·A·黄
刘明昌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Group Corp
Original Assignee
Sony Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Group Corp filed Critical Sony Group Corp
Publication of CN114025701A publication Critical patent/CN114025701A/zh
Application granted granted Critical
Publication of CN114025701B publication Critical patent/CN114025701B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/03—Automatic limiting or abutting means, e.g. for safety
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/06—Measuring instruments not otherwise provided for
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/37—Surgical systems with images on a monitor during operation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/757—Matching configurations of points or features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/40—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/67—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for remote operation
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2055—Optical tracking systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2065—Tracking using image or pattern recognition
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/03—Automatic limiting or abutting means, e.g. for safety
    • A61B2090/033—Abutting means, stops, e.g. abutting on tissue or skin
    • A61B2090/036—Abutting means, stops, e.g. abutting on tissue or skin abutting on tissue or skin
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/06—Measuring instruments not otherwise provided for
    • A61B2090/061—Measuring instruments not otherwise provided for for measuring dimensions, e.g. length
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/06—Measuring instruments not otherwise provided for
    • A61B2090/067—Measuring instruments not otherwise provided for for measuring angles
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/08—Accessories or related features not otherwise provided for
    • A61B2090/0807—Indication means
    • A61B2090/0811—Indication means for the position of a particular part of an instrument with respect to the rest of the instrument, e.g. position of the anvil of a stapling instrument
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/361—Image-producing devices, e.g. surgical cameras
    • A61B2090/3612—Image-producing devices, e.g. surgical cameras with images taken automatically
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/37—Surgical systems with images on a monitor during operation
    • A61B2090/371—Surgical systems with images on a monitor during operation with simultaneous use of two cameras
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images
    • G06V2201/034—Recognition of patterns in medical or anatomical images of medical instruments

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Surgery (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Public Health (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Biomedical Technology (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Veterinary Medicine (AREA)
  • Animal Behavior & Ethology (AREA)
  • Molecular Biology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Primary Health Care (AREA)
  • Epidemiology (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Pathology (AREA)
  • Robotics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Urology & Nephrology (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Gynecology & Obstetrics (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)
  • User Interface Of Digital Computer (AREA)
CN202080034221.4A 2019-06-20 2020-06-10 手术工具尖端和朝向确定 Active CN114025701B (zh)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US16/447,514 US11399896B2 (en) 2019-06-20 2019-06-20 Surgical tool tip and orientation determination
US16/447,514 2019-06-20
PCT/IB2020/055457 WO2020254920A1 (en) 2019-06-20 2020-06-10 Surgical tool tip and orientation determination

Publications (2)

Publication Number Publication Date
CN114025701A CN114025701A (zh) 2022-02-08
CN114025701B true CN114025701B (zh) 2025-05-23

Family

ID=71094653

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202080034221.4A Active CN114025701B (zh) 2019-06-20 2020-06-10 手术工具尖端和朝向确定

Country Status (6)

Country Link
US (1) US11399896B2 (https=)
EP (1) EP3965685A1 (https=)
JP (1) JP7395125B2 (https=)
KR (2) KR20220005064A (https=)
CN (1) CN114025701B (https=)
WO (1) WO2020254920A1 (https=)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021177183A1 (ja) * 2020-03-05 2021-09-10 日本電気株式会社 監視装置、監視システム、監視方法およびプログラム記録媒体
US11593931B2 (en) * 2020-06-09 2023-02-28 Howmedica Osteonics Corp. Surgical kit inspection systems and methods for inspecting surgical kits having parts of different types
AU2022254686B2 (en) * 2021-04-06 2025-07-10 Digital Surgery Systems, Inc. System, method, and apparatus for tracking a tool via a digital surgical microscope
US12536250B2 (en) 2021-04-30 2026-01-27 Sony Group Corporation Electronic device and method for smoke level estimation
EP4691411A3 (en) * 2021-06-16 2026-02-25 Digital Surgery Limited Joint identification and pose estimation of surgical instruments
CN116687560A (zh) * 2023-06-06 2023-09-05 北京天智航医疗科技股份有限公司 一种膝关节置换手术辅助定位方法、电子设备及存储介质
CN116763461A (zh) * 2023-08-07 2023-09-19 上海微创医疗机器人（集团）股份有限公司 一种器械就位方法及系统
KR20250155673A (ko) * 2024-04-23 2025-10-31 인제대학교 산학협력단 증강현실 및 인공지능 기반의 척추 수술 가이드 방법 및 그 시스템

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2002951762A0 (en) * 2002-10-01 2002-10-17 Spinemed Australia Pty Limited Intervertebral disc restoration
US9867669B2 (en) * 2008-12-31 2018-01-16 Intuitive Surgical Operations, Inc. Configuration marker design and detection for instrument tracking
DE102007013807B4 (de) * 2007-03-22 2014-11-13 Siemens Aktiengesellschaft Verfahren zur Unterstützung der Navigation interventioneller Werkzeuge bei Durchführung von CT- bzw. MRT-gesteuerten Interventionen in einer vorgegebenen Interventionsebene
CN101438955A (zh) * 2008-12-25 2009-05-27 哈尔滨工业大学 一种导管的自动导航方法
CN102789651B (zh) * 2012-07-03 2015-03-11 河北工业大学 一种化工易泄漏部件的三维模型库建立方法
CA2933684A1 (en) 2012-12-14 2014-06-19 The Trustees Of Columbia University In The City Of New York Markerless tracking of robotic surgical tools
KR20140108047A (ko) 2013-02-28 2014-09-05 삼성전자주식회사 물체 추적 방법, 물체의 표시 상태 판단 방법 및 물체 추적이 가능한 제어 장치
US9905000B2 (en) 2015-02-19 2018-02-27 Sony Corporation Method and system for surgical tool localization during anatomical surgery
WO2016169361A1 (en) * 2015-04-22 2016-10-27 Bio-Medical Engineering (HK) Limited Robotic devices and systems for performing single incision procedures and natural orifice translumenal endoscopic surgical procedures, and methods of configuring robotic devices and systems
CN105232161B (zh) * 2015-10-16 2017-05-17 北京天智航医疗科技股份有限公司 一种手术机器人标志点识别定位方法
US10973585B2 (en) 2016-09-21 2021-04-13 Alcon Inc. Systems and methods for tracking the orientation of surgical tools
US10002424B2 (en) * 2016-09-22 2018-06-19 Sony Corporation Image processing system and method to reconstruct a three-dimensional (3D) anatomical surface
US10529088B2 (en) 2016-12-02 2020-01-07 Gabriel Fine Automatically determining orientation and position of medically invasive devices via image processing
CN108154552A (zh) * 2017-12-26 2018-06-12 中国科学院深圳先进技术研究院 一种立体腹腔镜三维模型重建方法和装置
US11813027B2 (en) * 2018-06-15 2023-11-14 Waldemar Link Gmbh & Co. Kg System and method for positioning a surgical tool

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Articulated Multi-Instrument 2-D Pose Estimation Using Fully Convolutional Networks;Du Xiaofei ET AL;Journal | [J] IEEE transactions on medical imaging;20180531;1276-1287 *
FSNet: Pose Estimation of Endoscopic Surgical Tools Using Feature Stacked Network;YaKui Chu ET AL;2019 The 9th International Workshop on Computer Science and Engineering;20190617;427-431页 *
WESIERSKI DANIEL ET AL."instrument detection and pose estimation with rigid part mixtures model in video-assisted surgeries".MEDICAL IMAGE ANALYSIS,OXFORD UNIVERSITY PRESS.2018,244-265. *

Also Published As

Publication number Publication date
CN114025701A (zh) 2022-02-08
WO2020254920A1 (en) 2020-12-24
KR20220005064A (ko) 2022-01-12
JP2022530795A (ja) 2022-07-01
JP7395125B2 (ja) 2023-12-11
US20200397509A1 (en) 2020-12-24
US11399896B2 (en) 2022-08-02
EP3965685A1 (en) 2022-03-16
KR20250034174A (ko) 2025-03-10

Similar Documents

Publication Publication Date Title
CN114025701B (zh) 手术工具尖端和朝向确定
US11625834B2 (en) Surgical scene assessment based on computer vision
US20250387175A1 (en) Context-awareness systems and methods for a computer-assisted surgical system
US12279841B2 (en) Robotic surgical safety via video processing
Jacob et al. Gestonurse: a robotic surgical nurse for handling surgical instruments in the operating room
JP2021531910A (ja) ロボット操作手術器具の位置を追跡システムおよび方法
EP3678571A1 (en) Surgical recognition system
CN112543940B (zh) 针对外科手术视频的主导工具检测系统
EP3373834A1 (en) Surgical system with training or assist functions
US12256890B2 (en) Systems and methods for guiding surgical procedures
KR102427736B1 (ko) 수술 영상 분석 기반의 복강경 카메라 조종 로봇 및 이를 이용한 복강경 카메라 촬영 뷰 조정 방법
US10963698B2 (en) Tool handedness determination for surgical videos
US20260031212A1 (en) Automatic content tagging in videos of minimally invasive surgeries
CN115624387B (zh) 缝合控制方法、控制系统、可读存储介质及机器人系统

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant