WO2022170562A1 - Procédé et système de navigation d'endoscope digestif - Google Patents

Procédé et système de navigation d'endoscope digestif Download PDF

Info

Publication number
WO2022170562A1
WO2022170562A1 PCT/CN2021/076523 CN2021076523W WO2022170562A1 WO 2022170562 A1 WO2022170562 A1 WO 2022170562A1 CN 2021076523 W CN2021076523 W CN 2021076523W WO 2022170562 A1 WO2022170562 A1 WO 2022170562A1
Authority
WO
WIPO (PCT)
Prior art keywords
displacement vector
images
digestive endoscope
consecutive frames
displacement
Prior art date
Application number
PCT/CN2021/076523
Other languages
English (en)
Chinese (zh)
Inventor
熊璟
谭敏
夏泽洋
谢高生
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Priority to PCT/CN2021/076523 priority Critical patent/WO2022170562A1/fr
Publication of WO2022170562A1 publication Critical patent/WO2022170562A1/fr

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B23/00—Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes
    • G09B23/28—Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes for medicine

Definitions

  • the present invention relates to the technical field of medical image processing, and more particularly, to a digestive endoscope navigation method and system.
  • Colonoscopy is one of the important methods for diagnosing malignant tumors in anorectal surgery.
  • the doctor controls the colonoscope to be inserted from the patient's anus.
  • the examination is divided into two stages: forward and backward.
  • the doctor looks for the cavity advancement lens based on clinical experience and colonoscopy images. , until the tail of the cecum is reached, and then the regression phase is performed to observe whether there are polyps or other lesions in the intestinal tract.
  • doctors In this traditional colonoscopy operation, doctors only rely on endoscopic imaging and their own experience to find the advancing lens in the center of the lumen.
  • colonoscopy robots such as microcapsule endoscopes
  • solutions for endoscopic navigation mainly include:
  • the principle of the contour recognition method is based on the structural characteristics of the colon itself, such as using the inherent colon ring shape of the colon to calculate the direction of its curvature, and finally determine the direction of the center of the lumen.
  • this texture analysis-based navigation method has the same disadvantages as the dark area extraction method, that is, the robustness is poor or even completely ineffective when the image is occluded or blurred.
  • the endoscope is too close to the intestinal wall, the light angle received by the endoscope head is too narrow, and the intestinal muscle lines and dark areas may even be confused.
  • Three-dimensional reconstruction method The principle of the three-dimensional reconstruction method is to obtain information such as brightness, contour, and feature points from the image, and finally estimate the approximate depth information, and use the deepest point as the direction of the lens movement.
  • this 3D reconstruction method mostly uses 2D image shadows to obtain depth information for reconstruction, so it is sensitive to illumination, and the final navigation direction error is also large.
  • the purpose of the present invention is to overcome the above-mentioned defects of the prior art, and to provide a digestive endoscope navigation method and system to assist in solving the problem of cavity loss that often occurs in traditional operations.
  • a digestive endoscope navigation method includes the following steps:
  • the twin neural network model is trained by using training data, wherein the training data reflects the correspondence between the distribution characteristics of the displacement vectors of two consecutive frames of digestive endoscope images and the motion pattern of the digestive endoscope, and each displacement vector is the relationship between the two consecutive frames of images. Feature points at the same location are connected;
  • the continuous video stream of digestive endoscope is acquired in real time, and two consecutive frames of images are input into the trained twin neural network model to identify the motion pattern of the digestive endoscope according to the distribution characteristics of the displacement vector, and calculate the position coordinates of the next frame of the corresponding motion pattern, and then Output motion trajectory.
  • a digestive endoscope navigation system includes:
  • Training module used to train the twin neural network model using the training data, wherein the training data reflects the correspondence between the distribution characteristics of the displacement vectors of two consecutive frames of images of the digestive endoscope and the motion pattern of the digestive endoscope, and each displacement vector is Two consecutive frames of images are connected with feature points at the same position;
  • Prediction module It is used to obtain the continuous video stream of digestive endoscope in real time, and input two consecutive frames of images into the trained twin neural network model to identify the movement pattern of digestive endoscope according to the distribution characteristics of displacement vectors, and calculate the next step of the corresponding movement pattern. Frame position coordinates, and then output the motion trajectory.
  • the present invention has the advantage that "learning" based on the data set is no longer dependent on features such as dark areas or contours of a single frame picture, and has better global adaptability;
  • the neural network is used to learn displacement features based on pure images, so that the global error is more controllable, thereby providing accurate digestive endoscopic navigation and positioning.
  • FIG. 1 is a flowchart of a digestive endoscope navigation method according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of a process of a digestive endoscope navigation method according to an embodiment of the present invention
  • FIG. 3 is a schematic diagram of a displacement vector extraction algorithm according to an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of a displacement vector prediction process based on a twin neural network according to an embodiment of the present invention
  • FIG. 5 is a schematic diagram of a process of calculating the position of the next frame when the digestive endoscope is in a forward posture according to an embodiment of the present invention.
  • the digestive endoscope navigation method provided by the present invention includes: fusing the key points (or feature points) of two frames of images, and using the displacement vector distribution of the fused images to carry out offline marking to obtain the true value of the training data; building a twin neural network for supervised Pre-training, the model weight is obtained after the training is completed; the trained model is used for testing, and the position coordinates of the next frame of the digestive endoscope lens are estimated according to the distribution of the output displacement vector, and then the complete motion trajectory is output to realize the navigation of the digestive endoscope.
  • the provided digestive endoscope navigation method includes the following steps.
  • Step S110 extracting displacement vectors of two consecutive frames of images in the video stream to construct training data.
  • the feature point refers to the two frames of images. similarities in .
  • the displacement of the lens can be obtained by finding the feature points of the two images, and the connection of the feature points of the two images about the same position constitutes the displacement vector.
  • the displacement vector extraction includes: first, using SURF (Speeded up Robust Features) feature point matching algorithm to extract feature points of two frames of images; The images of 50% transparency are also superimposed to obtain a fused image; the feature points of the two images are connected on the fused image to obtain multiple displacement vectors, and the different distributions of the displacement vectors represent different motion modes of the lens.
  • SURF Speeded up Robust Features
  • the movement of the digestive endoscope lens in the intestine is classified into three movement modes, such as forward posture, backward posture, and movement in the image plane.
  • the movement in the plane can be further subdivided into rotation and translation. .
  • Figure 3 (a) is the basic process of the displacement vector extraction algorithm, and finally a fusion image with displacement vector is obtained;
  • Figure 3 (b) is a schematic diagram of three types of motion modes, which are divided into forward, There are three basic modes of retreat and in-plane motion;
  • Figure 3(c) is an enlarged schematic diagram of the forward attitude.
  • the displacement vector can be extracted by the optical flow method.
  • the optical flow is the instantaneous speed of the pixel motion of the spatially moving object on the observation imaging plane.
  • the optical flow method describes the time domain of the adjacent frame pixels.
  • the optical flow field is the projection of the displacement of the moving object in the three-dimensional space on the two-dimensional image plane. Therefore, the local optical flow method can also be used to extract the displacement vector, and then follow-up work such as offline labeling of the data set is performed.
  • the training data set includes 6302 clear colonoscopy images obtained from the colonoscopy video stream, including the cases where the center of the lumen is visible and invisible, of which 5041 images (accounting for the total data set 80%) as training samples, and the remaining 1261 images as test samples.
  • Step S120 using the training data to train the Siamese neural network model with the goal of minimizing the set loss function.
  • the supervised learning of the deep neural network in the deep learning method requires the ground truth of the given data sample. For example, to identify a cat from a picture, the network model needs to be trained with a large number of positive and negative samples in order to correctly simulate it. A complex mapping function with weighting coefficients from the training data to the final target. Such a training method needs to know in advance whether the corresponding picture is a cat or a dog, and such a label is the true value of the sample.
  • the task of identifying a cat or dog from a picture is called a classification task, and the task of implicitly predicting the output of a value is called a regression task.
  • the difference between the predicted output of the network and the true label is measured using a loss function.
  • the twin neural network is preferably used for learning and training.
  • two consecutive images of the video stream are sequentially input in time series, and GoogleNet is used as the backbone network to extract image features respectively, and the classification module performs feature fusion and then Three motion modes are predicted, and the regression module directly calculates the angle and length of the feature.
  • the deep Siamese neural network constructed in this embodiment is divided into a classification module and a regression module, the classification module is responsible for the category output of the lens motion pattern, and the regression module predicts the distribution of the displacement vector in step S110.
  • the similarity of the distribution patterns of the displacement vectors is measured by the following three indicators: the coordinates of the feature points of the two frames of images, the length of the displacement vectors, and the angle ⁇ of the displacement vectors.
  • the displacement vector extraction algorithm in the above step S110 actually serves for offline labeling of the dataset required for network input. Since the continuous lens motion displacement needs to be estimated in the deep learning task of the present invention, the network needs to input the above-mentioned two frames of images at the same time, and the twin network is two neural networks with the same weight.
  • ⁇ coord , ⁇ angle , ⁇ length , ⁇ class are the weights of the corresponding items, which can be set as needed
  • x ij , y ij are the coordinate values of the feature points, and is the estimated feature point coordinate value
  • i represents the feature point index
  • j represents the image index of two consecutive frames
  • p i (c) represents the true value of the category (ie, the corresponding motion mode category), represents a category estimate.
  • ⁇ coord takes a small value of 0.1.
  • ⁇ angle and ⁇ length are both 0.5.
  • classification loss for example, the commonly used cross entropy loss is used
  • C represents the number of categories
  • ⁇ class for example, takes a larger weight value of 0.5.
  • the angle and length loss parts in the composite loss function no longer use a simple square loss, but use the Wasserstein distance to measure whether the two distributions are close or not, expressed as:
  • Wasserstein distance is used to measure the similarity of two distributions P 1 and P 2. Compared with JS divergence and KL divergence, it has more advantages. Even if the two distributions do not overlap or overlap very little, they can still reflect the distance of the distribution.
  • the present invention does not limit the specific structure of the twin neural network model, and the number of layers, the dimensions of input and output, etc. can be set as required.
  • the invention adopts the twin neural network model, takes two consecutive frames of images as input, and outputs the representation embedded in the high-dimensional space to compare the similarity of the displacement vector distribution patterns.
  • the representation of different labels can be maximized, and the minimum The representation of the same label is quantized, that is, in this way, the distances of the learned similar features are close, and the distances of different features are separated.
  • Step S130 calculating the position coordinates of the next frame.
  • the motion posture of the lens can be determined according to the distribution pattern of the displacement vector output by the twin neural network. Taking the forward posture as an example, the coordinates of the forward center need to be estimated.
  • the forward center is the projection of the forward position coordinates of the next frame of the lens on the image coordinate system.
  • the calculation method is to take the intersection of the inverse extension lines of each displacement vector. Further, after obtaining the forward center, the position coordinates of the next frame can be calculated according to the geometric relationship, as shown in FIG. 5 .
  • the forward center (x 2 , y 2 , z 2 ) can be obtained by taking the reverse extension of the displacement vector according to the vector distribution, the forward distance l Taking the mean value of the displacement vectors, the calculation formula of the position coordinates (x 3 , y 3 , z 3 ) of the next frame is expressed as:
  • the function d(p i1 , p i2 ) is used to calculate the distance of the feature matching point pair (ie the displacement vector), and p 1 and p 2 respectively represent the feature points extracted from two consecutive frames of images.
  • the length of the displacement vector is quite different from the angle, and the direction pointed by the arrow of the displacement vector can be gathered into a backward center.
  • the movement posture in the plane it can be divided into rotation and translation. At this time, the length of the displacement vector is not much different, and the vector connection cannot be gathered into the center. Specifically, it can be subdivided into the following three situations:
  • the rotation angle takes the maximum span angle
  • the difference between the two is the range of the rotation angle of the lens when performing the rotation movement.
  • the rotation center is the current position of the lens, and the result is the direction of the lens. Change;
  • an optional alternative to the maximum span angle for in-plane rotational motion is to average the angles or the median as the rotation angle.
  • the lens usually has both translation and rotation in the image plane. In this case, it can be considered to calculate the displacement length and angle of the translation motion first, and then calculate the change of the lens orientation caused by the rotation motion.
  • the present invention directly estimates the translation and rotation of the attitude transformation through the distribution of the displacement vector, and the translation and rotation can also be expressed by a 4 ⁇ 4 attitude transformation matrix T, and its elements can be obtained through neural network training.
  • the specific alternative is to set the weight of a certain layer in the network structure as the parameter of this pose matrix, and the weight can be updated with the back-propagation of the network during the training process, without the need to display the true value of the given matrix T to guide the training.
  • step S140 the complete motion trajectory of the lens is acquired, which is used for the navigation of the digestive endoscope.
  • the complete motion trajectory of the lens can be obtained by concatenating the position coordinate points of the next frame obtained in step S130 in series, which can be compared with the actual displacement trajectory in the verification stage to verify the feasibility of the present invention. Based on the connection direction between the position coordinate point of the next frame and the current position, it assists the doctor to perform surgery, or provides visual navigation for the movement of the colonoscopy robot.
  • the present invention also provides a digestive endoscope navigation system for implementing one or more aspects of the above method.
  • the system includes: a training module, which is used to train a twin neural network model by using training data, wherein the training data reflects the correspondence between the displacement vector distribution characteristics of two consecutive frames of digestive endoscope images and the movement pattern of digestive endoscope , each displacement vector is connected to the feature points of two consecutive frames of images about the same position; the prediction module, which is used to obtain the continuous video stream of digestive endoscopy in real time, inputs the two consecutive frames of images into the trained twin neural network model, to obtain according to Displacement vector distribution features identify the motion pattern of digestive endoscope, calculate the position coordinates of the next frame of the corresponding motion pattern, and then output the motion trajectory.
  • the present invention is based on the displacement vector extraction of the fusion of the key points of the two frames of images, combined with the twin neural network to predict the displacement vector, and then outputs the position coordinates of the next frame of the digestive endoscope, which provides a vision for digestive endoscopy, especially colonoscopy.
  • the navigation method can accurately identify the posture and movement direction of the digestive endoscope.
  • Two sets of bronchial model data with magnetic localization are used, including 1441 and 3333 internal moving images of the bronchial model and their corresponding 6-DOF rotation angles and camera space pose coordinates.
  • the output path of the deep convolutional neural network is consistent with the actual magnetic positioning space path to a certain extent, but with the increase of the path, the later errors are gradually accumulated, and outliers with large errors will appear.
  • the minimum error of the first set of data sets is 0.06144mm
  • the maximum error is 4.5234mm.
  • the prediction results fluctuate greatly, the minimum error is 0.0869mm, and the maximum error is 6.9547mm. But the error is still within the controllable range.
  • the present invention proposes a displacement vector extraction algorithm based on the fusion of key points of two frames of images, and at the same time builds a twin neural network model based on the algorithm to estimate the current motion mode of the digestive endoscope and give the position coordinates of the next frame, without Then rely on the local image to extract the contour of the dark area, so that the algorithm has better adaptability.
  • the lens motion pattern is learned from the video stream of colonoscopy surgery correctly operated by the doctor.
  • image processing technology was used to remove the specular effect caused by the specular reflection of the intestinal wall caused by flushing during the operation.
  • the features of two frames of images were extracted and fused to obtain the displacement vector.
  • the displacement vector of the neural network model was trained offline, and finally a set of digestive endoscopic navigation methods with better adaptability based on the deep learning method was formed.
  • the present invention may be a system, method and/or computer program product.
  • the computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the present invention.
  • a computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Non-exhaustive list of computer readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM) or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically coded devices, such as printers with instructions stored thereon Hole cards or raised structures in grooves, and any suitable combination of the above.
  • RAM random access memory
  • ROM read only memory
  • EPROM erasable programmable read only memory
  • flash memory static random access memory
  • SRAM static random access memory
  • CD-ROM compact disk read only memory
  • DVD digital versatile disk
  • memory sticks floppy disks
  • mechanically coded devices such as printers with instructions stored thereon Hole cards or raised structures in grooves, and any suitable combination of the above.
  • Computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (eg, light pulses through fiber optic cables), or through electrical wires transmitted electrical signals.
  • the computer readable program instructions described herein may be downloaded to various computing/processing devices from a computer readable storage medium, or to an external computer or external storage device over a network such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • a network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .
  • the computer program instructions for carrying out the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or instructions in one or more programming languages.
  • Source or object code written in any combination including object-oriented programming languages, such as Smalltalk, C++, Python, etc., and conventional procedural programming languages, such as the "C" language or similar programming languages.
  • the computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement.
  • the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through the Internet connect).
  • LAN local area network
  • WAN wide area network
  • custom electronic circuits such as programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs)
  • FPGAs field programmable gate arrays
  • PDAs programmable logic arrays
  • Computer readable program instructions are executed to implement various aspects of the present invention.
  • These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.
  • These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium on which the instructions are stored includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.
  • Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more functions for implementing the specified logical function(s) executable instructions.
  • the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are all equivalent.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Surgery (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Physics (AREA)
  • Pure & Applied Mathematics (AREA)
  • Computational Mathematics (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • Algebra (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Analysis (AREA)
  • Robotics (AREA)
  • Medicinal Chemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Endoscopes (AREA)

Abstract

Sont divulgués un procédé et un système de navigation d'endoscope digestif. Le procédé comprend : la fusion de points clés de deux trames d'images, et la réalisation d'un marquage hors ligne en utilisant une distribution de vecteur de déplacement d'une image fusionnée, afin d'obtenir une vraie valeur de données de formation ; la construction d'un réseau neuronal double pour effectuer la pré-formation supervisée ; et l'estimation des coordonnées de position de la trame suivante d'une lentille d'endoscopie digestive en utilisant une distribution de vecteur de déplacement émise par le modèle formé, et ensuite l'émission d'une trajectoire de mouvement complète, afin de réaliser la navigation d'un endoscope digestif. Grâce au procédé, la pose et le sens du mouvement d'un endoscope digestif lui-même peuvent être identifiés de manière précise, de sorte qu'une trajectoire de mouvement de celui-ci est saisie dans l'ensemble, et que l'adaptabilité est forte.
PCT/CN2021/076523 2021-02-10 2021-02-10 Procédé et système de navigation d'endoscope digestif WO2022170562A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/076523 WO2022170562A1 (fr) 2021-02-10 2021-02-10 Procédé et système de navigation d'endoscope digestif

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/076523 WO2022170562A1 (fr) 2021-02-10 2021-02-10 Procédé et système de navigation d'endoscope digestif

Publications (1)

Publication Number Publication Date
WO2022170562A1 true WO2022170562A1 (fr) 2022-08-18

Family

ID=82837452

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/076523 WO2022170562A1 (fr) 2021-02-10 2021-02-10 Procédé et système de navigation d'endoscope digestif

Country Status (1)

Country Link
WO (1) WO2022170562A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115953373A (zh) * 2022-12-22 2023-04-11 青岛创新奇智科技集团股份有限公司 玻璃缺陷检测方法、装置、电子设备及存储介质
CN117281616A (zh) * 2023-11-09 2023-12-26 武汉真彩智造科技有限公司 一种基于混合现实的手术控制方法及系统
CN117671012A (zh) * 2024-01-31 2024-03-08 临沂大学 术中内窥镜绝对与相对位姿计算的方法、装置及设备
CN117796745A (zh) * 2024-02-29 2024-04-02 四川大学 一种估计消化内镜镜头进退距离的方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180296281A1 (en) * 2017-04-12 2018-10-18 Bio-Medical Engineering (HK) Limited Automated steering systems and methods for a robotic endoscope
CN110111366A (zh) * 2019-05-06 2019-08-09 北京理工大学 一种基于多级损失量的端到端光流估计方法
CN110376605A (zh) * 2018-09-18 2019-10-25 北京京东尚科信息技术有限公司 地图构建方法、导航方法和装置
CN111415564A (zh) * 2020-03-02 2020-07-14 武汉大学 基于人工智能的胰腺超声内镜检查导航方法及系统
US20200297444A1 (en) * 2019-03-21 2020-09-24 The Board Of Trustees Of The Leland Stanford Junior University Systems and methods for localization based on machine learning
CN111915573A (zh) * 2020-07-14 2020-11-10 武汉楚精灵医疗科技有限公司 一种基于时序特征学习的消化内镜下病灶跟踪方法
CN112330729A (zh) * 2020-11-27 2021-02-05 中国科学院深圳先进技术研究院 图像深度预测方法、装置、终端设备及可读存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180296281A1 (en) * 2017-04-12 2018-10-18 Bio-Medical Engineering (HK) Limited Automated steering systems and methods for a robotic endoscope
CN110376605A (zh) * 2018-09-18 2019-10-25 北京京东尚科信息技术有限公司 地图构建方法、导航方法和装置
US20200297444A1 (en) * 2019-03-21 2020-09-24 The Board Of Trustees Of The Leland Stanford Junior University Systems and methods for localization based on machine learning
CN110111366A (zh) * 2019-05-06 2019-08-09 北京理工大学 一种基于多级损失量的端到端光流估计方法
CN111415564A (zh) * 2020-03-02 2020-07-14 武汉大学 基于人工智能的胰腺超声内镜检查导航方法及系统
CN111915573A (zh) * 2020-07-14 2020-11-10 武汉楚精灵医疗科技有限公司 一种基于时序特征学习的消化内镜下病灶跟踪方法
CN112330729A (zh) * 2020-11-27 2021-02-05 中国科学院深圳先进技术研究院 图像深度预测方法、装置、终端设备及可读存储介质

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"Thesis Submitted in Partial Fulfillment of the Requirementsfor the Degree of Master of Engineering Huazhong University of Science & Technology", 31 December 2018, HUAZHONG UNIVERSITY OF SCIENCE AND TECHNOLOGY, CN, article ZOU, TIANHAO: "Research on Data-driven Grid Background Suppression Algorithm under Dynamic Platform", pages: 1 - 65, XP009539003 *
SHI GUOQIANG: "Object tracking algorithm based on jointly-optimized strong-coupled Siamese region proposal network", JOURNAL OF COMPUTER APPLICATIONS, JISUANJI YINGYONG, CN, vol. 40, no. 10, 10 October 2020 (2020-10-10), CN , pages 2822 - 2830, XP055957890, ISSN: 1001-9081 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115953373A (zh) * 2022-12-22 2023-04-11 青岛创新奇智科技集团股份有限公司 玻璃缺陷检测方法、装置、电子设备及存储介质
CN115953373B (zh) * 2022-12-22 2023-12-15 青岛创新奇智科技集团股份有限公司 玻璃缺陷检测方法、装置、电子设备及存储介质
CN117281616A (zh) * 2023-11-09 2023-12-26 武汉真彩智造科技有限公司 一种基于混合现实的手术控制方法及系统
CN117281616B (zh) * 2023-11-09 2024-02-06 武汉真彩智造科技有限公司 一种基于混合现实的手术控制方法及系统
CN117671012A (zh) * 2024-01-31 2024-03-08 临沂大学 术中内窥镜绝对与相对位姿计算的方法、装置及设备
CN117671012B (zh) * 2024-01-31 2024-04-30 临沂大学 术中内窥镜绝对与相对位姿计算的方法、装置及设备
CN117796745A (zh) * 2024-02-29 2024-04-02 四川大学 一种估计消化内镜镜头进退距离的方法
CN117796745B (zh) * 2024-02-29 2024-05-03 四川大学 一种估计消化内镜镜头进退距离的方法

Similar Documents

Publication Publication Date Title
CN112766416B (zh) 一种消化内镜导航方法和系统
WO2022170562A1 (fr) Procédé et système de navigation d'endoscope digestif
Visentini-Scarzanella et al. Deep monocular 3D reconstruction for assisted navigation in bronchoscopy
CN108685560B (zh) 用于机器人内窥镜的自动化转向系统和方法
US10733745B2 (en) Methods, systems, and computer readable media for deriving a three-dimensional (3D) textured surface from endoscopic video
Lin et al. Video‐based 3D reconstruction, laparoscope localization and deformation recovery for abdominal minimally invasive surgery: a survey
Qin et al. Surgical instrument segmentation for endoscopic vision with data fusion of cnn prediction and kinematic pose
Shao et al. Pfrl: Pose-free reinforcement learning for 6d pose estimation
WO2022156425A1 (fr) Procédé et système de positionnement d'instrument chirurgical mini-invasif
Liu et al. Sage: slam with appearance and geometry prior for endoscopy
Rau et al. Bimodal camera pose prediction for endoscopy
van der Stap et al. Towards automated visual flexible endoscope navigation
Lee et al. Weakly supervised segmentation for real‐time surgical tool tracking
Rodríguez et al. Tracking monocular camera pose and deformation for SLAM inside the human body
Lepetit Recent advances in 3d object and hand pose estimation
Song et al. BDIS: Bayesian dense inverse searching method for real-time stereo surgical image matching
Van der Stap et al. The use of the focus of expansion for automated steering of flexible endoscopes
US20230267679A1 (en) Method and System for Reconstructing the Three-Dimensional Surface of Tubular Organs
Yang et al. 3D reconstruction from endoscopy images: A survey
Jin et al. A Self-supervised Approach for Detecting the Edges of Haustral Folds in Colonoscopy Video
Xu et al. Graph-based Pose Estimation of Texture-less Surgical Tools for Autonomous Robot Control
JP2024537198A (ja) 内視鏡的処置の品質検査を分析するためのコンピュータ実装システム及び方法
US11227406B2 (en) Fusing deep learning and geometric constraint for image-based localization
Jeong et al. Depth estimation of endoscopy using sim-to-real transfer
Nguyen et al. Improving the Hand Pose Estimation from Egocentric Vision via HOPE-Net and Mask R-CNN

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21925225

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 11.01.2024)