CN110811491A - Online disease identification endoscope with three-dimensional reconstruction function - Google Patents

Online disease identification endoscope with three-dimensional reconstruction function Download PDF

Info

Publication number
CN110811491A
CN110811491A CN201911231160.6A CN201911231160A CN110811491A CN 110811491 A CN110811491 A CN 110811491A CN 201911231160 A CN201911231160 A CN 201911231160A CN 110811491 A CN110811491 A CN 110811491A
Authority
CN
China
Prior art keywords
endoscope
dimensional
dimensional reconstruction
processing system
disease
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911231160.6A
Other languages
Chinese (zh)
Inventor
马晋平
杨杰
杨光谱
张天豪
王颖钊
陈松林
李少军
杨习锋
曾晨光
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Sun Shing Biotech Co ltd
First Affiliated Hospital of Sun Yat Sen University
Original Assignee
Guangzhou Sun Shing Biotech Co ltd
First Affiliated Hospital of Sun Yat Sen University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Sun Shing Biotech Co ltd, First Affiliated Hospital of Sun Yat Sen University filed Critical Guangzhou Sun Shing Biotech Co ltd
Priority to CN201911231160.6A priority Critical patent/CN110811491A/en
Publication of CN110811491A publication Critical patent/CN110811491A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/31Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for the rectum, e.g. proctoscopes, sigmoidoscopes, colonoscopes
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/005Flexible endoscopes
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A61B1/0661Endoscope light sources
    • A61B1/0676Endoscope light sources at distal tip of an endoscope
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B10/00Other methods or instruments for diagnosis, e.g. instruments for taking a cell sample, for biopsy, for vaccination diagnosis; Sex determination; Ovulation-period determination; Throat striking implements
    • A61B10/02Instruments for taking cell samples or for biopsy
    • A61B10/04Endoscopic instruments
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/40Analysis of texture

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Surgery (AREA)
  • Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • General Health & Medical Sciences (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Molecular Biology (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Optics & Photonics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Software Systems (AREA)
  • Geometry (AREA)
  • Computer Graphics (AREA)
  • Endoscopes (AREA)

Abstract

The invention provides an endoscope with a three-dimensional reconstruction function, which can perform real-time reconstruction aiming at three-dimensional forms of pathological changes in the process of in-line endoscopy based on a computer monocular vision mode, enriches the information acquisition modes of a standard case database of an endoscope system, improves the visualization level of the forms of the pathological changes, has strong compatibility, can perform more accurate disease matching by integrating the acquired three-dimensional forms of local characteristics of diseases into a database, and improves the efficiency of detecting the pathological changes in the process of clinical early screening. The invention relates to an online disease identification endoscope with a three-dimensional reconstruction function, which comprises an endoscope body, a display and an endoscope processing system, wherein the endoscope body is provided with a plurality of endoscope channels; the endoscope body and the display are connected with the endoscope processing system, the endoscope processing system generates a three-dimensional reconstruction image according to the received two-dimensional image sent by the endoscope body, the three-dimensional reconstruction image is matched with the standard case database information, and the disease type and position result is judged according to the matching degree.

Description

Online disease identification endoscope with three-dimensional reconstruction function
Technical Field
The invention relates to the technical field of medical instruments, in particular to an online disease identification endoscope with a three-dimensional reconstruction function.
Background
Colonoscopy is a diagnostic and therapeutic method for examining the pathological changes in the intestinal tract, and is suitable for diagnosing, treating and treating polyps, ulcerative colitis, chronic colitis, colon cancer and the like. 40.5 new gastric cancers and 32.5 deaths of gastric cancers in China are public health problems to be urgently solved. Research shows that if a patient can receive endoscopic examination and treatment in time at the early cancer stage, the five-year survival rate can be greatly improved (up to 90 percent), and the method has important significance for reducing the incidence rate and the death rate of gastric cancer and saving medical resources. At present, suspicious lesion can be detected by an endoscope for early cancer, and final judgment is carried out in a pathological mode. However, the doctor sees a local two-dimensional image in the current endoscopic process, and the overall shape, lesion and the like of the colon and rectum are not displayed comprehensively; the method of making slices/smears on pathology and observing whether cells become cancerous under a microscope is also time-consuming, which is not favorable for the detection efficiency and the detection cost of early cancer.
Disclosure of Invention
The invention provides an endoscope with a three-dimensional reconstruction function, which can perform real-time reconstruction aiming at three-dimensional forms of pathological changes in the process of in-line endoscopy based on a computer monocular vision mode, enriches the information acquisition modes of a standard case database of an endoscope system, improves the visualization level of the forms of the pathological changes, has strong compatibility, can perform more accurate disease matching by integrating the acquired three-dimensional forms of local characteristics of diseases into a database, and improves the efficiency of detecting the pathological changes in the process of clinical early screening.
The invention relates to an online disease identification endoscope with a three-dimensional reconstruction function, which comprises an endoscope body, a display and an endoscope processing system, wherein the endoscope body is provided with a plurality of endoscope channels; the endoscope body and the display are connected with the endoscope processing system, the endoscope processing system generates a three-dimensional reconstruction image according to the received two-dimensional image sent by the endoscope body, matches the three-dimensional reconstruction image with standard case database information, and judges disease types and position results according to matching degrees.
Preferably, the endoscope body consists of an endoscope distal end, an endoscope proximal end and an endoscope handle.
Preferably, the distal end of the endoscope is made of a soft material and has a multi-channel structure, and a camera, an illumination light source, a water outlet, an air outlet and a biopsy channel are respectively distributed on the section of the distal end of the endoscope.
Preferably, the endoscope proximal end and the endoscope distal end are connected, and are made of hard materials and have a multi-lumen structure.
The invention provides an endoscope with a three-dimensional reconstruction function, which can perform real-time reconstruction aiming at three-dimensional forms of pathological changes in the process of in-line endoscopy based on a computer monocular vision mode, enriches the acquisition modes of endoscope information, improves the visualization level of the forms of pathological changes, and can perform disease matching more accurately and quickly by integrating the acquired three-dimensional forms of local characteristics of diseases into a database, thereby improving the efficiency of detecting the pathological changes in the process of clinical early screening.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is an overall block diagram of an online disease recognition endoscope with a three-dimensional reconstruction function according to the present invention;
FIG. 2 is a schematic diagram of a process for three-dimensional reconstruction of an endoscopic disease morphology;
FIG. 3 is a diagram of endoscopic disease data collection and disease intelligence determination.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention relates to an online disease identification endoscope with a three-dimensional reconstruction function. Referring to fig. 1, the display is connected to the endoscope processing system, and the endoscope body is connected to the endoscope processing system.
The endoscope body consists of an endoscope far end, an endoscope near end and an endoscope handle.
The distal end of the endoscope is made of soft materials, is of a multi-cavity structure and at least has the function of turning in four directions in a single plane. The section of the distal end of the endoscope is respectively distributed with a camera, an illumination light source, a water outlet, an air outlet and a biopsy channel.
The endoscope near end is connected with the endoscope far end, is made of hard materials and has a multi-cavity structure. The near end of the endoscope mainly plays a role in rigid support and corresponding smoothness of light, electricity, gas, water paths and biopsy channels in the endoscopic examination process.
The endoscope handle mainly plays a role in controlling water, electricity, gas and light and adjusting the steering of the distal end of the endoscope.
The endoscope display mainly displays the shape of an endoscope insertion section in real time in the process of endoscopy, displays a two-dimensional image and a three-dimensional reconstruction image of related diseases, and displays the judgment result of the type and the position of the related diseases.
The endoscope processing system has the functions of collecting two-dimensional images and videos, integrating three-dimensional reconstruction model information by using a related algorithm, matching diseases and intelligently diagnosing diseases, and is specifically shown in fig. 2 and 3.
The specific process is as follows: the monocular vision mode of the computer is started in the endoscope body endoscope entering process, the reconstruction mode does not change the information obtaining mode of the existing mainstream endoscope, and the compatibility is strong. When suspicious lesions are found, a clinician can select the suspicious lesion positions in a key mode through an endoscope handle to acquire two-dimensional information of relevant positions (an endoscope camera acquires two-dimensional images and sends the two-dimensional images to an endoscope processing system), the endoscope processing system extracts image feature points and three-dimensional coordinates according to the acquired two-dimensional images, online three-dimensional reconstruction is automatically carried out after 3D point cloud data are extracted, colorectal color textures are generated based on color adjustment images, after reconstruction is completed, the endoscope processing system can match the two-dimensional images and three-dimensional reconstruction results with information in a standard case database, and intelligently judges and outputs disease types and position results, so that online auxiliary diagnosis of diseases is completed.
The disease standard case database can be based on the existing database, can also be automatically acquired based on clinical lesion data (including but not limited to two-dimensional and three-dimensional image information of lesion positions), and can introduce analysis results such as later-stage corresponding pathological smears/slices, metabolic mass spectrometry and the like to supplement, perfect and standardize the database data. The disease matching and intelligent judgment can be achieved through a disease recognition model, the model is mainly obtained through training based on an artificial intelligence deep learning network, a certain threshold value can be set according to the matching degree of the disease information, and when the matching degree of the intelligent diagnosis information and the standard case data exceeds or equals to the threshold value, the disease is judged to belong to; otherwise, if the matching degree of the intelligent diagnosis information and the standard case data is smaller than the threshold value, the endoscope system prompts manual intervention, and a clinician judges the disease and manually adds the disease to a corresponding disease database. The disease identification model can correct and perfect relevant model parameters by contacting new cases or feeding back the results of later intelligent diagnosis.
In addition, the endoscope processing system can complete the reconstruction of the integral colorectal track while entering the scene, corresponding electronic tags can be added on a colorectal model at the first time when a lesion is found, the electronic tags can be used for disease position backtracking after disease intelligent judgment, and certain help is provided for the collection of subsequent possible pathological samples.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (4)

1. An online disease identification endoscope with a three-dimensional reconstruction function is characterized by comprising an endoscope body, a display and an endoscope processing system; the endoscope body and the display are connected with the endoscope processing system, the endoscope processing system generates a three-dimensional reconstruction image according to the received two-dimensional image sent by the endoscope body, matches the three-dimensional reconstruction image with standard case database information, and judges disease types and position results according to matching degrees.
2. The endoscope of claim 1, wherein the endoscope body is comprised of an endoscope distal end, an endoscope proximal end, and an endoscope handle.
3. The endoscope of claim 2, wherein the distal end of the endoscope is made of a soft material and has a multi-lumen structure, and a camera, an illumination light source, a water outlet, an air outlet and a biopsy channel are respectively distributed on a cross section of the distal end of the endoscope.
4. An endoscope according to claim 2 and wherein said proximal and distal endoscope ends are joined and formed of a rigid material in a multilumen configuration.
CN201911231160.6A 2019-12-05 2019-12-05 Online disease identification endoscope with three-dimensional reconstruction function Pending CN110811491A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911231160.6A CN110811491A (en) 2019-12-05 2019-12-05 Online disease identification endoscope with three-dimensional reconstruction function

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911231160.6A CN110811491A (en) 2019-12-05 2019-12-05 Online disease identification endoscope with three-dimensional reconstruction function

Publications (1)

Publication Number Publication Date
CN110811491A true CN110811491A (en) 2020-02-21

Family

ID=69543840

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911231160.6A Pending CN110811491A (en) 2019-12-05 2019-12-05 Online disease identification endoscope with three-dimensional reconstruction function

Country Status (1)

Country Link
CN (1) CN110811491A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022190366A1 (en) * 2021-03-12 2022-09-15 オリンパス株式会社 Shape measurement system for endoscope and shape measurement method for endoscope

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011055245A1 (en) * 2009-11-04 2011-05-12 Koninklijke Philips Electronics N.V. Collision avoidance and detection using distance sensors
CN103040430A (en) * 2013-01-24 2013-04-17 北京理工大学 Three-dimensional endoscope apparatus
CN103984957A (en) * 2014-05-04 2014-08-13 中国科学院深圳先进技术研究院 Automatic early warning system for suspicious lesion area of capsule endoscope image
CN104434001A (en) * 2014-11-18 2015-03-25 中国科学院半导体研究所 Monocular endoscope system based on omnibearing three-dimensional stereovision
US20170046833A1 (en) * 2015-08-10 2017-02-16 The Board Of Trustees Of The Leland Stanford Junior University 3D Reconstruction and Registration of Endoscopic Data
CN108814537A (en) * 2018-04-09 2018-11-16 中北大学 Colon three-dimensionalreconstruction based on colon indagation microrobot realizes system and method
CN110049709A (en) * 2016-12-07 2019-07-23 奥林巴斯株式会社 Image processing apparatus
CN110367912A (en) * 2019-07-12 2019-10-25 深圳先进技术研究院 A kind of endoscope and endoscopic system

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011055245A1 (en) * 2009-11-04 2011-05-12 Koninklijke Philips Electronics N.V. Collision avoidance and detection using distance sensors
CN103040430A (en) * 2013-01-24 2013-04-17 北京理工大学 Three-dimensional endoscope apparatus
CN103984957A (en) * 2014-05-04 2014-08-13 中国科学院深圳先进技术研究院 Automatic early warning system for suspicious lesion area of capsule endoscope image
CN104434001A (en) * 2014-11-18 2015-03-25 中国科学院半导体研究所 Monocular endoscope system based on omnibearing three-dimensional stereovision
US20170046833A1 (en) * 2015-08-10 2017-02-16 The Board Of Trustees Of The Leland Stanford Junior University 3D Reconstruction and Registration of Endoscopic Data
CN110049709A (en) * 2016-12-07 2019-07-23 奥林巴斯株式会社 Image processing apparatus
CN108814537A (en) * 2018-04-09 2018-11-16 中北大学 Colon three-dimensionalreconstruction based on colon indagation microrobot realizes system and method
CN110367912A (en) * 2019-07-12 2019-10-25 深圳先进技术研究院 A kind of endoscope and endoscopic system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022190366A1 (en) * 2021-03-12 2022-09-15 オリンパス株式会社 Shape measurement system for endoscope and shape measurement method for endoscope
JP7462255B2 (en) 2021-03-12 2024-04-05 オリンパス株式会社 ENDOSCOPIC SHAPE MEASURING SYSTEM AND ENDOSCOPIC SHAPE MEASURING METHOD

Similar Documents

Publication Publication Date Title
CN110495847B (en) Advanced learning-based auxiliary diagnosis system and examination device for early cancer of digestive tract
CN110600122B (en) Digestive tract image processing method and device and medical system
Zhou et al. Polyp detection and radius measurement in small intestine using video capsule endoscopy
CN107145840B (en) Endoscope expert diagnosis knowledge embedded computer aided WCE sequence image data identification method
JP2020073081A (en) Image diagnosis assistance apparatus, learned model, image diagnosis assistance method, and image diagnosis assistance program
US20180263568A1 (en) Systems and Methods for Clinical Image Classification
WO2017027638A1 (en) 3d reconstruction and registration of endoscopic data
CN109948671B (en) Image classification method, device, storage medium and endoscopic imaging equipment
US20210406737A1 (en) System and methods for aggregating features in video frames to improve accuracy of ai detection algorithms
KR20200038121A (en) Endoscopic device and method for diagnosing gastric lesion based on gastric endoscopic image obtained in real time
CN111839428A (en) Method for improving detection rate of colonoscope adenomatous polyps based on deep learning
CN115049666A (en) Endoscope virtual biopsy device based on color wavelet covariance depth map model
JP2005349199A (en) Medical three-dimensional image display, three-dimensional image processing method, computer tomographic apparatus, work station and computer program product
CN105869151B (en) Tongue segmentation and tongue fur tongue nature separation method
CN110811491A (en) Online disease identification endoscope with three-dimensional reconstruction function
CN116030303B (en) Video colorectal lesion typing method based on semi-supervised twin network
CN117218127A (en) Ultrasonic endoscope auxiliary monitoring system and method
KR20220102172A (en) Control method, apparatus, and program of lesion determination system acquired through real-time image
CN106370618B (en) Method for separating human tongue and tongue coating by dual-wavelength spectral difference index
CN110811527A (en) Endoscope with shape estimation and disease online auxiliary diagnosis functions
Sun et al. RGB pixel brightness characteristics of linked color imaging in early gastric cancer: a pilot study
WO2019088008A1 (en) Image processing apparatus, image processing method, program, and endoscope system
Gupta et al. A review on deep learning-based polyp segmentation for efficient colorectal cancer screening
Cao et al. Transformer for computer-aided diagnosis of laryngeal carcinoma in pcle images
Baima et al. Dense Swin Transformer for Classification of Thyroid Nodules

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200221

RJ01 Rejection of invention patent application after publication