CN110974142A - Real-time synchronous endoscope lesion positioning method and system for confocal laser microscopy endoscope - Google Patents
Real-time synchronous endoscope lesion positioning method and system for confocal laser microscopy endoscope Download PDFInfo
- Publication number
- CN110974142A CN110974142A CN201911327592.7A CN201911327592A CN110974142A CN 110974142 A CN110974142 A CN 110974142A CN 201911327592 A CN201911327592 A CN 201911327592A CN 110974142 A CN110974142 A CN 110974142A
- Authority
- CN
- China
- Prior art keywords
- endoscope
- lesion
- confocal
- image
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/273—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for the upper alimentary canal, e.g. oesophagoscopes, gastroscopes
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
- A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/0002—Operational features of endoscopes provided with data storages
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00163—Optical arrangements
- A61B1/00188—Optical arrangements with focusing or zooming features
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Surgery (AREA)
- Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Pathology (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Biophysics (AREA)
- Physics & Mathematics (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Optics & Photonics (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Gastroenterology & Hepatology (AREA)
- Signal Processing (AREA)
- Endoscopes (AREA)
Abstract
The invention provides a real-time synchronous endoscope lesion positioning method and system of a confocal laser microscopy endoscope. The method comprises the steps of receiving a notice of observation start of the confocal micro-endoscope, synchronously pre-collecting a white light endoscope image of the current position of a probe of the confocal micro-endoscope in real time for caching, and calling a pre-trained neural network model to identify the part corresponding to the currently cached white light endoscope image; in the observation process of the confocal micro-endoscope, judging whether a notification that the confocal micro-endoscope finds a lesion is received or not in real time, if so, storing the current cached white light endoscope image into a lesion image file storage library, and performing associated storage with the current confocal micro-endoscope image; and if not, acquiring the corresponding white-light endoscope image to replace the cached white-light endoscope image when the probe of the confocal microscope starts to observe the notification next time, and continuously judging whether the notification that the confocal microscope finds the lesion is received or not in real time.
Description
Technical Field
The invention belongs to the field of real-time synchronous endoscope lesion positioning, and particularly relates to a real-time synchronous endoscope lesion positioning method and system of a confocal laser micro-endoscope.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
The confocal micro-endoscope can amplify the mucous membrane structure by 1000 times, observe the histological morphology in real time and accurately diagnose various digestive tract mucous membrane pathological changes. The inventor finds that when an endoscope doctor finishes observation of the confocal endoscope and prepares for biopsy, the inventor often finds that the position of lesion found under the confocal endoscope can not be determined under the white light endoscope, and the position can not be accurately positioned when biopsy is taken, so that misdiagnosis and missed diagnosis are caused. In clinical practice, the endoscopist sometimes marks the mucosa with a confocal probe by touching it hard, but this method often damages the mucosa and has limited effectiveness.
Disclosure of Invention
In order to solve the problems, the invention provides a confocal laser micro-endoscope real-time synchronous endoscope lesion positioning method and a confocal laser micro-endoscope real-time synchronous endoscope lesion positioning system.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention provides a real-time synchronous endoscope lesion positioning method of a confocal laser microscopy endoscope, which comprises the following steps:
receiving a notice of observing the confocal micro-endoscope, synchronously pre-collecting a white light endoscope image of the current position of a probe of the confocal micro-endoscope in real time for caching, and calling a pre-trained neural network model to identify the part corresponding to the currently cached white light endoscope image;
in the observation process of the confocal micro-endoscope, judging whether a notification that the confocal micro-endoscope finds a lesion is received or not in real time, if so, storing the current cached white light endoscope image into a lesion image file storage library, and performing associated storage with the current confocal micro-endoscope image; and if not, acquiring the corresponding white-light endoscope image to replace the cached white-light endoscope image when the probe of the confocal microscope starts to observe the notification next time, and continuously judging whether the notification that the confocal microscope finds the lesion is received or not in real time.
The second aspect of the present invention provides a confocal laser microscopy real-time synchronous endoscope lesion positioning system, comprising:
the real-time synchronous acquisition module is used for receiving a notice of observing the confocal micro-endoscope, synchronously pre-acquiring a white light endoscope image of the current position of a probe of the confocal micro-endoscope in real time for caching, and calling a pre-trained neural network model to identify the part corresponding to the currently cached white light endoscope image;
the detected lesion notification judging module is used for judging whether a detected lesion notification of the confocal micro-endoscope is received in real time in the observation process of the confocal micro-endoscope, if so, storing the current cached white light endoscope image into a lesion image file storage library and storing the current cached white light endoscope image in association with the current confocal micro-endoscope image; and if not, acquiring the corresponding white-light endoscope image to replace the cached white-light endoscope image when the probe of the confocal microscope starts to observe the notification next time, and continuously judging whether the notification that the confocal microscope finds the lesion is received or not in real time.
The invention has the beneficial effects that:
the method comprises the steps of synchronously pre-acquiring a white light endoscope image of the current position of a probe of the confocal micro-endoscope in real time for caching while receiving a notice of observing the confocal micro-endoscope, and simultaneously calling a pre-trained neural network model to identify the part corresponding to the currently cached white light endoscope image; in the process of observing the confocal micro-endoscope, judging whether a notification that the confocal micro-endoscope finds the pathological changes is received in real time, and storing the current cached white light endoscope image into a pathological change image file storage library and storing the white light endoscope image in association with the current confocal micro-endoscope image when the notification that the confocal micro-endoscope finds the pathological changes is received; when the notification that the confocal micro-endoscope finds the pathological changes is not received, acquiring a corresponding white-light endoscope image to replace the cached white-light endoscope image when the probe of the confocal micro-endoscope starts to observe the notification next time, and continuously judging whether the notification that the confocal micro-endoscope finds the pathological changes is received or not in real time;
according to the method, the corresponding incidence relation between the confocal micro-endoscope image and the white-light endoscope image is utilized, the white-light endoscope image is utilized to mark the observation position of the confocal probe, the accuracy of lesion location is improved, the condition that the mucosa is damaged by forcibly touching the mucosa by the confocal probe is avoided, the lesion part of the mucosa of the digestive tract is identified by utilizing the pre-trained neural network model, and the identification efficiency of the lesion part is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention.
FIG. 1 is a flowchart of a real-time synchronized endoscopic lesion localization method of confocal laser microscopy according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of a confocal laser microscopy real-time synchronized endoscope lesion location system according to an embodiment of the present invention.
Detailed Description
The invention is further described with reference to the following figures and examples.
It is to be understood that the following detailed description is exemplary and is intended to provide further explanation of the invention as claimed. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
Example 1
FIG. 1 is a flowchart of a real-time synchronized endoscopic lesion localization method of confocal laser microscopy according to this embodiment.
The detailed scheme of the real-time synchronous endoscope lesion location method of the confocal laser microscopy endoscope of the embodiment is given below with reference to fig. 1:
as shown in fig. 1, the method for real-time synchronized endoscopic lesion location in confocal laser microscopy according to this embodiment at least includes:
step S101: and receiving a notice of observing the confocal micro-endoscope, synchronously pre-acquiring a white light endoscope image of the current position of the confocal micro-endoscope probe in real time for caching, and calling a pre-trained neural network model to identify the part corresponding to the currently cached white light endoscope image.
In the step, when the confocal micro-endoscope starts to observe, the confocal micro-endoscope works and collects images of the confocal micro-endoscope in real time; when the notification that the confocal micro-endoscope starts to observe is received, a white light endoscope image of the current position of the confocal micro-endoscope probe is synchronously pre-acquired in real time, so that the synchronization of the confocal micro-endoscope image and the white light endoscope image is realized, and data support is provided for the subsequent identification of the lesion part of the alimentary tract mucous membrane.
In specific implementation, the white light endoscope image synchronously pre-acquired in real time at the current position of the confocal microscopy endoscope probe is associated and cached with a preset unique identifier.
The unique identifier may be any one or any combination of numbers, letters or other symbols. According to the embodiment, the white light endoscopic images are cached in the association mode through the unique identifiers, so that the accuracy of later-period white light endoscopic image retrieval and the cache ordering are facilitated, and meanwhile the rapidity of white light endoscopic image retrieval is improved.
In specific implementation, the input end of the neural network model inputs the white light endoscopic image and outputs a part corresponding to the currently cached white light endoscopic image; the training process comprises the following steps:
dividing the white light endoscope images marked with a preset number of known positions into two groups, namely a training set and a test set;
and training the convolutional neural network model by using the training set, screening out the convolutional neural network model with the optimal parameters by using the test set, and finishing the training of the convolutional neural network model.
The convolutional neural network has the characteristic learning ability and can carry out translation invariant classification on input information according to the hierarchical structure of the convolutional neural network.
Step S102: in the observation process of the confocal micro-endoscope, judging whether a notification that the confocal micro-endoscope finds a lesion is received or not in real time, if so, storing the current cached white light endoscope image into a lesion image file storage library, and performing associated storage with the current confocal micro-endoscope image; and if not, acquiring the corresponding white-light endoscope image to replace the cached white-light endoscope image when the probe of the confocal microscope starts to observe the notification next time, and continuously judging whether the notification that the confocal microscope finds the lesion is received or not in real time.
In specific implementation, when the notification that the confocal micro-endoscope finds the lesion is judged not to be received, the collected white light endoscope image of the current position of the confocal micro-endoscope probe at the current moment is cached after the unique identifier is updated.
The embodiment associates the white light endoscope image with the unique identifier, thereby avoiding the phenomenon that the white light endoscope image is stored wrongly, which affects the later image retrieval and affects the accuracy of the whole lesion location.
As a specific implementation manner, in the observation process of the confocal micro-endoscope, before determining whether a notification of finding a lesion by the confocal micro-endoscope is received in real time, the method further includes:
and identifying the digestive tract mucous membrane lesion of the confocal micro-endoscope image acquired in the confocal micro-endoscope observation process.
Specifically, the process of identifying the lesion of the alimentary tract mucosa of a confocal micro-endoscope image acquired in the observation process of the confocal micro-endoscope comprises the following steps:
training a digestive tract mucous membrane lesion recognition model, and outputting a result that a current confocal micro-endoscope image has digestive tract mucous membrane lesion by using the trained digestive tract mucous membrane lesion recognition model; the digestive tract mucous membrane lesion identification model is a preset neural network model; for example: the BP neural network comprises an input layer, a connecting layer and an output layer, wherein the input layer comprises 3 input quantities which are respectively input with color characteristics, texture characteristics and shape characteristics; the output layer is 0 or 1, and when the output layer is 0, the digestive tract mucous membrane lesion is not identified; when 1, the explanation identifies the lesion of the mucosa of the digestive tract.
In specific implementation, the process of training the digestive tract mucosal lesion recognition model comprises the following steps:
acquiring a plurality of confocal micro-endoscope images with digestive tract mucous membrane lesion, extracting color characteristics, texture characteristics and shape characteristics of the confocal micro-endoscope images and taking the color characteristics, the texture characteristics and the shape characteristics as training characteristic sets;
and inputting the training feature set into the digestive tract mucous membrane lesion recognition model for training until the accuracy of the digestive tract mucous membrane lesion result is judged to meet the preset requirement, and finishing the digestive tract mucous membrane lesion recognition model.
According to the embodiment, the trained digestive tract mucous membrane lesion recognition model is utilized, whether digestive tract mucous membrane lesions exist or not is automatically recognized, and the accuracy and the speed of recognition of the whole digestive tract mucous membrane lesions are improved.
In the embodiment, while the notification of the start of observation of the confocal micro-endoscope is received, a white light endoscope image of the current position of a probe of the confocal micro-endoscope is synchronously pre-acquired in real time for caching, and a pre-trained neural network model is called to identify the part corresponding to the currently cached white light endoscope image; in the process of observing the confocal micro-endoscope, judging whether a notification that the confocal micro-endoscope finds the pathological changes is received in real time, and storing the current cached white light endoscope image into a pathological change image file storage library and storing the white light endoscope image in association with the current confocal micro-endoscope image when the notification that the confocal micro-endoscope finds the pathological changes is received; when the notification that the confocal micro-endoscope finds the pathological changes is not received, acquiring a corresponding white-light endoscope image to replace the cached white-light endoscope image when the probe of the confocal micro-endoscope starts to observe the notification next time, and continuously judging whether the notification that the confocal micro-endoscope finds the pathological changes is received or not in real time; in the embodiment, the position of the confocal probe observed is identified by utilizing the white light endoscope image according to the corresponding incidence relation between the confocal micro-endoscope image and the white light endoscope image, so that the accuracy of lesion positioning is improved, the condition that the confocal probe is used for forcibly touching the mucosa to mark the injury of the mucosa is avoided, the lesion part of the mucosa of the digestive tract is identified by utilizing the pre-trained neural network model, and the recognition efficiency of the lesion part is improved.
Example 2
Fig. 2 is a schematic structural diagram of a confocal laser microscopy real-time synchronous endoscope lesion location system according to the embodiment.
The detailed scheme of the real-time synchronous endoscope lesion positioning system of the confocal laser microscopy endoscope of the embodiment is given below with reference to fig. 2:
as shown in fig. 2, the real-time synchronized endoscopic lesion positioning system of confocal laser microscopy according to the present embodiment at least includes:
(1) and the real-time synchronous acquisition module is used for receiving a notice of observing the confocal micro-endoscope, synchronously pre-acquiring a white light endoscope image of the current position of the probe of the confocal micro-endoscope in real time for caching, and calling a pre-trained neural network model to identify the part corresponding to the currently cached white light endoscope image.
When the confocal micro-endoscope starts to observe, the confocal micro-endoscope works and collects images of the confocal micro-endoscope in real time; when the notification that the confocal micro-endoscope starts to observe is received, a white light endoscope image of the current position of the confocal micro-endoscope probe is synchronously pre-acquired in real time, so that the synchronization of the confocal micro-endoscope image and the white light endoscope image is realized, and data support is provided for the subsequent identification of the lesion part of the alimentary tract mucous membrane.
In specific implementation, the white light endoscope image synchronously pre-acquired in real time at the current position of the confocal microscopy endoscope probe is associated and cached with a preset unique identifier.
The unique identifier may be any one or any combination of numbers, letters or other symbols. According to the embodiment, the white light endoscopic images are cached in the association mode through the unique identifiers, so that the accuracy of later-period white light endoscopic image retrieval and the cache ordering are facilitated, and meanwhile the rapidity of white light endoscopic image retrieval is improved.
In specific implementation, the input end of the neural network model inputs the white light endoscopic image and outputs a part corresponding to the currently cached white light endoscopic image; the training process comprises the following steps:
dividing the white light endoscope images marked with a preset number of known positions into two groups, namely a training set and a test set;
and training the convolutional neural network model by using the training set, screening out the convolutional neural network model with the optimal parameters by using the test set, and finishing the training of the convolutional neural network model.
The convolutional neural network has the characteristic learning ability and can carry out translation invariant classification on input information according to the hierarchical structure of the convolutional neural network.
(2) The detected lesion notification judging module is used for judging whether a detected lesion notification of the confocal micro-endoscope is received in real time in the observation process of the confocal micro-endoscope, if so, storing the current cached white light endoscope image into a lesion image file storage library and storing the current cached white light endoscope image in association with the current confocal micro-endoscope image; and if not, acquiring the corresponding white-light endoscope image to replace the cached white-light endoscope image when the probe of the confocal microscope starts to observe the notification next time, and continuously judging whether the notification that the confocal microscope finds the lesion is received or not in real time.
In specific implementation, when the notification that the confocal micro-endoscope finds the lesion is judged not to be received, the collected white light endoscope image of the current position of the confocal micro-endoscope probe at the current moment is cached after the unique identifier is updated.
The embodiment associates the white light endoscope image with the unique identifier, thereby avoiding the phenomenon that the white light endoscope image is stored wrongly, which affects the later image retrieval and affects the accuracy of the whole lesion location.
As a specific implementation manner, in the observation process of the confocal micro-endoscope, before determining whether a notification of finding a lesion by the confocal micro-endoscope is received in real time, the method further includes:
and identifying the digestive tract mucous membrane lesion of the confocal micro-endoscope image acquired in the confocal micro-endoscope observation process.
Specifically, the process of identifying the lesion of the alimentary tract mucosa of a confocal micro-endoscope image acquired in the observation process of the confocal micro-endoscope comprises the following steps:
training a digestive tract mucous membrane lesion recognition model, and outputting a result that a current confocal micro-endoscope image has digestive tract mucous membrane lesion by using the trained digestive tract mucous membrane lesion recognition model; the digestive tract mucous membrane lesion identification model is a preset neural network model; for example: the BP neural network comprises an input layer, a connecting layer and an output layer, wherein the input layer comprises 3 input quantities which are respectively input with color characteristics, texture characteristics and shape characteristics; the output layer is 0 or 1, and when the output layer is 0, the digestive tract mucous membrane lesion is not identified; when 1, the explanation identifies the lesion of the mucosa of the digestive tract.
In specific implementation, the process of training the digestive tract mucosal lesion recognition model comprises the following steps:
acquiring a plurality of confocal micro-endoscope images with digestive tract mucous membrane lesion, extracting color characteristics, texture characteristics and shape characteristics of the confocal micro-endoscope images and taking the color characteristics, the texture characteristics and the shape characteristics as training characteristic sets;
and inputting the training feature set into the digestive tract mucous membrane lesion recognition model for training until the accuracy of the digestive tract mucous membrane lesion result is judged to meet the preset requirement, and finishing the digestive tract mucous membrane lesion recognition model.
According to the embodiment, the trained digestive tract mucous membrane lesion recognition model is utilized, whether digestive tract mucous membrane lesions exist or not is automatically recognized, and the accuracy and the speed of recognition of the whole digestive tract mucous membrane lesions are improved.
In the embodiment, while the notification of the start of observation of the confocal micro-endoscope is received, a white light endoscope image of the current position of a probe of the confocal micro-endoscope is synchronously pre-acquired in real time for caching, and a pre-trained neural network model is called to identify the part corresponding to the currently cached white light endoscope image; in the process of observing the confocal micro-endoscope, judging whether a notification that the confocal micro-endoscope finds the pathological changes is received in real time, and storing the current cached white light endoscope image into a pathological change image file storage library and storing the white light endoscope image in association with the current confocal micro-endoscope image when the notification that the confocal micro-endoscope finds the pathological changes is received; when the notification that the confocal micro-endoscope finds the pathological changes is not received, acquiring a corresponding white-light endoscope image to replace the cached white-light endoscope image when the probe of the confocal micro-endoscope starts to observe the notification next time, and continuously judging whether the notification that the confocal micro-endoscope finds the pathological changes is received or not in real time; in the embodiment, the position of the confocal probe observed is identified by utilizing the white light endoscope image according to the corresponding incidence relation between the confocal micro-endoscope image and the white light endoscope image, so that the accuracy of lesion positioning is improved, the condition that the confocal probe is used for forcibly touching the mucosa to mark the injury of the mucosa is avoided, the lesion part of the mucosa of the digestive tract is identified by utilizing the pre-trained neural network model, and the recognition efficiency of the lesion part is improved.
Example 3
A computer-readable storage medium, having stored thereon a computer program, wherein the program, when executed by a processor, performs the steps of a method for real-time synchronized endoscopic lesion localization in a confocal laser microscopy as illustrated in fig. 1.
In the embodiment, while the notification of the start of observation of the confocal micro-endoscope is received, a white light endoscope image of the current position of a probe of the confocal micro-endoscope is synchronously pre-acquired in real time for caching, and a pre-trained neural network model is called to identify the part corresponding to the currently cached white light endoscope image; in the process of observing the confocal micro-endoscope, judging whether a notification that the confocal micro-endoscope finds the pathological changes is received in real time, and storing the current cached white light endoscope image into a pathological change image file storage library and storing the white light endoscope image in association with the current confocal micro-endoscope image when the notification that the confocal micro-endoscope finds the pathological changes is received; when the notification that the confocal micro-endoscope finds the pathological changes is not received, acquiring a corresponding white-light endoscope image to replace the cached white-light endoscope image when the probe of the confocal micro-endoscope starts to observe the notification next time, and continuously judging whether the notification that the confocal micro-endoscope finds the pathological changes is received or not in real time; in the embodiment, the position of the confocal probe observed is identified by utilizing the white light endoscope image according to the corresponding incidence relation between the confocal micro-endoscope image and the white light endoscope image, so that the accuracy of lesion positioning is improved, the condition that the confocal probe is used for forcibly touching the mucosa to mark the injury of the mucosa is avoided, the lesion part of the mucosa of the digestive tract is identified by utilizing the pre-trained neural network model, and the recognition efficiency of the lesion part is improved.
Example 4
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method for real-time synchronized endoscopic lesion localization in confocal laser microscopy as shown in fig. 1 when executing the program.
In the embodiment, while the notification of the start of observation of the confocal micro-endoscope is received, a white light endoscope image of the current position of a probe of the confocal micro-endoscope is synchronously pre-acquired in real time for caching, and a pre-trained neural network model is called to identify the part corresponding to the currently cached white light endoscope image; in the process of observing the confocal micro-endoscope, judging whether a notification that the confocal micro-endoscope finds the pathological changes is received in real time, and storing the current cached white light endoscope image into a pathological change image file storage library and storing the white light endoscope image in association with the current confocal micro-endoscope image when the notification that the confocal micro-endoscope finds the pathological changes is received; when the notification that the confocal micro-endoscope finds the pathological changes is not received, acquiring a corresponding white-light endoscope image to replace the cached white-light endoscope image when the probe of the confocal micro-endoscope starts to observe the notification next time, and continuously judging whether the notification that the confocal micro-endoscope finds the pathological changes is received or not in real time; in the embodiment, the position of the confocal probe observed is identified by utilizing the white light endoscope image according to the corresponding incidence relation between the confocal micro-endoscope image and the white light endoscope image, so that the accuracy of lesion positioning is improved, the condition that the confocal probe is used for forcibly touching the mucosa to mark the injury of the mucosa is avoided, the lesion part of the mucosa of the digestive tract is identified by utilizing the pre-trained neural network model, and the recognition efficiency of the lesion part is improved.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A confocal laser microscopy endoscope real-time synchronous endoscope lesion positioning method is characterized by comprising the following steps:
receiving a notice of observing the confocal micro-endoscope, synchronously pre-collecting a white light endoscope image of the current position of a probe of the confocal micro-endoscope in real time for caching, and calling a pre-trained neural network model to identify the part corresponding to the currently cached white light endoscope image;
in the observation process of the confocal micro-endoscope, judging whether a notification that the confocal micro-endoscope finds a lesion is received or not in real time, if so, storing the current cached white light endoscope image into a lesion image file storage library, and performing associated storage with the current confocal micro-endoscope image; and if not, acquiring the corresponding white-light endoscope image to replace the cached white-light endoscope image when the probe of the confocal microscope starts to observe the notification next time, and continuously judging whether the notification that the confocal microscope finds the lesion is received or not in real time.
2. The method for real-time synchronized endoscopic lesion localization of confocal laser endomicroscopy according to claim 1, wherein the white light endoscopic image of the current position of the real-time synchronized pre-captured confocal endomicroscopy probe is cached in association with a predetermined unique identifier.
3. The method for real-time synchronized endoscopic lesion localization in confocal laser endomicroscopy according to claim 1, wherein when it is determined that no notification of lesion discovery in the confocal endomicroscopy is received, the collected white light endoscopic image of the current position of the probe of the confocal laser endomicroscopy at the current time is cached after updating the unique identifier.
4. The method for real-time synchronized endoscope lesion location in confocal laser microscopy according to claim 1, wherein during the observation of the confocal laser microscopy, before determining whether a notification of lesion discovery is received from the confocal laser microscopy in real time, the method further comprises:
and identifying the digestive tract mucous membrane lesion of the confocal micro-endoscope image acquired in the confocal micro-endoscope observation process.
5. The confocal laser endomicroscopy real-time synchronous endoscopic lesion localization method according to claim 4, wherein the process of identifying the gastrointestinal mucosal lesion of the confocal endomicroscopy image acquired in the confocal laser endomicroscopy observation process comprises:
training a digestive tract mucous membrane lesion recognition model, and outputting a result that a current confocal micro-endoscope image has digestive tract mucous membrane lesion by using the trained digestive tract mucous membrane lesion recognition model; the digestive tract mucous membrane lesion identification model is a preset neural network model;
the process of training the digestive tract mucous membrane lesion recognition model comprises the following steps:
acquiring a plurality of confocal micro-endoscope images with digestive tract mucous membrane lesion, extracting color characteristics, texture characteristics and shape characteristics of the confocal micro-endoscope images and taking the color characteristics, the texture characteristics and the shape characteristics as training characteristic sets;
and inputting the training feature set into the digestive tract mucous membrane lesion recognition model for training until the accuracy of the digestive tract mucous membrane lesion result is judged to meet the preset requirement, and finishing the digestive tract mucous membrane lesion recognition model.
6. The utility model provides a real-time synchronization scope pathological change positioning system of confocal laser micro-endoscope which characterized in that includes:
the real-time synchronous acquisition module is used for receiving a notice of observing the confocal micro-endoscope, synchronously pre-acquiring a white light endoscope image of the current position of a probe of the confocal micro-endoscope in real time for caching, and calling a pre-trained neural network model to identify the part corresponding to the currently cached white light endoscope image;
the detected lesion notification judging module is used for judging whether a detected lesion notification of the confocal micro-endoscope is received in real time in the observation process of the confocal micro-endoscope, if so, storing the current cached white light endoscope image into a lesion image file storage library and storing the current cached white light endoscope image in association with the current confocal micro-endoscope image; and if not, acquiring the corresponding white-light endoscope image to replace the cached white-light endoscope image when the probe of the confocal microscope starts to observe the notification next time, and continuously judging whether the notification that the confocal microscope finds the lesion is received or not in real time.
7. The confocal laser endomicroscopy real-time synchronized endoscopic lesion localization system according to claim 6, wherein the real-time synchronous acquisition module is configured to buffer the white light endoscopic image of the current real-time synchronous pre-acquired confocal endomicroscopy probe position with a predetermined unique identifier.
8. The confocal laser endomicroscopy real-time synchronized endoscopic lesion localization system according to claim 6, wherein in the lesion discovery notification judgment module, when it is judged that the confocal endomicroscopy lesion discovery notification is not received, the collected white light endomicroscopy image of the current position of the confocal endomicroscopy probe at the current time is cached after updating the unique identifier.
9. The confocal laser endomicroscopy real-time synchronized endoscope lesion location system according to claim 6, wherein in the lesion discovery notification determining module, during observation of the confocal endomicroscopy, determining whether a lesion discovery notification is received in real time, further comprises:
and identifying the digestive tract mucous membrane lesion of the confocal micro-endoscope image acquired in the confocal micro-endoscope observation process.
10. The confocal laser endomicroscopy real-time synchronous endoscopic lesion location system according to claim 9, wherein in the lesion discovery notification judgment module, the process of identifying the gastrointestinal mucosal lesion of the confocal endomicroscopy image obtained in the confocal laser endomicroscopy observation process comprises:
training a digestive tract mucous membrane lesion recognition model, and outputting a result that a current confocal micro-endoscope image has digestive tract mucous membrane lesion by using the trained digestive tract mucous membrane lesion recognition model; the digestive tract mucous membrane lesion identification model is a preset neural network model;
the process of training the digestive tract mucous membrane lesion recognition model comprises the following steps:
acquiring a plurality of confocal micro-endoscope images with digestive tract mucous membrane lesion, extracting color characteristics, texture characteristics and shape characteristics of the confocal micro-endoscope images and taking the color characteristics, the texture characteristics and the shape characteristics as training characteristic sets;
and inputting the training feature set into the digestive tract mucous membrane lesion recognition model for training until the accuracy of the digestive tract mucous membrane lesion result is judged to meet the preset requirement, and finishing the digestive tract mucous membrane lesion recognition model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911327592.7A CN110974142B (en) | 2019-12-20 | 2019-12-20 | Real-time synchronous endoscope lesion positioning system of confocal laser microscopy endoscope |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911327592.7A CN110974142B (en) | 2019-12-20 | 2019-12-20 | Real-time synchronous endoscope lesion positioning system of confocal laser microscopy endoscope |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110974142A true CN110974142A (en) | 2020-04-10 |
| CN110974142B CN110974142B (en) | 2020-08-18 |
Family
ID=70073633
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911327592.7A Active CN110974142B (en) | 2019-12-20 | 2019-12-20 | Real-time synchronous endoscope lesion positioning system of confocal laser microscopy endoscope |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110974142B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101637379A (en) * | 2008-07-31 | 2010-02-03 | 奥林巴斯医疗株式会社 | Image display apparatus, endoscope system using the same, and image display method |
| US20100245552A1 (en) * | 2009-03-26 | 2010-09-30 | Olympus Corporation | Image processing device, imaging device, computer-readable storage medium, and image processing method |
| CN109146884A (en) * | 2018-11-16 | 2019-01-04 | 青岛美迪康数字工程有限公司 | Endoscopy monitoring method and device |
| CN109222865A (en) * | 2018-10-17 | 2019-01-18 | 卓外（上海）医疗电子科技有限公司 | A kind of multi-modality imaging endoscopic system |
| CN109460717A (en) * | 2018-10-22 | 2019-03-12 | 青岛美迪康数字工程有限公司 | Alimentary canal Laser scanning confocal microscope lesion image-recognizing method and device |
| CN109523522A (en) * | 2018-10-30 | 2019-03-26 | 腾讯科技（深圳）有限公司 | Processing method, device, system and the storage medium of endoscopic images |
| CN110325100A (en) * | 2017-03-01 | 2019-10-11 | 富士胶片株式会社 | Endoscopic system and its operating method |
| CN110495847A (en) * | 2019-08-23 | 2019-11-26 | 重庆天如生物科技有限公司 | Alimentary canal morning cancer assistant diagnosis system and check device based on deep learning |
-
2019
- 2019-12-20 CN CN201911327592.7A patent/CN110974142B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101637379A (en) * | 2008-07-31 | 2010-02-03 | 奥林巴斯医疗株式会社 | Image display apparatus, endoscope system using the same, and image display method |
| US20100245552A1 (en) * | 2009-03-26 | 2010-09-30 | Olympus Corporation | Image processing device, imaging device, computer-readable storage medium, and image processing method |
| CN110325100A (en) * | 2017-03-01 | 2019-10-11 | 富士胶片株式会社 | Endoscopic system and its operating method |
| CN109222865A (en) * | 2018-10-17 | 2019-01-18 | 卓外（上海）医疗电子科技有限公司 | A kind of multi-modality imaging endoscopic system |
| CN109460717A (en) * | 2018-10-22 | 2019-03-12 | 青岛美迪康数字工程有限公司 | Alimentary canal Laser scanning confocal microscope lesion image-recognizing method and device |
| CN109523522A (en) * | 2018-10-30 | 2019-03-26 | 腾讯科技（深圳）有限公司 | Processing method, device, system and the storage medium of endoscopic images |
| CN109146884A (en) * | 2018-11-16 | 2019-01-04 | 青岛美迪康数字工程有限公司 | Endoscopy monitoring method and device |
| CN110495847A (en) * | 2019-08-23 | 2019-11-26 | 重庆天如生物科技有限公司 | Alimentary canal morning cancer assistant diagnosis system and check device based on deep learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110974142B (en) | 2020-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10803582B2 (en) | Image diagnosis learning device, image diagnosis device, image diagnosis method, and recording medium for storing program | |
| CN111951221B (en) | Glomerular cell image recognition method based on deep neural network | |
| CN112614128B (en) | System and method for assisting biopsy under endoscope based on machine learning | |
| US9171477B2 (en) | Method and system for recognizing and assessing surgical procedures from video | |
| WO2021147429A1 (en) | Endoscopic image display method, apparatus, computer device, and storage medium | |
| JP2006209698A (en) | Target tracking device, microscope system and target tracking program | |
| CN111127426B (en) | Gastric mucosa cleanliness evaluation method and system based on deep learning | |
| US11944262B2 (en) | Endoscope processor, information processing device, and endoscope system | |
| EP3853763A1 (en) | Image feature detection | |
| CN107230198B (en) | Gastroscope image intelligent processing method and processing device | |
| CN111145200B (en) | Blood vessel center line tracking method combining convolutional neural network and cyclic neural network | |
| CN111144271A (en) | Method and system for automatically identifying biopsy parts and biopsy quantity under endoscope | |
| CN110974142B (en) | Real-time synchronous endoscope lesion positioning system of confocal laser microscopy endoscope | |
| CN110648304A (en) | Intelligent auxiliary diagnosis method for handheld hard endoscope | |
| CN112863699B (en) | ESD preoperative discussion system based on mobile terminal | |
| CN112022073A (en) | Real-time synchronous endoscope lesion positioning method and system for confocal laser microscopy endoscope | |
| CN113990456A (en) | Deep learning-based graphical analysis and screening method and system for early cancers of digestive tract | |
| CN111067473A (en) | Gastric precancerous resection boundary marking system and method | |
| CN111126474B (en) | Confocal laser micro-endoscope digestive tract image identification method and system | |
| JP6710853B2 (en) | Probe-type confocal laser microscope endoscopic image diagnosis support device | |
| Ciobanu et al. | Steps towards computer-assisted classification of colonoscopy video frames | |
| CN110795965B (en) | Method and system for automatically identifying pathogenic phase in fungus microscopic image | |
| JP7126673B2 (en) | Endoscope image processing program and endoscope system | |
| CN117132840B (en) | Peptic ulcer classification method and system based on AHS classification and Forrest classification | |
| CN111127443B (en) | PTAM-based lumen image feature point detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Li Zhen Inventor after: Feng Jian Inventor after: Zuo Xiuli Inventor after: Li Yanqing Inventor after: Liu Guanqun Inventor after: Yang Xiaoyun Inventor after: Lai Yonghang Inventor after: Shao Xuejun Inventor before: Li Zhen Inventor before: Feng Jian Inventor before: Zuo Xiuli Inventor before: Li Yanqing Inventor before: Liu Guanqun Inventor before: Yang Xiaoyun Inventor before: Lai Yonghang Inventor before: Shao Xuejun |