CN116188466B - Method and device for determining in-vivo residence time of medical instrument - Google Patents

Method and device for determining in-vivo residence time of medical instrument Download PDF

Info

Publication number
CN116188466B
CN116188466B CN202310459642.7A CN202310459642A CN116188466B CN 116188466 B CN116188466 B CN 116188466B CN 202310459642 A CN202310459642 A CN 202310459642A CN 116188466 B CN116188466 B CN 116188466B
Authority
CN
China
Prior art keywords
image
recognition model
image recognition
determining
images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310459642.7A
Other languages
Chinese (zh)
Other versions
CN116188466A (en
Inventor
招展鹏
麦春明
周可
王羽嗣
王云忠
刘思德
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Side Medical Technology Co ltd
Original Assignee
Guangzhou Side Medical Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Side Medical Technology Co ltd filed Critical Guangzhou Side Medical Technology Co ltd
Priority to CN202310459642.7A priority Critical patent/CN116188466B/en
Publication of CN116188466A publication Critical patent/CN116188466A/en
Application granted granted Critical
Publication of CN116188466B publication Critical patent/CN116188466B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/54—Extraction of image or video features relating to texture
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10068—Endoscopic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30092—Stomach; Gastric
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

The invention provides a method and a device for determining in-vivo stay time of a medical instrument, which belong to the field of image processing, wherein a detection image is input into a first image recognition model trained in advance to obtain a first recognition result; determining a starting point image according to the first identification result, wherein the starting point image is used for representing that the medical instrument reaches a preset part in the body of the subject; inputting the detection image into a pre-trained second image recognition model to obtain a second recognition result; determining an end point image according to the second identification result, wherein the end point image is used for representing that the medical instrument leaves a preset part in the body of the subject; and obtaining the stay time according to the time corresponding to the start point image and the time corresponding to the end point image. According to the invention, the first image recognition model and the second image recognition model are trained in advance, the stay time is determined according to the starting point image and the ending point image and the corresponding time, manual film reading is not needed, the efficiency is improved, and the film reading burden of doctors is reduced.

Description

Method and device for determining in-vivo residence time of medical instrument
Technical Field
The invention relates to the technical field of image processing, in particular to a method and a device for determining in-vivo residence time of a medical instrument.
Background
During a surgical examination, images of a specific part of a patient's body need to be taken in order for a physician to analyze the patient's condition.
At present, the determination of the stay time of the existing medical instrument in the body is mainly carried out by manually calibrating a reader, tens of thousands of images are checked, and the doctor needs to find out the corresponding images from a large number of images, so that time is wasted, the fatigue of the doctor is caused, and the reading burden of the doctor is increased.
Disclosure of Invention
The invention provides a method and a device for determining the in-vivo residence time of a medical instrument, which are used for solving the defects that in the prior art, the determination of the in-vivo residence time of the medical instrument is mainly performed by manual calibration by a reader, so that time is wasted, the fatigue of the doctor is caused, and the reading burden of the doctor is increased.
The invention provides a method for determining in-vivo residence time of a medical instrument, which is used for acquiring at least one detection image in a subject, and comprises the following steps:
inputting the detection image into a pre-trained first image recognition model to obtain a first recognition result output by the first image recognition model;
determining a starting point image according to the first identification result, wherein the starting point image is used for representing that the medical instrument reaches a preset part in the body of the subject;
inputting the detection image into a pre-trained second image recognition model to obtain a second recognition result output by the second image recognition model;
determining an end point image according to the second identification result, wherein the end point image is used for representing that the medical instrument leaves a preset part in the body of the subject;
and obtaining the stay time according to the time corresponding to the starting point image and the time corresponding to the ending point image.
According to the method for determining the in-vivo residence time of the medical instrument provided by the invention, the detection image is input into a pre-trained first image recognition model to obtain a first recognition result output by the first image recognition model, and the method comprises the following steps:
selecting at least one target image from the detection images based on a preset sliding window;
inputting the target image into a pre-trained first image recognition model to obtain a recognition result output by the first image recognition model;
and carrying out consistency check on the identification result output by the first image identification model, and obtaining the first identification result according to the check result.
According to the method for determining the in-vivo residence time of the medical instrument provided by the invention, the detection image is input into a pre-trained second image recognition model to obtain a second recognition result output by the second image recognition model, and the method comprises the following steps:
selecting at least one target image from the detection images based on a preset sliding window;
inputting the target image into a pre-trained second image recognition model to obtain a recognition result output by the second image recognition model;
and carrying out consistency check on the identification result output by the second image identification model, and obtaining the second identification result according to the check result.
The method for determining the in-vivo residence time of the medical instrument provided by the invention further comprises the following steps:
sorting the detection images based on the shooting time sequence of the detection images;
and inputting the detection images arranged behind the starting point image into a pre-trained second image recognition model to obtain a second recognition result output by the second image recognition model.
According to the method for determining the in-vivo residence time of the medical instrument provided by the invention, the first image recognition model or the second image recognition model is trained, and the method comprises the following steps:
acquiring a labeling image corresponding to at least one part in the body of the subject;
classifying the marked images according to the preset parts to obtain a classified data set;
training the neural network model according to the classification data set to obtain the first image recognition model or the second image recognition model.
According to the method for determining the in-vivo residence time of the medical instrument provided by the invention, the neural network model is trained according to the classification data set, and the method comprises the following steps:
preprocessing the marked images in the classified data set;
and inputting the preprocessed labeling image into the neural network model, and training the neural network model.
According to the method for determining the in-vivo residence time of the medical instrument provided by the invention, the preprocessing of the labeling images in the classified data sets comprises the following steps:
and carrying out texture extraction processing and feature dimension reduction processing on the marked image.
The present invention also provides a medical instrument in-vivo residence time determining apparatus for acquiring at least one image in a subject, the apparatus comprising:
the first recognition module is used for inputting the image into a pre-trained first image recognition model to obtain a first recognition result output by the first image recognition model;
the first determining module is used for determining a starting point image according to the first identification result, wherein the starting point image is used for representing that the medical instrument reaches a preset part in the body of the subject;
the second recognition module is used for inputting the image into a pre-trained second image recognition model to obtain a second recognition result output by the second image recognition model;
the second determining module is used for determining an end point image according to the second identification result, wherein the end point image is used for representing that the medical instrument leaves a preset part in the body of the subject;
and the duration determining module is used for obtaining the stay duration according to the time corresponding to the starting point image and the time corresponding to the ending point image.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the method for determining the in-vivo residence time of the medical instrument when executing the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method of determining in vivo residence time of a medical device as described in any one of the above.
According to the method and the device for determining the in-vivo stay time of the medical instrument, the first image recognition model and the second image recognition model are trained in advance, the starting point image and the ending point image are determined, the stay time is determined according to the time corresponding to the starting point image and the ending point image, the stay time can be determined without manual film reading, efficiency is improved, and film reading burden of doctors is reduced.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for determining the in-vivo residence time of a medical device provided by the invention;
FIG. 2 is a schematic flow chart of step S110 in FIG. 1 according to the present invention;
FIG. 3 is a schematic flow chart of step S130 in FIG. 1 according to the present invention;
FIG. 4 is a second flowchart of step S130 in FIG. 1 according to the present invention;
FIG. 5 is a schematic flow chart of a training model provided by the present invention;
FIG. 6 is a schematic illustration of annotation image partitioning provided by the present invention;
FIG. 7 is a schematic representation of a esophageal two-classification dataset provided by the present invention;
FIG. 8 is a schematic representation of a duodenal classification dataset provided by the present invention;
FIG. 9 is a flow chart of texture extraction process according to the present invention;
FIG. 10 is a second flow chart of the training model according to the present invention;
FIG. 11 is a functional block diagram of a medical device in-vivo residence time determination apparatus provided by the present invention;
fig. 12 is a schematic structural diagram of an electronic device provided by the present invention;
fig. 13 is a flow chart of the feature extraction process provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the description of the embodiments of the present application, it should be noted that, directions or positional relationships indicated by terms such as "center", "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., are based on those shown in the drawings, are merely for convenience in describing the embodiments of the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the embodiments of the present application. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the embodiments of the present application, it should be noted that, unless explicitly specified and limited otherwise, the terms "connected," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium. The specific meaning of the terms in the embodiments of the present application will be understood by those of ordinary skill in the art in a specific context.
In the examples herein, a first feature "on" or "under" a second feature may be either the first and second features in direct contact, or the first and second features in indirect contact via an intermediary, unless expressly stated and defined otherwise. Moreover, a first feature being "above," "over" and "on" a second feature may be a first feature being directly above or obliquely above the second feature, or simply indicating that the first feature is level higher than the second feature. The first feature being "under", "below" and "beneath" the second feature may be the first feature being directly under or obliquely below the second feature, or simply indicating that the first feature is less level than the second feature.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the embodiments of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Fig. 1 is a flowchart of a method for determining an in-vivo residence time of a medical device according to the present invention, and as shown in fig. 1, the present invention provides a method for determining an in-vivo residence time of a medical device, where the medical device is configured to obtain at least one detection image in a subject, and the method includes:
s110, inputting the detection image into a pre-trained first image recognition model to obtain a first recognition result output by the first image recognition model;
s120, determining a starting point image according to the first identification result, wherein the starting point image is used for representing that the medical instrument reaches a preset part in the body of the subject;
s130, inputting the detection image into a pre-trained second image recognition model to obtain a second recognition result output by the second image recognition model;
s140, determining an end point image according to the second identification result, wherein the end point image is used for representing that the medical instrument leaves a preset part in the body of the subject;
s150, obtaining the stay time according to the time corresponding to the starting point image and the time corresponding to the ending point image.
Optionally, the medical apparatus of the present invention refers to an apparatus for examining a certain portion in a patient, and may acquire an image of a corresponding portion in the patient, so that a doctor may examine the condition of the patient.
Specifically, the invention is described in detail by taking a capsule gastroscope as an example.
The capsule gastroscope is a digestive system examination instrument, does not need a guide core, and basically eliminates the pain of patients. When the patient swallows the capsule gastroscope, the capsule gastroscope advances along with the peristaltic motion of the digestive tract, and the image of the digestive tract is taken and sent to a recording instrument outside the patient, and finally discharged outside the patient.
The invention can determine stomach images in a large number of gastroscope images, determine the stay time of the capsule gastroscope in the stomach, and the capsule gastroscope advances from the esophagus of a subject to the stomach and then enters the duodenum from the stomach.
The first image recognition model is used for judging whether the corresponding part of the capsule gastroscope is an esophagus, if the corresponding part is not the esophagus, the end of the esophagus part is indicated, and the corresponding detection image is a starting point image (a first stomach image) which indicates that the capsule gastroscope reaches the stomach of the subject.
The second image recognition model is used for judging whether the position corresponding to the capsule gastroscope is the duodenum, if so, the capsule enters the duodenum through the stomach, the previous detection image of the corresponding detection image is an end point image (the last image of the stomach), and the capsule endoscope is separated from the stomach of the subject.
Finally, according to the first image of the stomach and the last image of the stomach, the middle part is the stomach image which needs to be read, and then according to the shooting time of the two images, the time difference value is calculated, so that the stay time of the capsule gastroscope in the stomach can be approximately obtained.
It can be understood that the method and the device determine the start point image and the end point image by training the first image recognition model and the second image recognition model in advance, determine the stay time according to the respective corresponding time of the start point image and the end point image, and can determine the stay time without manual film reading, thereby improving the efficiency and reducing the film reading burden of doctors.
Fig. 2 is a schematic flowchart of step S110 in fig. 1 provided in the present invention, as shown in fig. 2, on the basis of the foregoing embodiment, as an optional embodiment, the inputting the detected image into a pre-trained first image recognition model to obtain a first recognition result output by the first image recognition model includes:
s210, selecting at least one target image from the detection images based on a preset sliding window;
s220, inputting the target image into a pre-trained first image recognition model to obtain a recognition result output by the first image recognition model;
s230, consistency verification is conducted on the identification result output by the first image identification model, and the first identification result is obtained according to the verification result.
Optionally, setting the length parameter of the sliding window as N (for example, N is 3), and performing consistency check on the reasoning results of the N continuous frames of images in the sliding interval (the category with the largest appearance result in the N image recognition results is used as the category of the current sliding window).
For example, three images are selected from the detection images, a first image recognition model is input, and the first image recognition model recognizes that two images are esophagus, and the current corresponding part of the capsule is considered to be esophagus.
It can be understood that by setting the sliding window, the invention carries out consistency check on the identification result output by the first image identification model, carries out forward reasoning on the image sequence under the capsule gastroscope, and can improve the robustness and accuracy of the model identification result.
Fig. 3 is a schematic flowchart of step S130 in fig. 1 provided by the present invention, as shown in fig. 3, on the basis of the foregoing embodiment, as an optional embodiment, according to a method for determining in-vivo residence time of a medical apparatus and instrument provided by the present invention, the inputting the detected image into a pre-trained second image recognition model, to obtain a second recognition result output by the second image recognition model, includes:
s310, selecting at least one target image from the detection images based on a preset sliding window;
s320, inputting the target image into a pre-trained second image recognition model to obtain a recognition result output by the second image recognition model;
s330, consistency verification is conducted on the identification result output by the second image identification model, and the second identification result is obtained according to the verification result.
Optionally, setting the length parameter of the sliding window as N (for example, N is 3), and performing consistency check on the reasoning results of the N continuous frames of images in the sliding interval (the category with the largest appearance result in the N image recognition results is used as the category of the current sliding window).
For example, three images are selected from the detection images, a second image recognition model is input, and the second image recognition model recognizes that two images are the duodenum, and then the part corresponding to the capsule currently is considered to be the duodenum.
It can be understood that by setting the sliding window, the invention carries out consistency check on the identification result output by the second image identification model, carries out forward reasoning on the image sequence under the capsule gastroscope, and can improve the robustness and accuracy of the model identification result.
Fig. 4 is a second schematic flow chart of step S130 in fig. 1 provided by the present invention, as shown in fig. 4, a method for determining in-vivo residence time of a medical device provided by the present invention, which further includes:
s410, sorting the detection images based on the shooting time sequence of the detection images;
s420, inputting the detection images arranged behind the initial point image into a pre-trained second image recognition model to obtain a second recognition result output by the second image recognition model.
Optionally, a preset time period may be further set, the detected images arranged after the start point image is in the preset time period are input into a pre-trained second image recognition model, and a second recognition result output by the second image recognition model is obtained.
For example, the shooting time of the starting point image is T, the preset time period is T0, and the detected image with the shooting time of t+t0 can be input into the second image recognition model.
It can be understood that after the starting point image is determined, the detection image after the starting point image is input into the second image recognition model, so that the images required to be recognized by the second image recognition model can be reduced, and the recognition efficiency is improved.
FIG. 5 is a schematic flow chart of a training model provided by the present invention, as shown in FIG. 5, training the first image recognition model or the second image recognition model, including:
s510, obtaining a labeling image corresponding to at least one part in the body of the subject;
s520, classifying the marked images according to the preset parts to obtain a classified data set;
and S530, training a neural network model according to the classification data set to obtain the first image recognition model or the second image recognition model.
Fig. 6 is a schematic view of labeling image division provided by the present invention, fig. 7 is a schematic view of esophageal two-classification data set provided by the present invention, and fig. 8 is a schematic view of duodenal two-classification data set provided by the present invention, as shown in fig. 6-8, firstly, a gastric image database of a subject is obtained by a capsule endoscope system, then, classification labeling is performed on the images by adopting a manual mode, and the images are divided into the following three categories: "0. Esophagus", "1. Duodenum", "2. Others". For the esophagus two-class model, selecting ' 0 ' esophagus ' as a class I, selecting ' 1 ' duodenum ' and ' 2 ' other ' esophagus as a class II, forming an esophagus two-class data set, and selecting class images according to proportion, and dividing the class images into a training set and a test set; for the duodenum classification model, selecting ' 1 ' duodenum ' as a first category, selecting ' 0 ' esophagus ' and ' 2 ' other ' as a second category, forming a duodenum classification data set, selecting a category image according to a proportion, and dividing the category image into a training set and a test set.
Optionally, the training the neural network model according to the classification data set includes:
preprocessing the marked images in the classified data set;
and inputting the preprocessed labeling image into the neural network model, and training the neural network model.
Optionally, the preprocessing the labeling image in the classified dataset includes:
and carrying out texture extraction processing and feature dimension reduction processing on the marked image.
Fig. 9 is a schematic flow chart of a texture extraction process according to the present invention, and as shown in fig. 9, the texture extraction process includes: the method comprises the steps of extracting the characteristics of an image by adopting a circular LBP (Local Binary Pattern ) algorithm, respectively calculating the radius r=1, 2, and the sampling frequency p=2, 1, wherein the corresponding r and p combination is equivalent to extracting texture characteristic diagrams of three different frequencies in the sigma=4, 8 and 16 fields from an original image, and then carrying out fusion average on the three images.
The feature dimension reduction processing comprises the following steps: in order to reduce redundant features of the image and increase the calculation speed, the image is subjected to dimension reduction by using a PCA (Principal Components Analysis, principal component analysis) algorithm before being input into the network, the output dimension is maintained at 1024, and the image subjected to dimension reduction is input into the network to execute training.
FIG. 10 is a second flow chart of the training model provided by the present invention, as shown in FIG. 10, the model training includes: and using the shufflelet as a main network, respectively training the neural network model by combining the texture feature map input after the dimension reduction, and evaluating the effect of the trained model according to the test set to obtain an optimal model.
It can be appreciated that the invention provides a technical scheme for training the first image recognition model and the second image recognition model, the processing speed of the model can be increased through texture extraction processing, the redundant features of the image can be reduced through feature dimension reduction processing, and the calculation speed is increased.
Fig. 13 is a schematic flow chart of a feature extraction process provided in the present invention, as shown in fig. 13, on the basis of the above embodiment, as an alternative embodiment, the feature extraction process performed on an image includes:
carrying out H-channel color feature extraction processing, LBP texture feature extraction processing and resnet feature extraction processing on the marked image;
if the number of the marked images of the classified data set is smaller than a preset value, the H channel color features, the LBP texture features and the resnet features are spliced, and the dimension of the spliced image features is reduced based on a principal component analysis method; and inputting the image characteristics subjected to dimension reduction into a classifier formed by a full-connection layer for classification, and obtaining a classification result.
If the number of the marked images of the classified data set is not smaller than a preset value, training the neural network model through different input channels by using the H channel color feature, the LBP texture feature and the resnet feature.
On the basis of the above embodiment, as an optional embodiment, if the number of the labeling images of the classified dataset is smaller than a preset value, the noise features of the labeling images are collected, and according to the consistency of the noise features of the same labeling image, the H-channel color features, the LBP texture features and the resnet features are positioned and spliced, so that the alignment of the H-channel color features, the LBP texture features and the resnet features is realized.
Based on the above embodiment, as an optional embodiment, if the number of labeled images in the classification dataset is not less than a preset value, training the neural network model by using the H-channel color feature, the LBP texture feature and the resnet feature through different input channels includes:
converting the H channel color features, LBP texture features and resnet features into feature images under different receptive fields by using convolution kernels with different scales; processing the feature map based on a multi-scale channel attention mechanism and a spatial attention mechanism to obtain the salient features of the marked image;
and (3) carrying out channel characteristic stitching on the salient features and the marked images, and training the neural network model based on the stitched images.
The in-vivo residence time determining device for medical instruments provided by the invention is described below, and the in-vivo residence time determining device for medical instruments described below and the in-vivo residence time determining method for medical instruments described above can be referred to correspondingly.
Fig. 11 is a schematic block diagram of an in-vivo residence time determining apparatus for a medical device according to the present invention, and as shown in fig. 11, the present invention also provides an in-vivo residence time determining apparatus for a medical device for acquiring at least one image in a subject, the apparatus comprising:
a first recognition module 1110, configured to input the image into a first image recognition model trained in advance, and obtain a first recognition result output by the first image recognition model;
a first determining module 1120, configured to determine a start point image according to the first identification result, where the start point image is used to characterize that the medical instrument reaches a preset location in the subject;
the second recognition module 1130 is configured to input the image into a pre-trained second image recognition model, and obtain a second recognition result output by the second image recognition model;
a second determining module 1140, configured to determine an end point image according to the second identification result, where the end point image is used to characterize that the medical device leaves a preset location in the subject;
and a duration determining module 1150, configured to obtain a stay duration according to the time corresponding to the start point image and the time corresponding to the end point image.
As an embodiment, the first identifying module 1110 is further configured to:
selecting at least one target image from the detection images based on a preset sliding window;
inputting the target image into a pre-trained first image recognition model to obtain a recognition result output by the first image recognition model;
and carrying out consistency check on the identification result output by the first image identification model, and obtaining the first identification result according to the check result.
As an embodiment, the second identifying module 1130 is further configured to:
selecting at least one target image from the detection images based on a preset sliding window;
inputting the target image into a pre-trained second image recognition model to obtain a recognition result output by the second image recognition model;
and carrying out consistency check on the identification result output by the second image identification model, and obtaining the second identification result according to the check result.
As an embodiment, the second identifying module 1130 is further configured to:
sorting the detection images based on the shooting time sequence of the detection images;
and inputting the detection images arranged behind the starting point image into a pre-trained second image recognition model to obtain a second recognition result output by the second image recognition model.
As an embodiment, the device further comprises a training module, wherein the training module is used for:
acquiring a labeling image corresponding to at least one part in the body of the subject;
classifying the marked images according to the preset parts to obtain a classified data set;
training the neural network model according to the classification data set to obtain the first image recognition model or the second image recognition model.
As an embodiment, the training module is further configured to:
preprocessing the marked images in the classified data set;
and inputting the preprocessed labeling image into the neural network model, and training the neural network model.
As an embodiment, the training module is further configured to:
and carrying out texture extraction processing and feature dimension reduction processing on the marked image.
Fig. 12 illustrates a physical structure diagram of an electronic device, as shown in fig. 12, which may include: processor 1210, communication interface (Communications Interface), 1220, memory 1230 and communication bus 1240, wherein processor 1210, communication interface 1220 and memory 1230 communicate with each other via communication bus 1240. Processor 1210 may invoke logic instructions in memory 1230 to perform a medical instrument in vivo dwell time determination method for acquiring at least one detection image in a subject, the method comprising:
inputting the detection image into a pre-trained first image recognition model to obtain a first recognition result output by the first image recognition model;
determining a starting point image according to the first identification result, wherein the starting point image is used for representing that the medical instrument reaches a preset part in the body of the subject;
inputting the detection image into a pre-trained second image recognition model to obtain a second recognition result output by the second image recognition model;
determining an end point image according to the second identification result, wherein the end point image is used for representing that the medical instrument leaves a preset part in the body of the subject;
and obtaining the stay time according to the time corresponding to the starting point image and the time corresponding to the ending point image.
In addition, the logic instructions in the memory 1230 described above may be implemented in the form of software functional units and sold or used as a stand-alone product, stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being storable on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of executing the method for determining a stay-in-body time of a medical instrument provided by the above methods, the medical instrument being configured to acquire at least one detection image in a subject, the method comprising:
inputting the detection image into a pre-trained first image recognition model to obtain a first recognition result output by the first image recognition model;
determining a starting point image according to the first identification result, wherein the starting point image is used for representing that the medical instrument reaches a preset part in the body of the subject;
inputting the detection image into a pre-trained second image recognition model to obtain a second recognition result output by the second image recognition model;
determining an end point image according to the second identification result, wherein the end point image is used for representing that the medical instrument leaves a preset part in the body of the subject;
and obtaining the stay time according to the time corresponding to the starting point image and the time corresponding to the ending point image.
In yet another aspect, the present invention provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the method for determining a residence time in a medical instrument provided by the above methods, the medical instrument being configured to acquire at least one detection image in a subject, the method comprising:
inputting the detection image into a pre-trained first image recognition model to obtain a first recognition result output by the first image recognition model;
determining a starting point image according to the first identification result, wherein the starting point image is used for representing that the medical instrument reaches a preset part in the body of the subject;
inputting the detection image into a pre-trained second image recognition model to obtain a second recognition result output by the second image recognition model;
determining an end point image according to the second identification result, wherein the end point image is used for representing that the medical instrument leaves a preset part in the body of the subject;
and obtaining the stay time according to the time corresponding to the starting point image and the time corresponding to the ending point image.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A method for determining a length of stay in a body of a medical device for acquiring at least one detection image in a subject, the method comprising:
inputting the detection image into a pre-trained first image recognition model to obtain a first recognition result output by the first image recognition model;
determining a starting point image according to the first identification result, wherein the starting point image is used for representing that the medical instrument reaches a preset part in the body of the subject;
inputting the detection image into a pre-trained second image recognition model to obtain a second recognition result output by the second image recognition model;
determining an end point image according to the second identification result, wherein the end point image is used for representing that the medical instrument leaves a preset part in the body of the subject;
obtaining a stay time according to the time corresponding to the starting point image and the time corresponding to the ending point image;
and carrying out feature extraction processing on the marked images in the process of training the first image recognition model and the second image recognition model, wherein the feature extraction processing comprises the following steps:
carrying out H-channel color feature extraction processing, LBP texture feature extraction processing and resnet feature extraction processing on the marked image;
if the number of the marked images of the classified data set is smaller than a preset value, the H channel color features, the LBP texture features and the resnet features are spliced, and the dimension of the spliced image features is reduced based on a principal component analysis method; inputting the image characteristics subjected to dimension reduction into a classifier formed by a full-connection layer for classification, and obtaining a classification result;
if the number of the marked images of the classified data set is not smaller than a preset value, training the neural network model through different input channels by using the H channel color feature, the LBP texture feature and the resnet feature;
if the number of the marked images of the classified data set is smaller than a preset value, collecting noise characteristics of the marked images, and positioning and splicing the H channel color characteristics, the LBP texture characteristics and the resnet characteristics according to the consistency of the noise characteristics of the same marked image to realize the alignment of the H channel color characteristics, the LBP texture characteristics and the resnet characteristics.
2. The method for determining the in-vivo residence time of a medical device according to claim 1, wherein the inputting the detection image into a pre-trained first image recognition model to obtain a first recognition result output by the first image recognition model comprises:
selecting at least one target image from the detection images based on a preset sliding window;
inputting the target image into a pre-trained first image recognition model to obtain a recognition result output by the first image recognition model;
and carrying out consistency check on the identification result output by the first image identification model, and obtaining the first identification result according to the check result.
3. The method for determining the in-vivo residence time of a medical device according to claim 1, wherein the inputting the detection image into a pre-trained second image recognition model to obtain a second recognition result output by the second image recognition model comprises:
selecting at least one target image from the detection images based on a preset sliding window;
inputting the target image into a pre-trained second image recognition model to obtain a recognition result output by the second image recognition model;
and carrying out consistency check on the identification result output by the second image identification model, and obtaining the second identification result according to the check result.
4. The method for determining the in-vivo residence time of a medical device according to claim 1, further comprising:
sorting the detection images based on the shooting time sequence of the detection images;
and inputting the detection images arranged behind the starting point image into a pre-trained second image recognition model to obtain a second recognition result output by the second image recognition model.
5. The method of claim 1, wherein training the first image recognition model or the second image recognition model comprises:
acquiring a labeling image corresponding to at least one part in the body of the subject;
classifying the marked images according to the preset parts to obtain a classified data set;
training the neural network model according to the classification data set to obtain the first image recognition model or the second image recognition model.
6. The method of claim 5, wherein training a neural network model from the classification dataset comprises:
preprocessing the marked images in the classified data set;
and inputting the preprocessed labeling image into the neural network model, and training the neural network model.
7. The method of claim 6, wherein preprocessing the labeled images in the classified dataset comprises:
and carrying out texture extraction processing and feature dimension reduction processing on the marked image.
8. A medical device in-vivo dwell time determination apparatus for acquiring at least one image of a subject, the apparatus comprising:
the first recognition module is used for inputting the image into a pre-trained first image recognition model to obtain a first recognition result output by the first image recognition model;
the first determining module is used for determining a starting point image according to the first identification result, wherein the starting point image is used for representing that the medical instrument reaches a preset part in the body of the subject;
the second recognition module is used for inputting the image into a pre-trained second image recognition model to obtain a second recognition result output by the second image recognition model;
the second determining module is used for determining an end point image according to the second identification result, wherein the end point image is used for representing that the medical instrument leaves a preset part in the body of the subject;
the duration determining module is used for obtaining the stay duration according to the time corresponding to the starting point image and the time corresponding to the ending point image;
and carrying out feature extraction processing on the marked images in the process of training the first image recognition model and the second image recognition model, wherein the feature extraction processing comprises the following steps:
carrying out H-channel color feature extraction processing, LBP texture feature extraction processing and resnet feature extraction processing on the marked image;
if the number of the marked images of the classified data set is smaller than a preset value, the H channel color features, the LBP texture features and the resnet features are spliced, and the dimension of the spliced image features is reduced based on a principal component analysis method; inputting the image characteristics subjected to dimension reduction into a classifier formed by a full-connection layer for classification, and obtaining a classification result;
if the number of the marked images of the classified data set is not smaller than a preset value, training the neural network model through different input channels by using the H channel color feature, the LBP texture feature and the resnet feature;
if the number of the marked images of the classified data set is smaller than a preset value, collecting noise characteristics of the marked images, and positioning and splicing the H channel color characteristics, the LBP texture characteristics and the resnet characteristics according to the consistency of the noise characteristics of the same marked image to realize the alignment of the H channel color characteristics, the LBP texture characteristics and the resnet characteristics.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method for determining the in-vivo residence time of a medical device according to any one of claims 1 to 7 when the program is executed by the processor.
10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the method of in vivo residence time determination of a medical device according to any one of claims 1 to 7.
CN202310459642.7A 2023-04-26 2023-04-26 Method and device for determining in-vivo residence time of medical instrument Active CN116188466B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310459642.7A CN116188466B (en) 2023-04-26 2023-04-26 Method and device for determining in-vivo residence time of medical instrument

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310459642.7A CN116188466B (en) 2023-04-26 2023-04-26 Method and device for determining in-vivo residence time of medical instrument

Publications (2)

Publication Number Publication Date
CN116188466A CN116188466A (en) 2023-05-30
CN116188466B true CN116188466B (en) 2023-07-21

Family

ID=86436846

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310459642.7A Active CN116188466B (en) 2023-04-26 2023-04-26 Method and device for determining in-vivo residence time of medical instrument

Country Status (1)

Country Link
CN (1) CN116188466B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117393101A (en) * 2023-12-13 2024-01-12 广州思德医疗科技有限公司 Endoscope report auxiliary generation method and device

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101571964A (en) * 2008-12-04 2009-11-04 武汉理工大学 Fully-automatic image timing and information management system for athletic competition
CN102542165A (en) * 2011-12-23 2012-07-04 三星半导体(中国)研究开发有限公司 Operating device and operating method for three-dimensional virtual chessboard
CN102945364A (en) * 2012-10-31 2013-02-27 黑龙江省电力有限公司信息通信分公司 Staff station status detection system based on motion image identification
WO2020181570A1 (en) * 2019-03-08 2020-09-17 上海达显智能科技有限公司 Intelligent smoke removal device and control method thereof
CN114782388A (en) * 2022-04-29 2022-07-22 小荷医疗器械(海南)有限公司 Endoscope advance and retreat time determining method and device based on image recognition
CN115620919A (en) * 2022-10-26 2023-01-17 河南科技大学 Coronary heart disease intelligent auxiliary dialectic device based on machine learning
CN115661728A (en) * 2022-12-29 2023-01-31 北京正大创新医药有限公司 Virus sampling in-place judgment method based on image recognition and virus sampling system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114118124B (en) * 2021-09-29 2023-09-12 北京百度网讯科技有限公司 Image detection method and device

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101571964A (en) * 2008-12-04 2009-11-04 武汉理工大学 Fully-automatic image timing and information management system for athletic competition
CN102542165A (en) * 2011-12-23 2012-07-04 三星半导体(中国)研究开发有限公司 Operating device and operating method for three-dimensional virtual chessboard
CN102945364A (en) * 2012-10-31 2013-02-27 黑龙江省电力有限公司信息通信分公司 Staff station status detection system based on motion image identification
WO2020181570A1 (en) * 2019-03-08 2020-09-17 上海达显智能科技有限公司 Intelligent smoke removal device and control method thereof
CN114782388A (en) * 2022-04-29 2022-07-22 小荷医疗器械(海南)有限公司 Endoscope advance and retreat time determining method and device based on image recognition
CN115620919A (en) * 2022-10-26 2023-01-17 河南科技大学 Coronary heart disease intelligent auxiliary dialectic device based on machine learning
CN115661728A (en) * 2022-12-29 2023-01-31 北京正大创新医药有限公司 Virus sampling in-place judgment method based on image recognition and virus sampling system

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Research on ResNet101 Network Chemical Reagent Label Image Classification Based on Transfer Learning;Zhengguang Xu 等;《2020 IEEE 2nd International Conference on Civil Aviation Safety and Information Technology (ICCASIT)》;全文 *
基于特征提取和胶囊网络的人脸表情识别;黄小刚 等;《智能计算机与应用》;第12卷(第10期);全文 *
消化道内窥镜图像异常的人工智能诊断方法研究进展;张璐璐;郭旭东;张娜;张林琪;张慧河;;生物医学工程学进展(第01期);全文 *

Also Published As

Publication number Publication date
CN116188466A (en) 2023-05-30

Similar Documents

Publication Publication Date Title
KR102210806B1 (en) Apparatus and method for diagnosing gastric lesion using deep learning of endoscopic images
JP6657480B2 (en) Image diagnosis support apparatus, operation method of image diagnosis support apparatus, and image diagnosis support program
CN110796613B (en) Automatic identification method and device for image artifacts
JP7167515B2 (en) Identification device, program, identification method, information processing device and identification device
Zhang et al. Diabetic retinopathy grading by a source-free transfer learning approach
JP2016531709A (en) Image analysis technology for diagnosing disease
CN116188466B (en) Method and device for determining in-vivo residence time of medical instrument
KR20200038121A (en) Endoscopic device and method for diagnosing gastric lesion based on gastric endoscopic image obtained in real time
CN111553892A (en) Lung nodule segmentation calculation method, device and system based on deep learning
CN113989284B (en) Helicobacter pylori assists detecting system and detection device
CN112741651B (en) Method and system for processing ultrasonic image of endoscope
JP7129058B2 (en) Medical image diagnosis support device, program, and medical image diagnosis support method
KR102191412B1 (en) Apparatus and method for diagnosing uterine lesion using deep leaking algorithm
CN114998644B (en) Tumor diagnosis system, construction method thereof, terminal device and storage medium
US11885862B2 (en) Deep learning based magnetic resonance imaging (MRI) examination acceleration
KR102360615B1 (en) Medical image diagnosis assistance apparatus and method using a plurality of medical image diagnosis algorithm for endoscope images
CN114581402A (en) Capsule endoscope quality inspection method, device and storage medium
KR102136107B1 (en) Apparatus and method for alignment of bone suppressed chest x-ray image
WO2021220272A1 (en) Systems and methods for analyzing a stream of images
CN112967246A (en) X-ray image auxiliary device and method for clinical decision support system
CN117457229B (en) Anesthesia depth monitoring system and method based on artificial intelligence
US20230134603A1 (en) Systems and methods for selecting images of event indicators
CN114708973B (en) Device and storage medium for evaluating human health
EP4282316A1 (en) Esophagogastroduodenoscopy (egd) system capable of enhancing reliability of edg inspection using deep learning
Yao Machine Learning and Image Processing for Clinical Outcome Prediction: Applications in Medical Data from Patients with Traumatic Brain Injury, Ulcerative Colitis, and Heart Failure

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant