CN113808137A

CN113808137A - Method, device, equipment and storage medium for screening image map of upper gastrointestinal endoscope

Info

Publication number: CN113808137A
Application number: CN202111375055.7A
Authority: CN
Inventors: 杨振宇; 胡珊; 张阔
Original assignee: Wuhan Endoangel Medical Technology Co Ltd
Current assignee: Wuhan Endoangel Medical Technology Co Ltd
Priority date: 2021-11-19
Filing date: 2021-11-19
Publication date: 2021-12-17

Abstract

The embodiment of the invention discloses a method, a device, equipment and a storage medium for screening an upper gastrointestinal endoscope image map. The method comprises the following steps: inputting each endoscope image in the endoscope image set of the upper digestive tract into a pre-trained part classification model to obtain the part category of each endoscope image; inputting all endoscope image maps of each part into a pre-trained target detection model to obtain a target image map set of each part; classifying each target image in the target image set according to a pre-trained classification model to obtain a classification result of each target image; and screening the endoscope image in the target image set of each part according to the classification result to obtain the screened endoscope image. The invention is based on the artificial intelligence technology, reduces the screening difficulty of the endoscope image map, saves the screening time of the endoscope image map, and improves the quality and the accuracy of the subsequently generated endoscope report.

Description

Method, device, equipment and storage medium for screening image map of upper gastrointestinal endoscope

Technical Field

The invention relates to the technical field of medical assistance, in particular to a method, a device, equipment and a storage medium for screening an upper gastrointestinal endoscope image.

Background

Esophagogastroduodenoscopy is a technique that assists physicians in performing upper gastrointestinal examinations and diagnoses by directly observing the condition of the upper gastrointestinal mucosa. With the popularization of electronic endoscopic digital imaging systems, image documents containing precise examination information generated by performing upper gastrointestinal examination by esophagogastroduodenoscope are an essential factor for endoscopic reports.

In the prior art, an endoscope report is generally generated by an examining physician operating an esophagogastroduodenoscope to acquire an image of the endoscope report and store the image in an image library of the endoscope report, and after the examination is finished, the reporting physician performs screening in the image library to obtain a picture which meets the current symptoms of a patient to generate the endoscope report. However, the same focus may be repeatedly exposed in the process of endoscope entering and endoscope withdrawing, so that the machine can repeatedly identify and form repeated images of the same focus, and further, more images are acquired by the esophagogastroduodenoscope, and the image acquisition is easily influenced by the subjectivity of an examining doctor, so that the difficulty of selecting the images in the image library by a reporting doctor is increased, the image with poor quality is easily selected, the difficulty of a generated endoscope report is increased, and the accuracy of the endoscope report is reduced.

Disclosure of Invention

The embodiment of the invention provides a method, a device, equipment and a storage medium for screening an upper gastrointestinal endoscope image map, which are used for solving the technical problems of poor difficulty and low quality in screening the upper gastrointestinal endoscope image map in the prior art.

In a first aspect, an embodiment of the present invention provides a method for screening an image map of an upper gastrointestinal endoscope, including:

inputting each endoscope image in the endoscope image set of the upper digestive tract into a pre-trained part classification model to obtain the part category of each endoscope image;

inputting all endoscope image maps of each part into a pre-trained target detection model to obtain a target image map set of each part;

classifying each target image in the target image set according to a pre-trained classification model to obtain a classification result of each target image;

and screening the endoscope image in the target image set of each part according to the classification result to obtain the screened endoscope image.

In a second aspect, an embodiment of the present invention provides a device for screening an image of an upper gastrointestinal endoscope, including:

the first input unit is used for inputting each endoscope image in the endoscope image set of the upper gastrointestinal tract into a pre-trained part classification model to obtain the part category of each endoscope image;

the second input unit is used for inputting all endoscope image maps of each part into a pre-trained target detection model to obtain a target image atlas of each part;

the first classification unit is used for classifying each target image in the target image set according to a pre-trained classification model to obtain a classification result of each target image;

and the first screening unit is used for screening the endoscope image in the target image set of each part according to the classification result to obtain the screened endoscope image.

In a third aspect, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method for filtering an image of an upper gastrointestinal endoscope as described in the first aspect.

In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program, when executed by a processor, causes the processor to execute the method for screening an image of an upper gastrointestinal endoscope according to the first aspect.

The embodiment of the invention provides a method, a device, equipment and a storage medium for screening an upper gastrointestinal endoscope image map. The method classifies the types of parts of each image in the endoscope image set in advance, detects the target of each endoscope image in the endoscope image set at each part to obtain the target image set at each part, and finally classifies and screens the target image set at each part to complete the screening of the images in the endoscope image set, thereby reducing the screening difficulty of the endoscope images, saving the screening time of the endoscope images and improving the quality and accuracy of the subsequent endoscope report generation.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic flow chart of a method for screening an image of an endoscope in the upper gastrointestinal tract according to an embodiment of the present invention;

FIG. 2 is a schematic flow chart of a method for screening an image of an endoscope in the upper gastrointestinal tract according to an embodiment of the present invention;

FIG. 3 is another schematic flow chart of a method for screening an image of an endoscope in the upper gastrointestinal tract according to an embodiment of the present invention;

FIG. 4 is another schematic flow chart of a method for screening an image of an endoscope in the upper gastrointestinal tract according to an embodiment of the present invention;

FIG. 5 is another schematic flow chart of a method for screening an image of an endoscope in the upper gastrointestinal tract according to an embodiment of the present invention;

FIG. 6 is another schematic flow chart of a method for screening an image of an endoscope in the upper gastrointestinal tract according to an embodiment of the present invention;

FIG. 7 is another schematic flow chart of a method for screening an image of an endoscope in the upper gastrointestinal tract according to an embodiment of the present invention;

FIG. 8 is a schematic view of a screening apparatus for an image of an endoscope in the upper gastrointestinal tract according to an embodiment of the present invention;

FIG. 9 is a schematic block diagram of a computer apparatus provided by an embodiment of the present invention;

fig. 10 is a flowchart of a specific application provided by an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

Referring to fig. 1, fig. 1 is a schematic flow chart illustrating a method for screening an image of an endoscope in an upper gastrointestinal tract according to an embodiment of the present invention. The method for screening the upper gastrointestinal endoscope image map is applied to the terminal equipment and is executed through application software installed in the terminal equipment. The terminal device is a terminal device with an internet access function, such as a desktop computer, a notebook computer, a tablet computer, or a mobile phone.

The method for screening the image of the upper gastrointestinal endoscope will be described in detail below.

As shown in FIG. 1, the method includes the following steps S110 to S140.

S110, inputting each endoscope image in the endoscope image set of the upper digestive tract into a pre-trained part classification model to obtain the part type of each endoscope image.

In this embodiment, the endoscope image atlas is an image atlas collected by an endoscopy physician in the upper gastrointestinal tract of a patient through an esophagogastroduodenoscope, and the region classification model is trained in advance and is used for performing region classification on each endoscope image in the endoscope image atlas. When an endoscopy physician performs image acquisition in the upper gastrointestinal tract of a patient through an esophagogastroduodenoscope, 26 parts on the upper gastrointestinal tract of the patient need to be subjected to image acquisition, and each part acquires multiple images. Wherein, 26 parts on the upper digestive tract are respectively: esophagus, cardia, antrum greater curvature, antrum posterior wall, antrum anterior wall, antrum lesser curvature, duodenal bulbus, duodenum descending part, euscopic corpus inferior greater curvature, euscopic corpus inferior posterior wall, euscopic corpus inferior anterior wall, euscopic corpus inferior lesser curvature, euscopic corpus superior greater curvature, euscopic corpus superior posterior wall, euscopic corpus superior anterior wall, euscopic corpus superior lesser curvature, inverted corpus fundus greater curvature, inverted corpus fundus posterior wall, inverted fundus ventral wall, inverted corpus superior posterior wall, inverted corpus superior lesser curvature, inverted corpus superior angle posterior wall, inverted corpus superior angle posterior wall, inverted gastrohorn anterior wall, and inverted gastrohorn lesser curvature.

Specifically, the part classification model is constructed by a ResNet50 neural network, in the training process of the part classification model, an image marked with a part type label is input into the part classification model in advance, then the part classification model outputs a multi-dimensional column vector, each dimension in the multi-dimensional column vector corresponds to the probability of one part, then network parameters of the part classification model are adjusted according to the maximum probability value until the part classification model reaches a convergence state, and then the training of the part classification model can be completed.

In other embodiments of the present invention, as shown in fig. 2, before step S110, steps S210 and S220 are further included.

S210, receiving an endoscope video of the upper digestive tract;

s220, preprocessing the endoscope video to obtain the endoscope image atlas.

Specifically, the endoscope image atlas is obtained from a video shot by an endoscopy physician in the upper gastrointestinal tract of a patient through an esophagogastroduodenoscope, the equipment decodes the endoscope video after receiving the endoscope video of the endoscope image atlas to obtain an image atlas of the endoscope video, and then performs size normalization processing on each image in the image atlas to obtain the endoscope image atlas. The image size normalization processing is mainly used for conveniently performing target detection and classification with unified standards on each image subsequently, and further screening out images which accord with the generated endoscope report.

In other inventive embodiments, as shown in FIG. 3, step S110 includes sub-steps S111 and S112.

S111, inputting each endoscope image map into the part classification model to obtain a first multi-dimensional column vector of each endoscope image map;

and S112, determining the part type of each endoscope image according to the first multi-dimensional column vector.

Specifically, the part classification model is constructed by a ResNet50 neural Network, wherein ResNet is an abbreviation of a Residual Network (Residual Network), and the Network is widely used in the fields of target classification and the like and is used as a part of a computer vision task main neural Network. After the endoscope image map is input into the part classification model, the part classification model outputs a multi-dimensional column vector of the endoscope image map, each dimension represents the probability of the part to which the endoscope image map belongs, and the higher the probability value is, the more likely the endoscope image map belongs to the category with the largest probability value is. For example [1, 0, 0, …, 0] represents the classification of the endoscopic image as a first site category in the upper digestive tract. After each endoscope image map in endoscope image map set carries out the position classification, according to the position classification of endoscope image map with endoscope image map storage to the folder of corresponding classification to all endoscope image maps carry out the target detection in the subsequent folder to every position classification, thereby screen out the endoscope image map that has the target area.

And S120, inputting all the endoscope image maps of each part into a pre-trained target detection model to obtain a target image map set of each part.

Specifically, the target detection model is trained in advance and is used for performing focus detection on an endoscope image map of each part to obtain whether a focus area exists in the endoscope image map or not, the target detection model is constructed by a Yolov3 network, the size of the endoscope image map of each part input into the target detection model is 352 x 352, after the endoscope image map is input into the target detection model, feature extraction is performed through a Darknet-53 feature network in the Yolov3 network, and the predicted values of 3 channels are finally output, wherein the predicted values of 3 channels respectively correspond to targets with different scales in large, medium and small three, and if the predicted values of 3 channels are 0, the endoscope image map is obtained to have no focus area.

In other embodiments of the present invention, as shown in fig. 4, before step S120, steps S310 and S320 are further included.

S310, inputting a preset training set into the target detection model to obtain coordinate loss, confidence coefficient loss and category loss of the target detection model;

s320, updating the network parameters of the target detection model according to the coordinate loss, the confidence coefficient loss and the category loss.

In this embodiment, the loss of the target detection model is composed of three parts, namely, a coordinate loss, a confidence loss and a category loss, where the coordinate loss is an error generated by generating coordinate information of a candidate frame containing a lesion region when the target detection model generates the candidate frame containing the lesion region of the endoscopic image map, the confidence is a probability that a lesion exists in each candidate frame, the confidence loss is an error for calculating a probability that a lesion exists in each candidate frame, and the category loss is an error whether a lesion exists in the endoscopic image map. The loss of the target detection model is expressed as: loss = lbox + lobj + lcls, where lbox is coordinate Loss, lobj is confidence Loss, and lcls is category Loss;

the coordinate loss function is:

the confidence loss function is:

the class loss function is:

where S × S is the mesh size, the number of candidate frames generated per mesh is B,

the jth Anchor Box representing the ith grid is responsible for this Object, and is 1 if it is responsible, or is 0 otherwise.

In addition, the training set is composed of an image set labeled with a focus region label, images in the image set need to be uniformly scaled to a size required by the target detection model before focus region labeling is carried out, after each image in the training set is input into the target detection model, the training of the target detection model can be completed by calculating the coordinate loss, the confidence loss and the category loss of the target detection model, and then adjusting the network parameters of the target detection model according to the coordinate loss, the confidence loss and the category loss until the target detection model reaches a convergence state.

In other inventive embodiments, as shown in FIG. 5, step S320 includes sub-steps S321 and S322.

S321, calculating the intersection ratio loss of the target detection model;

s322, updating the network parameters of the target detection model according to the intersection ratio loss, the coordinate loss, the confidence coefficient loss and the category loss.

Specifically, the intersection ratio loss is the intersection ratio loss between candidate frames of the image generated by the target detection model, and the network parameters of the target detection model are further updated and adjusted by calculating the intersection ratio loss generated by the target detection model, so that the target detection accuracy of the target detection model is better improved. Wherein IoU is an intersection ratio, which is a concept used in target detection and indicates the overlapping rate or degree of the candidate box and the original marked box, i.e. the ratio of the intersection and union of the candidate box and the original marked box, IoU is used to measure the overlapping degree of the predicted box and the real box in target detection.

In this embodiment, in order to consider the distance, overlap ratio and scale between the target and the Anchor, the cross-over ratio (IoU) loss is modified to Complete-IoU loss, which is the function:

where α is a weight function, v is a similarity parameter for measuring an aspect ratio, ρ represents an euclidean distance, b represents a center point, c represents a diagonal distance of a minimum bounding rectangle, and A, B are two intersecting candidate boxes, respectively.

S130, classifying each target image in the target image set according to a pre-trained classification model to obtain a classification result of each target image.

Specifically, the classification model is trained in advance and is used for classifying the types of the focus of an endoscope image map containing the focus, a training set required by the training of the classification model is formed by images marked with the types of the body of the range of the focus in advance, wherein the specific types of the focus comprise: macular tumors, polyps, erosion, ridges, barrett's esophagus, gastric mucosal ectopy, reflux esophagitis and the like, the classification model is constructed by a ResNet50 neural network, the image marked with a specific type label of the focus is input into the classification model in advance during the training process of the classification model, then the classification model outputs a multi-dimensional column vector, wherein each dimension in the multi-dimensional column vector corresponds to the probability of one part, then, the network parameters of the classification model are adjusted according to the maximum probability value until the classification model reaches a convergence state, and then the training of the classification model can be completed, after the training of the classification model is finished, each target image in the target image set is input into the trained classification model, and then outputting a multi-dimensional column vector of the target image map by the classification model, and finally obtaining the category of the target image map through the probability corresponding to each dimension in the multi-dimensional column vector.

In other inventive embodiments, as shown in fig. 6, step S130 includes sub-steps S131, S132, and S133.

S131, acquiring a minimum circumscribed rectangular frame of a detection target in each target image map;

s132, inputting the minimum circumscribed rectangle frame into the classification model to obtain a second multi-dimensional column vector of each target image map;

and S133, determining the type of each target image map according to the second multi-dimensional column vector.

In this embodiment, the classification model is constructed by a ResNet50 neural network, and the minimum bounding rectangle of the detection target in the target image map is obtained by a convex hull algorithm and a rotating caliper algorithm.

The core idea of the convex hull algorithm is as follows: a polygon is formed by a given number of points, which just frames the given point, while all vertices in the polygon are composed of the given part of points. The rotating caliper algorithm is to clamp and rotate two vertexes of the polygon formed by the convex hull algorithm to obtain the maximum distance and the minimum distance of the two vertexes in the polygon to generate the minimum rectangle of the polygon.

And S140, screening the endoscope image in the target image set of each part according to the classification result to obtain the screened endoscope image.

Specifically, the classification result is the probability that the focus in the target image map is the identified disease type, the probability of each target image map in a plurality of target image maps of the same disease type in the same part is ranked, and then the target image map with the maximum probability is selected from the ranked target image maps, so that the target image map can be used as the screened endoscope image map. And only the target image which is easier to generate a high-quality endoscope report is reserved for the plurality of target images of the same part and the same disease category, and the target image is the endoscope image after final screening.

In other inventive embodiments, as shown in FIG. 7, step S140 includes sub-steps S141 and S142.

S141, obtaining confidence degrees of a plurality of target image maps of the same type in each part according to the second multi-dimensional column vector;

and S142, screening the plurality of target images according to the confidence degrees to obtain the screened endoscope image map.

Specifically, the confidence degrees of a plurality of candidate frames output by the disease detection model for each target image map are obtained, then a plurality of target image maps of the same disease type and with the confidence degrees higher than a preset threshold value are selected, the probability of the classification model for outputting the plurality of target image maps is obtained, and finally the target image map with the maximum probability is selected from the target image maps, so that the target image map can be used as the screened endoscope image map, the high-quality endoscope report is ensured to be finally generated, the workload of doctors is reduced, and the generation efficiency of the endoscope report is improved.

In the method for screening the upper gastrointestinal endoscope image map provided by the embodiment of the invention, the part category of each endoscope image map is obtained by inputting each endoscope image map in the upper gastrointestinal endoscope image map set into a pre-trained part classification model; inputting all endoscope image maps of each part into a pre-trained target detection model to obtain a target image map set of each part; classifying each target image in the target image set according to a pre-trained classification model to obtain a classification result of each target image; and screening the endoscope image in the target image set of each part according to the classification result to obtain the screened endoscope image. The method classifies the types of the parts of each image in the endoscope image set in advance, detects the target of each image in the endoscope image set of each part to obtain the target image set of each part, and finally classifies and screens the target image set of each part to complete the screening of the images in the endoscope image set, thereby reducing the screening difficulty of the endoscope image set, saving the screening time of the endoscope image set, shortening the time for selecting the images in the report writing work of doctors, improving the diagnosis and treatment efficiency, and improving the quality and the accuracy of the subsequent endoscope report generation.

As shown in fig. 10, in the specific application process of the method for screening an upper gastrointestinal endoscopy image according to the present invention, a video of an upper gastrointestinal endoscopy image is preprocessed to obtain an endoscopy image set, then each piece of endoscopy image is subjected to part classification by a pre-trained part classification model to obtain a part type of each piece of endoscopy image, each piece of endoscopy image in each part is input into a target detection model to perform target detection to obtain a target image set of each part, each piece of target image in the target image set is classified to obtain a type of each piece of target image, and image screening is performed according to the type, so as to achieve simplification of the endoscopy image set. Wherein the target detection model is the lesion detection model in fig. 10.

The embodiment of the invention also provides a device 100 for screening the upper gastrointestinal endoscope image map, which is used for executing any one embodiment of the method for screening the upper gastrointestinal endoscope image map.

Specifically, referring to fig. 8, fig. 8 is a schematic block diagram of a screening apparatus 100 for an image of an endoscope in the upper gastrointestinal tract according to an embodiment of the present invention.

As shown in fig. 8, the screening apparatus 100 for an image of an upper gastrointestinal endoscope comprises: a first input unit 110, a second input unit 120, a first classification unit 130, and a first screening unit 140.

The first input unit 110 is configured to input each endoscopic image in an endoscopic image set of the upper gastrointestinal tract into a pre-trained part classification model, so as to obtain a part type of each endoscopic image.

In another embodiment, the screening apparatus 100 for an image map of an upper gastrointestinal endoscope further comprises: a receiving unit and a preprocessing unit.

A receiving unit for receiving an endoscope video of the upper digestive tract.

And the preprocessing unit is used for preprocessing the endoscope video to obtain the endoscope image atlas.

In another embodiment, the first input unit 110 includes: a third input unit and a first determination unit.

The third input unit is used for inputting each endoscope image map into the part classification model to obtain a first multi-dimensional column vector of each endoscope image map; and the first determining unit is used for determining the part type of each endoscope image according to the first multi-dimensional column vector.

The second input unit 120 is configured to input all the endoscopic image maps of each portion into a pre-trained target detection model, so as to obtain a target image atlas of each portion.

In another embodiment, the screening apparatus 100 for an image map of an upper gastrointestinal endoscope further comprises: a fourth input unit and a first updating unit.

And the fourth input unit is used for inputting a preset training set into the target detection model to obtain the coordinate loss, the confidence coefficient loss and the category loss of the target detection model.

And the first updating unit is used for updating the network parameters of the target detection model according to the coordinate loss, the confidence coefficient loss and the category loss.

In another embodiment, the first updating unit includes: a calculation unit and a second update unit.

The calculation unit is used for calculating the intersection ratio loss of the target detection model; and the second updating unit is used for updating the network parameters of the target detection model according to the intersection ratio loss, the coordinate loss, the confidence coefficient loss and the category loss.

The first classification unit 130 is configured to classify each target image in the target image set according to a pre-trained classification model, so as to obtain a classification result of each target image.

In another embodiment, the first classification unit 130 includes: the device comprises a first acquisition unit, a fifth input unit and a second determination unit.

The first acquisition unit is used for acquiring a minimum circumscribed rectangular frame of a detection target in each target image map; a fifth input unit, configured to input the minimum bounding rectangle into the classification model, so as to obtain a second multi-dimensional column vector of each target image map; and the second determining unit is used for determining the type of each target image map according to the second multi-dimensional column vector.

The first screening unit 140 is configured to perform screening of an endoscope image in the target image set of each location according to the classification result to obtain a screened endoscope image.

In another embodiment, the first screening unit 140 includes: a second obtaining unit and a second screening unit.

The second acquisition unit is used for acquiring the confidence degrees of a plurality of target image maps of the same type in each part according to the second multi-dimensional column vector; and the second screening unit is used for screening the plurality of target images according to the confidence degrees to obtain the screened endoscope image map.

The screening device 100 for the upper gastrointestinal endoscope image map provided by the embodiment of the invention is used for inputting each endoscope image map in the upper gastrointestinal endoscope image map set into a pre-trained part classification model to obtain the part category of each endoscope image map; inputting all endoscope image maps of each part into a pre-trained target detection model to obtain a target image map set of each part; classifying each target image in the target image set according to a pre-trained classification model to obtain a classification result of each target image; and screening the endoscope image in the target image set of each part according to the classification result to obtain the screened endoscope image.

Referring to fig. 9, fig. 9 is a schematic block diagram of a computer device according to an embodiment of the present invention.

Referring to fig. 9, the device 500 includes a processor 502, memory, and a network interface 505 connected by a system bus 501, where the memory may include a storage medium 503 and an internal memory 504.

The storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032, when executed, causes the processor 502 to perform a method for screening an image of an upper gastrointestinal endoscope.

The processor 502 is used to provide computing and control capabilities that support the operation of the overall device 500.

The internal memory 504 provides an environment for running the computer program 5032 in the non-volatile storage medium 503, and when the computer program 5032 is executed by the processor 502, the processor 502 can be enabled to execute the method for screening the image of the upper gastrointestinal tract endoscope.

The network interface 505 is used for network communication, such as providing transmission of data information. Those skilled in the art will appreciate that the configuration shown in fig. 9 is a block diagram of only a portion of the configuration associated with aspects of the present invention and does not constitute a limitation of the apparatus 500 to which aspects of the present invention may be applied, and that a particular apparatus 500 may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

Wherein the processor 502 is configured to run the computer program 5032 stored in the memory to implement the following functions: inputting each endoscope image in the endoscope image set of the upper digestive tract into a pre-trained part classification model to obtain the part category of each endoscope image; inputting all endoscope image maps of each part into a pre-trained target detection model to obtain a target image map set of each part; classifying each target image in the target image set according to a pre-trained classification model to obtain a classification result of each target image; and screening the endoscope image in the target image set of each part according to the classification result to obtain the screened endoscope image.

Those skilled in the art will appreciate that the embodiment of the apparatus 500 shown in fig. 9 does not constitute a limitation on the specific construction of the apparatus 500, and in other embodiments, the apparatus 500 may include more or fewer components than shown, or some components may be combined, or a different arrangement of components. For example, in some embodiments, the apparatus 500 may only include the memory and the processor 502, and in such embodiments, the structure and function of the memory and the processor 502 are the same as those of the embodiment shown in fig. 9, and are not repeated herein.

It should be understood that in the present embodiment, the Processor 502 may be a Central Processing Unit (CPU), and the Processor 502 may also be other general-purpose processors 502, a Digital Signal Processor 502 (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. The general-purpose processor 502 may be a microprocessor 502 or the processor 502 may be any conventional processor 502 or the like.

In another embodiment of the present invention, a computer storage medium is provided. The storage medium may be a nonvolatile computer-readable storage medium or a volatile storage medium. The storage medium stores a computer program 5032, wherein the computer program 5032 when executed by the processor 502 performs the steps of: inputting each endoscope image in the endoscope image set of the upper digestive tract into a pre-trained part classification model to obtain the part category of each endoscope image; inputting all endoscope image maps of each part into a pre-trained target detection model to obtain a target image map set of each part; classifying each target image in the target image set according to a pre-trained classification model to obtain a classification result of each target image; and screening the endoscope image in the target image set of each part according to the classification result to obtain the screened endoscope image.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses, devices and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the embodiments provided by the present invention, it should be understood that the disclosed apparatus, device and method can be implemented in other ways. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only a logical division, and there may be other divisions when the actual implementation is performed, or units having the same function may be grouped into one unit, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may also be an electric, mechanical or other form of connection.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present invention.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a storage medium. Based on such understanding, the technical solution of the present invention essentially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium and including instructions for causing a device 500 (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk.

While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for screening an image map of an upper gastrointestinal endoscope is characterized by comprising the following steps:

2. The method for screening an endoscopic image map of the upper gastrointestinal tract according to claim 1, wherein before inputting each endoscopic image map in the endoscopic image map set of the upper gastrointestinal tract into a pre-trained region classification model and obtaining a region classification of each endoscopic image map, the method further comprises:

receiving an endoscopic video of the upper digestive tract;

and preprocessing the endoscope video to obtain the endoscope image atlas.

3. The method for screening an upper gastrointestinal endoscope image map according to claim 1, wherein the step of inputting each endoscope image map in an upper gastrointestinal endoscope image map set into a pre-trained part classification model to obtain the part category of each endoscope image map comprises:

inputting each endoscope image map into the part classification model to obtain a first multi-dimensional column vector of each endoscope image map;

and determining the part type of each endoscope image according to the first multi-dimensional column vector.

4. The method for screening upper gastrointestinal endoscopy images of claim 1, wherein before inputting all endoscopic images of each site into a pre-trained target detection model to obtain a target image set of each site, the method further comprises:

inputting a preset training set into the target detection model to obtain the coordinate loss, the confidence coefficient loss and the category loss of the target detection model;

and updating the network parameters of the target detection model according to the coordinate loss, the confidence coefficient loss and the category loss.

5. The method for screening an image map of an endoscope in the upper gastrointestinal tract according to claim 4, wherein the updating the network parameters of the target detection model according to the coordinate loss, the confidence loss and the category loss comprises:

calculating the intersection ratio loss of the target detection model;

and updating the network parameters of the target detection model according to the intersection ratio loss, the coordinate loss, the confidence coefficient loss and the category loss.

6. The method for screening an image map of an upper gastrointestinal endoscope according to claim 1, wherein the classifying each target image map in the target image map set according to a pre-trained classification model to obtain a classification result of each target image map comprises:

acquiring a minimum circumscribed rectangular frame of a detection target in each target image map;

inputting the minimum circumscribed rectangle frame into the classification model to obtain a second multi-dimensional column vector of each target image map;

and determining the type of each target image map according to the second multi-dimensional column vector.

7. The method for screening an upper gastrointestinal endoscope image map according to claim 6, wherein the step of screening an endoscope image map in the target image map set of each part according to the classification result to obtain a screened endoscope image map comprises:

obtaining confidence degrees of a plurality of target images of the same type in each part according to the second multi-dimensional column vector;

and screening the plurality of target images according to the confidence degrees to obtain the screened endoscope image.

8. The utility model provides a sieving mechanism of upper gastrointestinal endoscope image map which characterized in that includes:

9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method for upper gastrointestinal endoscope image screening according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by a processor, causes the processor to execute the method of screening an image map of an upper gastrointestinal endoscope according to any one of claims 1 to 7.