CN116740363B - Cleanliness detection method and system based on intestinal region segmentation - Google Patents

Abstract

The invention provides a cleanliness detection method and a cleanliness detection system based on intestinal region segmentation, which are characterized in that a plurality of intestinal detection sample images are obtained for pixel-level semantic annotation to obtain an intestinal annotation sample image set; performing effective frame judgment pretreatment, completing image standardization and data enhancement operation, and obtaining an intestinal tract training sample image set; inputting the intestinal semantic segmentation network for iterative training to obtain an intestinal foreign matter semantic segmentation model; performing semantic segmentation on the video of the intestinal canal to be detected to obtain a foreign matter identification result of the segmented image of the intestinal canal to be detected, and integrating the foreign matter identification result to obtain the cleanliness of the foreign matter of the intestinal canal to be detected; and selecting the section of the intestinal tract to be detected with the lowest foreign matter cleanliness according to the detection duration of the intestinal tract to be detected, and outputting to form a cleanliness detection report. According to the invention, the region segmentation is performed on the intestinal tract by accurately utilizing the mirror withdrawal rate of the intestinal tract detection, the iterative training is performed on different characteristics of the intestinal tract in an adaptive manner, and better robustness is achieved for the detection of the cleanliness of the intestinal tract.

Description

Cleanliness detection method and system based on intestinal region segmentation

Technical Field

The application mainly relates to the technical field of artificial intelligence, in particular to a cleanliness detection method and system based on intestinal region segmentation.

Background

According to the chinese common new cancer data display (IARC panel data) released by the world health organization 2020, orthocolon cancer became the second most common cancer next to lung cancer. Colonoscopy can detect and resect intestinal tumors, reduces the occurrence of colorectal cancer, and is a gold standard for screening and diagnosis of colon cancer. However, the success of colonoscopy requires reliance on the quality of intestinal cleanliness, requiring clear visualization of the colonic mucosa, otherwise there are conditions of missed lesions and post-operative infections. Thus, bowel cleansing preparation is a prerequisite for colonoscopy.

Although high-quality intestinal canal preparation work is crucial, most patients still have insufficient intestinal canal preparation when receiving colonoscopy, and foreign matters such as fecal residues, fecal liquid and the like exist in the intestinal canal to shield intestinal mucosa, so that the detection rate of diseases is greatly influenced. Moreover, in performing colonoscopy, the intestinal tract cleaning score requires that the endoscopist complete qualitative scoring by means of memory after the colonoscopy is completed by observing the amount of intestinal foreign material in real time. The large number of colonoscopies gives endoscopists a tremendous amount of work, while also interfering with the accuracy of the physician's score for intestinal quality.

With the rapid development and application of computer vision, some artificial intelligence techniques are based on classification networks, classifying intestinal cleanliness into 4 classes according to BOSTON scoring criteria (BOSTON), and studying the intestinal cleanliness scores.

Disclosure of Invention

In order to reduce the fatigue of cervical vertebra and eyes of doctors, and simultaneously greatly improve the working efficiency and the diagnosis efficiency and accuracy of intestinal lesions, the invention discloses a cleanliness detection method and a cleanliness detection system based on intestinal region segmentation.

According to a first aspect of the present invention, the present invention claims a method for detecting cleanliness based on intestinal region segmentation, comprising:

acquiring a plurality of intestinal tract detection sample images, and performing pixel-level semantic annotation on the intestinal tract detection sample images to obtain an intestinal tract annotation sample image set;

performing effective frame judgment pretreatment on the intestinal tract labeling sample image set, and performing image standardization and data enhancement operation on the effective frame to obtain an intestinal tract training sample image set;

inputting the intestinal training sample image set into an intestinal semantic segmentation network for iterative training to obtain an intestinal foreign matter semantic segmentation model;

performing semantic segmentation on the video of the intestinal canal to be detected according to the intestinal canal foreign matter semantic segmentation model to obtain a foreign matter identification result of the intestinal canal segmentation image to be detected;

Integrating the foreign matter identification result to obtain the foreign matter cleanliness of the intestinal tract to be detected;

and selecting the section of the intestinal tract to be detected with the lowest foreign matter cleanliness to output according to the foreign matter cleanliness and the detection duration of the intestinal tract to be detected, and forming a cleanliness detection report.

According to a second aspect of the present invention, the present invention claims a cleanliness detection system based on intestinal region segmentation, comprising:

the labeling module is used for acquiring a plurality of intestinal tract detection sample images, and performing pixel-level semantic labeling on the intestinal tract detection sample images to obtain an intestinal tract labeling sample image set;

the preprocessing module is used for carrying out effective frame judgment preprocessing on the intestinal tract labeling sample image set, and carrying out image standardization and data enhancement operation on the effective frames to obtain an intestinal tract training sample image set;

the training module inputs the intestinal training sample image set into an intestinal semantic segmentation network for iterative training to obtain an intestinal foreign object semantic segmentation model;

the semantic segmentation module performs semantic segmentation on the video of the intestinal canal to be detected according to the intestinal canal foreign matter semantic segmentation model to obtain a foreign matter identification result of the intestinal canal segmentation image to be detected;

The cleanliness analysis module is used for integrating the foreign matter identification result to obtain the cleanliness of the foreign matters in the intestinal tract to be detected;

and the output module is used for selecting the section of the intestinal tract to be detected with the lowest foreign matter cleanliness to output according to the foreign matter cleanliness and the detection duration of the intestinal tract to be detected, so as to form a cleanliness detection report.

The invention provides a cleanliness detection method and a cleanliness detection system based on intestinal region segmentation, which are characterized in that a plurality of intestinal detection sample images are obtained for pixel-level semantic annotation to obtain an intestinal annotation sample image set; performing effective frame judgment pretreatment, completing image standardization and data enhancement operation, and obtaining an intestinal tract training sample image set; inputting the intestinal semantic segmentation network for iterative training to obtain an intestinal foreign matter semantic segmentation model; performing semantic segmentation on the video of the intestinal canal to be detected to obtain a foreign matter identification result of the segmented image of the intestinal canal to be detected, and integrating the foreign matter identification result to obtain the cleanliness of the foreign matter of the intestinal canal to be detected; and selecting the section of the intestinal tract to be detected with the lowest foreign matter cleanliness according to the detection duration of the intestinal tract to be detected, and outputting to form a cleanliness detection report. According to the invention, the region segmentation is performed on the intestinal tract by accurately utilizing the mirror withdrawal rate of the intestinal tract detection, the iterative training is performed on different characteristics of the intestinal tract in an adaptive manner, and better robustness is achieved for the detection of the cleanliness of the intestinal tract.

Drawings

FIG. 1 is a flowchart of a method for detecting cleanliness based on segmentation of intestinal regions according to the present application;

FIG. 2 is a network frame diagram of a method for detecting cleanliness based on intestinal region segmentation according to the present application;

FIG. 3 is a block diagram of the division of the feature map input into Queryinst as claimed in the present application;

FIG. 4 is a network structure diagram of a backbone network bottleneck layer of a method for detecting cleanliness based on intestinal region segmentation according to the present application;

fig. 5 is a block diagram of a cleanliness detection system based on intestinal region segmentation according to the present application.

Detailed Description

The following description will be given in detail of the technical solutions in the embodiments of the present application with reference to the accompanying drawings. Wherein, in the description of the embodiments of the present application, unless otherwise indicated, "/" means or, for example, a/B may represent a or B; the text "and/or" is merely an association relation describing the associated object, and indicates that three relations may exist, for example, a and/or B may indicate: the three cases where a exists alone, a and B exist together, and B exists alone, and furthermore, in the description of the embodiments of the present application, "plural" means two or more than two.

The terms "first," "second," and the like, are used below for descriptive purposes only and are not to be construed as implying or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature, and in the description of embodiments of the application, unless otherwise indicated, the meaning of "a plurality" is two or more.

According to a first embodiment of the present application, referring to fig. 1, the present application claims a cleanliness detection method based on intestinal region segmentation, comprising:

acquiring a plurality of intestinal tract detection sample images, and performing pixel-level semantic annotation on the intestinal tract detection sample images to obtain an intestinal tract annotation sample image set;

performing effective frame judgment pretreatment on the intestinal tract labeling sample image set, and performing image standardization and data enhancement operation on the effective frame to obtain an intestinal tract training sample image set;

inputting the intestinal training sample image set into an intestinal semantic segmentation network for iterative training to obtain an intestinal foreign matter semantic segmentation model;

performing semantic segmentation on the video of the intestinal canal to be detected according to the intestinal canal foreign matter semantic segmentation model to obtain a foreign matter identification result of the intestinal canal segmentation image to be detected;

Integrating the foreign matter identification result to obtain the cleanliness of the foreign matters in the intestinal tract to be detected;

and selecting the section of the intestinal tract to be detected with the lowest foreign matter cleanliness to output according to the foreign matter cleanliness and the detection duration of the intestinal tract to be detected, and forming a cleanliness detection report.

Further, a plurality of intestinal tract detection sample images are obtained, pixel-level semantic labeling is carried out on the intestinal tract detection sample images, and an intestinal tract labeling sample image set is obtained, specifically comprising:

acquiring a plurality of intestinal tract detection sample images from a colonoscopy historical video;

dividing an intestinal tract detection sample image into a background image and a foreign object image, wherein the background image comprises a clean intestinal mucosa area, a blurring area, a flushing area, a reflecting area and a pixel area of a surgical instrument, and the foreign object image comprises an intestinal tract inner excrement block, excrement liquid, a bulk bubble and a yellow mucosa range which causes visual interference due to excrement;

and selecting a pixel point area belonging to the intestinal foreign matter through a polygonal labeling tool frame, and carrying out pixel-level labeling on the image to obtain an intestinal labeling sample image set.

Further, performing effective frame judgment pretreatment on the intestinal tract labeling sample image set, and performing image standardization and data enhancement operation on the effective frame to obtain an intestinal tract training sample image set, wherein the method specifically comprises the following steps:

Preprocessing an intestinal labeling sample image set, taking a dyed, flushed, reflected and blurred image in the intestinal labeling sample image set as an invalid frame image, and taking an image containing intestinal foreign matters and intestinal mucosa as an effective frame image;

according to the data volume of the effective frame image in the intestinal labeling sample image set, according to the ineffective frame image: valid frame image = 1:10, selecting part of invalid frame images as negative samples, and removing redundant invalid frame images;

performing image standardization and data enhancement operation on all the effective frame images to obtain an intestinal training sample image set;

the image standardization realizes the centralization treatment through removing the mean value, and in the data enhancement, the enhancement of space class and color transformation class is carried out;

the spatial class includes operations of translating, flipping, and rotating the image, and the color transformation class includes noise, brightness, contrast, and blurring operations.

Further, inputting the intestinal training sample image set into an intestinal semantic segmentation network for iterative training to obtain an intestinal foreign object semantic segmentation model, which specifically comprises the following steps:

each pixel in the image set of the intestinal training sample is distributed to different object examples by adopting an end-to-end example segmentation method Queryinst, so as to obtain an intestinal training sample segmentation feature map;

Adopting RegNetX-3.2GF as a backbone network backbone, inputting the intestinal training sample segmentation feature map into a candidate frame and an interested region, wherein the interested region head structure performs fine tuning refinement on the generated candidate frame, and completing candidate frame classification and target position detection;

the method comprises the steps of acquiring characteristics of each intestinal training sample segmentation characteristic diagram based on a multi-head self-attention module and dynamic convolution by adopting a query object detector, sharing mask heads of different branches by using three parallel dynamic mask heads, and mutually utilizing advanced semantic characteristics;

each pixel in the image set of the intestinal training sample is distributed to different object examples by adopting an end-to-end example segmentation method Queryinst to obtain an intestinal training sample segmentation feature map, which comprises the following steps:

inputting an intestinal training sample image with the size of 480 multiplied by 480 into a first stage, and acquiring a first intestinal training sample segmentation feature map, wherein the size of the first intestinal training sample segmentation feature map is 120 multiplied by 120, and the first intestinal training sample segmentation feature map comprises mucosa features in a bottom intestinal mucosa of the intestinal training sample image;

inputting the first intestinal training sample segmentation feature map into a second stage, and acquiring a second intestinal training sample segmentation feature map, wherein the size of the second intestinal training sample segmentation feature map is 60×60, and the second intestinal training sample segmentation feature map comprises highlighting of a light region in the intestinal tract of an intestinal training sample image;

Inputting the second intestinal training sample segmentation feature map into a third stage and a fourth stage, and obtaining a third intestinal training sample segmentation feature map and a fourth intestinal training sample segmentation feature map, wherein the sizes of the third intestinal training sample segmentation feature map and the fourth intestinal training sample segmentation feature map are 30 multiplied by 30 and 15 multiplied by 15, and the segmentation region is formed by high-level semantic feature target foreign matters in the intestinal tract of the intestinal training sample image;

the neg uses an FPN feature pyramid to up-sample the segmentation feature map of the fourth intestinal training sample generated in the fourth stage and then uses the segmentation feature map of the fourth intestinal training sample to match with the third intestinal training sample in the third stageCarrying out 1X 1 convolution refinement on the segmentation feature map, and then carrying out residual connection to obtain a fifth intestinal training sample segmentation feature map f ₅ ；

Will f ₅ Up-sampling, performing 1×1 convolution refinement on the up-sampled and second intestinal training sample segmentation feature map in the second stage, and performing residual connection to integrate color and brightness features to obtain a sixth intestinal training sample segmentation feature map f ₆ ；

Will f ₆ Upsampling, performing 1×1 convolution refinement on the upsampled segmentation feature map of the first intestinal training sample in the first stage, and performing residual connection to integrate boundary information of foreign matters in the intestinal tract to obtain a segmentation feature map f of the seventh intestinal training sample ₇ ；

Specifically, extracting a plurality of candidate frames of a seventh intestinal training sample segmentation feature map, and initializing N anchor frames to obtain the intersection ratio of each anchor frame and the candidate frames;

setting a background separation preset threshold, wherein if the intersection ratio is larger than the background separation preset threshold, the region of the candidate frame is a foreground, otherwise, the region is a background;

inputting candidate boxes into the region of interest and giving a query Q _t Will Q _t Input to Multi-head self-attention Module to get feature enhanced query Q _t ^* Will Q _t ^* And the candidate frame is used as a clue, fine tuning of the refinement feature is performed under dynamic convolution, and a segmentation area of the first foreign object is generated under the segmentation head as a clue guide of the next stage;

repeating the steps of inquiring and inputting the multi-head self-attention module to obtain a segmentation area of the second foreign object and a segmentation area of the third foreign object;

fusing the first foreign body segmentation area, the second foreign body segmentation area and the third foreign body segmentation area to obtain an intestinal foreign body instance segmentation result;

and performing iterative training on the intestinal semantic segmentation network according to the intestinal foreign object example segmentation result to obtain an intestinal foreign object semantic segmentation model.

Wherein in this embodiment the entire network frame comprises four parts backbone, neck, rpn _head, roihead. In order to meet the real-time performance, the calculation cost is reduced. The RegNetX-3.2GF is adopted as a backbone network backbone, and the structure and feature diagram of the backbone network and the FPN are input into the query to realize segmentation, and the structure diagram is shown in fig. 2 and 3.

Stage1-4 represents four stages of a backbone network RegNetX-3.2GF, namely a first Stage, a second Stage, a third Stage and a fourth Stage, each Stage is composed of a plurality of bottleneck modules, and the bottleneck modules are shown in fig. 4.

In fig. 4, 1×1, 3×3 each represent a 1×1, 3×3 convolution operation with the relu activation formula; in Stage1-Stage4, there are 2, 6, 15, 2 bottleneck modules, bottleblock, respectively, and the number of 3×3 convolution packets per Stage is 2, 4, 9, 11, respectively. It is noted that there is a difference in the step size of the convolution operation among the different bottleneck modules, bottleBlock. I.e. the complement of the 3 x 3 convolution in the first bottleneck module of each stage is 2, the others are all 1. In table 1, s represents a step size, p represents a padding number, and g represents a grouping number.

TABLE 1

Further, performing semantic segmentation on the video of the intestinal canal to be detected according to the intestinal canal foreign matter semantic segmentation model to obtain a foreign matter identification result of the intestinal canal segmentation image to be detected, which specifically comprises:

when the condition that a back blind flap of the intestinal tract to be detected starts to a mirror-withdrawing process outside the body is identified, starting an intestinal foreign matter semantic segmentation model to carry out semantic segmentation on a video of the intestinal tract to be detected, and acquiring a mirror-withdrawing rate of the mirror-withdrawing process;

Dividing an intestinal tract to be detected into a ascending colon area, a transverse colon area and a descending colon area, and endowing the cleanliness of the three areas with different confidence degrees a1, a2 and a3;

the ascending colon area is close to the ileocecal valve, the foreign matter exists most in the area, and the area comprises solid manure and manure blocks, so that the influence on intestinal examination is large, and a1=1.2;

the transverse colon area is mainly composed of manure residues, partial small manure blocks exist, and interference is reduced through repeated flushing, so that a2=1.0;

the descending colon area comprises a descending colon and a sigmoid colon, the foreign matters are small-area fecal residues and residual fecal liquid, the influence is minimal, and a3=0.8;

dividing image frame boundaries of a ascending colon region, a transverse colon region and a descending colon region of the intestinal tract to be detected according to the scope-withdrawing rate, and acquiring an image frame set of the three regions;

and carrying out semantic segmentation on the image frame set according to the intestinal foreign matter semantic segmentation model, and obtaining a foreign matter identification result of the intestinal segmentation image to be detected through weighted calculation.

Wherein in this embodiment, the real-time segmentation is performed only during the mirror withdrawal (i.e., during the recognition of the beginning of the ileocecal valve to the outside of the body) and the intestinal cleanliness score is calculated. For why the intestinal cleanliness score is calculated only during the mirror-back procedure, there are the following reasons:

The goal of the boston intestinal scoring criteria is to provide for non-flushable removal of foreign matter from the intestinal tract, and during the endoscopic procedure, the surgeon typically reduces the diagnostic effort of the foreign matter from the intestinal tract by flushing. The mirror-withdrawing process is selected, namely the target is accurate, the interference of washable foreign matters is eliminated, and the method is closer to the Boston scoring in a real scene.

To complete the lower gastrointestinal surgery, the endoscope needs to be moved to the ileocecal valve, and in the process of entering the endoscope, the endoscope inevitably has operations such as large-amplitude rotation and the like in order to enter a deeper gastrointestinal region. This can lead to a large number of blurred, reflected images, etc., causing serious noise interference. In the process of lens withdrawal, a doctor can observe the bulge-shaped lesion, so that the endoscope is stable, and the segmentation accuracy of the model is facilitated.

The model only calls the model for the effective frame image in the mirror withdrawal process, and the segmentation result is output. This is because in the process of entering the lens, the situation such as rapid disturbance of the lens can affect the segmentation effect of the model.

If invalid frame pictures such as flushing, blurring and dyeing exist in the mirror-withdrawing process, the model can skip the frame number, and interference of the situation on model scoring is prevented.

Specifically, in this embodiment, the method for constructing the real-time enteroscopy rate segment includes determining a moving route of the enteroscopy, and obtaining rates and resistances of all the enteroscopy enteroscopies at a time T in each observation segment on the enteroscopy; taking the speed and resistance of all the speculum enteroscopes at the moment T of each observation section as input variables, taking the standard deviation of the displacement, the speed and the resistance of the speculum as output variables, defining the number of intestinal edge units as random decision units, and carrying out data analysis according to the size of the efficiency value to determine the speed section of each decision unit at the current moment; and constructing a lens transporting speed section between adjacent decision units by adopting a general interpolation method, connecting the speed sections of each observation section on the lens transporting route so as to form the lens transporting speed section on the lens transporting route at the current moment, and updating and iterating the lens transporting speed section at intervals of delta t.

On the basis of determining the moving route of the enteroscope, determining the number N and the distance D of intestinal marginal units on the moving route of the enteroscope by a medical detection system, and acquiring information such as the speed, the resistance and the like of all the enteroscopes passing through the corresponding N observation sections T at the moment by each intestinal marginal unit;

according to the obtained speed and resistance of each observation region enteroscope at the moment T, determining the mirror transporting speed section of each observation section at the moment T by adopting a general data analysis method;

Interpolation calculation is carried out on the upper boundary and the lower boundary of the mirror-transporting speed sections of the two adjacent observation sections by adopting a general interpolation calculation method respectively, so that the mirror-transporting speed sections of the two adjacent observation sections are connected to form a mirror-transporting speed section on a mirror-transporting route at the moment T;

as the enteroscope moves, the interval Δt iteratively updates the enteroscope rate segment once in time and space, thereby establishing a real-time enteroscope rate segment.

The range of the observation section is determined by the observation range corresponding to the intestinal limbal unit.

The specific method for acquiring the mirror conveying speed section of each observation section at the moment T comprises the following steps:

firstly, taking the number N of intestinal canal edge units on a transit mirror route as a random decision unit number, taking the speed and resistance of all the intestinal mirrors passing through each observation section T at the moment as input variables, and taking the displacement, the standard deviation of the speed and the standard deviation of the resistance of the intestinal mirrors as output variables;

analysis of C using general data ² And the R model establishes an evaluation formula and an optimization model, and calculates the efficiency value theta of each decision unit.

And transversely comparing theta values corresponding to input variable data of all the enteroscopes in the observation section, determining the upper extreme value and the lower extreme value of the speed of the enteroscopes in the observation section, wherein the larger the theta value is, the more effective the enteroscopy detection is, the smaller the theta value is or the input index is 0, and defaulting to the lower extreme value of the speed section.

The specific construction method of the real-time enteron transit mirror speed section comprises the following steps:

determining the position of the current enteroscope on the moving route at the moment T+delta T according to the time interval delta T on the time domain, and updating the information such as the number N of intestinal edge units, the interval and the like on the mirror conveying route according to the position of the current enteroscope at the moment T+delta T;

and acquiring data of the speed and the resistance of all the speculum enteroscopes of each observation section T+delta T according to the updated intestinal canal edge unit information, and then repeatedly completing updating of the speed section on the speculum enteroscope in a time domain and a space domain, thereby establishing a real-time speculum speed section.

The method takes all enteroscope speeds and resistances at the moment T of each observation section obtained in real time as input variables, takes the enteroscope displacement, speed standard deviation and resistance standard deviation as output variables, takes the number of intestinal edge units as decision units, and comprises the following specific calculation processes:

taking the number N of intestinal canal edge units on a transit mirror route as a random decision unit number, taking the speed and resistance of all the intestinal mirrors passing through each observation section T at the moment as input variables, and taking the displacement, the standard deviation of the speed and the standard deviation of the resistance of the intestinal mirrors as output variables;

the enteroscope speed and resistance are defined as input variables, and the enteroscope displacement, speed standard deviation and resistance standard deviation are defined as output variables, and the output variables are three. Wherein X is _ij Representing the speed and resistance value of the jth intestinal region to the ith enteroscope, X is ensured _ij >0, and thus takes absolute value for the resistance,Y _rj representing the output of the jth intestinal region to the jth enteroscopy output; v _i Metrics ("weights") representing ith enteroscopy rate and resistance, u _r Representing a measure of the output of the r-th enteroscope ("weight").

Two input indexes and three output indexes are selected together, and an evaluation formula is shown as formula 1:

wherein the weight coefficient satisfies h _j Is less than or equal to 1.

Further, the foreign matter recognition result is integrated to obtain the cleanliness of the foreign matters in the intestinal tract to be detected, which specifically comprises:

calculating the foreign matter area and the duty ratio of each image frame in the image frame set according to the foreign matter identification result;

classifying the foreign matters in the intestinal tract to be detected according to the area and the proportion of the foreign matters;

wherein p represents the area ratio of the foreign matter of each image frame, and a represents the confidence of different areas of the intestinal tract;

classification as class I small foreign bodies when p x a < 0.15%;

when 0.15% < pa <1.5%, classifying as foreign matter in class II;

when 1.5% < pa <15%, classifying into class III large foreign matter;

when pa >15%, classifying into class IV extra-large foreign bodies;

when foreign substances are classified into class III and class IV in an image frame, the intestinal cleanliness of the image frame is poor;

On the basis of each foreign matter classification, grading the intestinal cleanliness of the whole image frame according to the sum of the proportion of all foreign matter examples of the whole image frame, and dividing the image frame into 4 grades from high to low by using a Boston intestinal grading scale, wherein the grades are respectively 3, 2, 1 and 0;

the score is proportional to the cleanliness of the image frame.

Wherein in this embodiment, the confidence level may also be adjusted based on the mirror-down rate of the different regions.

Presetting a threshold rho of a mirror-withdrawal rate data peak Gu Chazhi;

selecting a starting point vt1 and an ending point vtn of the mirror withdrawal rate data intestinal region as initial data region points, calculating any data point confidence coefficient of the mirror withdrawal rate data intestinal region and the confidence coefficient of the current data region, and converting vt1 and vt2 into corresponding linked list nodes to be stored in an intestinal detection storage table;

selecting a data area with rate mutation from the current area by utilizing an intestinal area representation strategy according to the processing result, selecting a data point vtk with rate mutation from the area as a new data area point, and subdividing the current data area again;

judging the number num of data points in the current intestinal tract detection storage table _cur If the limit of rho is exceeded, if not, continuing to execute, otherwise, terminating the step, and completing the establishment of the corresponding intestinal detection storage table;

Through the intestinal detection memory table established based on the intestinal region and the corresponding DCR, the data multi-resolution simplified representation in the range of (0, ρ) under any data compression rate DCR is realized.

Calculating any data point confidence for the mirror-down rate data intestinal region, comprising:

assuming that a fitted straight line of the intestinal region of the withdrawal rate data is a line segment connecting the start point vt1 and the end point vtn, that is, the boundary = (vt 1, vtn) of the intestinal region, the confidence fe_spi of any data point vti on the intestinal region is a vertical distance from the data point vti to the boundary of the intestinal region, and the specific calculation is as shown in the formula:

in the mirror-back detection point v _t1 Including a specific time t ₁ Actual detected data value v at this point in time ₁ ，v _t1 ＝(v ₁ ，t ₁ ) The method comprises the steps of carrying out a first treatment on the surface of the Mirror-withdrawal detection point v _tn Including a specific time t _n Actual detected data value v at this point in time _n ，v _tn ＝(v _n ，t _n ) The method comprises the steps of carrying out a first treatment on the surface of the Mirror-withdrawal detection point v _ti Including a specific time t _i Actual detected data value v at this point in time _i ，v _ti ＝(v _i ，t _i ) The method comprises the steps of carrying out a first treatment on the surface of the Starting point v _t1 Confidence of (v) end point v _tn Confidence levels of (2) are all 0;

further, according to the cleanliness of the foreign matters and the detection duration of the intestinal canal to be detected, selecting the intestinal canal section to be detected with the lowest cleanliness of the foreign matters for outputting to form a cleanliness detection report, which specifically comprises:

the score output standard takes the score with lowest intestinal preparation quality score of a time period as the score of the time period at fixed time intervals; the cumulative score duty cycle is output after the intestinal examination is completed.

Judging the current distance that the endoscope has been retracted by the fe_spi, and further judging that the position belongs to the ascending colon, the transverse colon or the descending colon region.

In this embodiment, specific scoring details are as follows, where S represents the total foreign object ratio of the frame image and score represents the intestinal cleanliness score of the frame image.

(1) When all example foreign objects in the image are of class I, score=3 if S < 1.5%. Score=2 if 1.5% <=s < =2.0%. 2.0% <=s < =15.0%, score=1.

(2) When the largest example foreign matter in the image is class II, score=3 if S < 1.0%. Score=2 if 1.0% <=s < =2.0%. 2.0% <=s < =15.0%, score=1.

(3) When the largest example foreign matter in the image is class III, score=1.

(4) Score=0 when there is a class IV foreign object or S > =15% in the image.

According to a second embodiment of the present invention, referring to fig. 5, the present invention claims a cleanliness detection system based on intestinal region segmentation, comprising:

the labeling module is used for acquiring a plurality of intestinal tract detection sample images, and performing pixel-level semantic labeling on the intestinal tract detection sample images to obtain an intestinal tract labeling sample image set;

the preprocessing module is used for carrying out effective frame judgment preprocessing on the intestinal tract labeling sample image set, carrying out image standardization and data enhancement operation on the effective frame, and obtaining an intestinal tract training sample image set;

The training module inputs the intestinal training sample image set into an intestinal semantic segmentation network for iterative training to obtain an intestinal foreign matter semantic segmentation model;

the semantic segmentation module performs semantic segmentation on the video of the intestinal canal to be detected according to the intestinal canal foreign matter semantic segmentation model to obtain a foreign matter identification result of the intestinal canal segmentation image to be detected;

the cleanliness analysis module is used for integrating the foreign matter identification result to obtain the cleanliness of the foreign matters in the intestinal tract to be detected;

the output module is used for selecting the section of the intestinal tract to be detected with the lowest foreign matter cleanliness to output according to the foreign matter cleanliness and the detection duration of the intestinal tract to be detected, so as to form a cleanliness detection report.

Further, the training module specifically includes:

each pixel in the image set of the intestinal training sample is distributed to different object examples by adopting an end-to-end example segmentation method Queryinst, so as to obtain an intestinal training sample segmentation feature map;

adopting RegNetX-3.2GF as a backbone network backbone, inputting the intestinal training sample segmentation feature map into a candidate frame and an interested region, wherein the interested region head structure performs fine tuning refinement on the generated candidate frame, and completing candidate frame classification and target position detection;

The method comprises the steps of acquiring characteristics of each intestinal training sample segmentation characteristic diagram based on a multi-head self-attention module and dynamic convolution by adopting a query object detector, sharing mask heads of different branches by using three parallel dynamic mask heads, and mutually utilizing advanced semantic characteristics;

each pixel in the image set of the intestinal training sample is distributed to different object examples by adopting an end-to-end example segmentation method Queryinst to obtain an intestinal training sample segmentation feature map, which comprises the following steps:

inputting an intestinal training sample image with the size of 480 multiplied by 480 into a first stage, and acquiring a first intestinal training sample segmentation feature map, wherein the size of the first intestinal training sample segmentation feature map is 120 multiplied by 120, and the first intestinal training sample segmentation feature map comprises mucosa features in a bottom intestinal mucosa of the intestinal training sample image;

inputting the first intestinal training sample segmentation feature map into a second stage, and acquiring a second intestinal training sample segmentation feature map, wherein the size of the second intestinal training sample segmentation feature map is 60×60, and the second intestinal training sample segmentation feature map comprises highlighting of a light region in the intestinal tract of an intestinal training sample image;

inputting the second intestinal training sample segmentation feature map into a third stage and a fourth stage, and obtaining a third intestinal training sample segmentation feature map and a fourth intestinal training sample segmentation feature map, wherein the sizes of the third intestinal training sample segmentation feature map and the fourth intestinal training sample segmentation feature map are 30 multiplied by 30 and 15 multiplied by 15, and the segmentation region is formed by high-level semantic feature target foreign matters in the intestinal tract of the intestinal training sample image;

The neg uses an FPN feature pyramid to up-sample the segmentation feature map of the fourth intestinal training sample generated in the fourth stage, and then carries out 1 multiplied by 1 convolution refinement on the segmentation feature map of the third intestinal training sample in the third stage and then carries out residual connection to obtain a segmentation feature map f of the fifth intestinal training sample ₅ ；

Will f ₅ Up-sampling, performing 1×1 convolution refinement on the up-sampled and second intestinal training sample segmentation feature map in the second stage, and performing residual connection to integrate color and brightness features to obtain a sixth intestinal training sample segmentation feature map f ₆ ；

Will f ₆ Upsampling, performing 1×1 convolution refinement on the upsampled segmentation feature map of the first intestinal training sample in the first stage, and performing residual connection to integrate boundary information of foreign matters in the intestinal tract to obtain a segmentation feature map f of the seventh intestinal training sample ₇ ；

Extracting a plurality of candidate frames of the seventh intestinal training sample segmentation feature map, initializing N anchor frames, and obtaining the intersection ratio of each anchor frame and the candidate frames;

setting a background separation preset threshold, wherein if the intersection ratio is larger than the background separation preset threshold, the region of the candidate frame is a foreground, otherwise, the region is a background;

inputting candidate boxes into the region of interest and giving a query Q _t Will Q _t Input to Multi-head self-attention Module to get feature enhanced query Q _t ^* Will Q _t ^* And the candidate frame is used as a clue, fine tuning of the refinement feature is performed under dynamic convolution, and a segmentation area of the first foreign object is generated under the segmentation head as a clue guide of the next stage;

repeating the steps of inquiring and inputting the multi-head self-attention module to obtain a segmentation area of the second foreign object and a segmentation area of the third foreign object;

fusing the first foreign body segmentation area, the second foreign body segmentation area and the third foreign body segmentation area to obtain an intestinal foreign body instance segmentation result;

and performing iterative training on the intestinal semantic segmentation network according to the intestinal foreign object example segmentation result to obtain an intestinal foreign object semantic segmentation model.

Further, the semantic segmentation module specifically includes:

when the condition that a back blind flap of the intestinal tract to be detected starts to a mirror-withdrawing process outside the body is identified, starting an intestinal foreign matter semantic segmentation model to carry out semantic segmentation on a video of the intestinal tract to be detected, and acquiring a mirror-withdrawing rate of the mirror-withdrawing process;

judging which colon area the scope is positioned in through the fe_spi, dividing the intestinal tract to be detected into a ascending colon area, a transverse colon area and a descending colon area, and giving different confidence degrees a1, a2 and a3 to the cleanliness of the three areas;

the ascending colon area is close to the ileocecal valve, the foreign matter exists most in the area, and the area comprises solid manure and manure blocks, so that the influence on intestinal examination is large, and a1=1.2;

The transverse colon area is mainly composed of manure residues, partial small manure blocks exist, and interference is reduced through repeated flushing, so that a2=1.0;

the descending colon area comprises a descending colon and a sigmoid colon, the foreign matters are small-area fecal residues and residual fecal liquid, the influence is minimal, and a3=0.8;

dividing image frame boundaries of a ascending colon region, a transverse colon region and a descending colon region of the intestinal tract to be detected according to the scope-withdrawing rate, and acquiring an image frame set of the three regions;

performing semantic segmentation on the image frame set according to the intestinal foreign matter semantic segmentation model, and performing weighted calculation to obtain a foreign matter identification result of the intestinal segmented image to be detected;

the cleanliness analysis module specifically comprises:

calculating the foreign matter area and the duty ratio of each image frame in the image frame set according to the foreign matter identification result;

classifying the foreign matters in the intestinal tract to be detected according to the area and the proportion of the foreign matters;

wherein p represents the area ratio of the foreign matter of each image frame, and a represents the confidence of different areas of the intestinal tract;

classification as class I small foreign bodies when p x a < 0.15%;

when 0.15% < pa <1.5%, classifying as foreign matter in class II;

when 1.5% < pa <15%, classifying into class III large foreign matter;

when pa >15%, classifying into class IV extra-large foreign bodies;

When foreign substances are classified into class III and class IV in an image frame, the intestinal cleanliness of the image frame is poor;

on the basis of each foreign matter classification, grading the intestinal cleanliness of the whole image frame according to the sum of the proportion of all foreign matter examples of the whole image frame, and dividing the image frame into 4 grades from high to low by using a Boston intestinal grading scale, wherein the grades are respectively 3, 2, 1 and 0;

the score is proportional to the cleanliness of the image frame.

Those skilled in the art will appreciate that various modifications and improvements can be made to the disclosure. For example, the various devices or components described above may be implemented in hardware, or may be implemented in software, firmware, or a combination of some or all of the three.

A flowchart is used in this disclosure to describe the steps of a method according to an embodiment of the present disclosure. It should be understood that the steps that follow or before do not have to be performed in exact order. Rather, the various steps may be processed in reverse order or simultaneously. Also, other operations may be added to these processes.

Those of ordinary skill in the art will appreciate that all or a portion of the steps of the methods described above may be implemented by a computer program to instruct related hardware, and the program may be stored in a computer readable storage medium, such as a read only memory, a magnetic disk, or an optical disk. Alternatively, all or part of the steps of the above embodiments may be implemented using one or more integrated circuits. Accordingly, each module/unit in the above embodiment may be implemented in the form of hardware, or may be implemented in the form of a software functional module. The present disclosure is not limited to any specific form of combination of hardware and software.

Unless defined otherwise, all terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

The foregoing is illustrative of the present disclosure and is not to be construed as limiting thereof. Although a few exemplary embodiments of this disclosure have been described, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without materially departing from the novel teachings and advantages of this disclosure. Accordingly, all such modifications are intended to be included within the scope of this disclosure as defined in the claims. It is to be understood that the foregoing is illustrative of the present disclosure and is not to be construed as limited to the specific embodiments disclosed, and that modifications to the disclosed embodiments, as well as other embodiments, are intended to be included within the scope of the appended claims. The disclosure is defined by the claims and their equivalents.

In the description of the present specification, reference to the terms "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

While embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that: many changes, modifications, substitutions and variations may be made to the embodiments without departing from the spirit and principles of the invention, the scope of which is defined by the claims and their equivalents.

Claims (8)

1. A cleanliness detection method based on intestinal region segmentation, comprising:

acquiring a plurality of intestinal tract detection sample images, and performing pixel-level semantic annotation on the intestinal tract detection sample images to obtain an intestinal tract annotation sample image set;

performing effective frame judgment pretreatment on the intestinal tract labeling sample image set, and performing image standardization and data enhancement operation on the effective frame to obtain an intestinal tract training sample image set;

inputting the intestinal training sample image set into an intestinal semantic segmentation network for iterative training to obtain an intestinal foreign matter semantic segmentation model;

performing semantic segmentation on the video of the intestinal canal to be detected according to the intestinal canal foreign matter semantic segmentation model to obtain a foreign matter identification result of the intestinal canal segmentation image to be detected;

integrating the foreign matter identification result to obtain the foreign matter cleanliness of the intestinal tract to be detected;

Selecting the section of the intestinal tract to be detected with the lowest foreign matter cleanliness to output according to the foreign matter cleanliness and the detection duration of the intestinal tract to be detected, and forming a cleanliness detection report;

inputting the intestinal training sample image set into an intestinal semantic segmentation network for iterative training to obtain an intestinal foreign object semantic segmentation model, which specifically comprises the following steps:

using an end-to-end instance segmentation method Queryinst to distribute each pixel in the intestinal training sample image set to different object instances to obtain an intestinal training sample segmentation feature map;

adopting RegNetX-3.2GF as a backbone network backbone, inputting the intestinal training sample segmentation feature map into a candidate frame and a region of interest, wherein a region of interest head structure performs fine tuning refinement on the generated candidate frame, and completing candidate frame classification and target position detection;

the method comprises the steps of acquiring characteristics of each intestinal training sample segmentation characteristic diagram based on a multi-head self-attention module and dynamic convolution by adopting a query object detector, sharing mask heads of different branches by using three parallel dynamic mask heads, and mutually utilizing advanced semantic characteristics;

and distributing each pixel in the intestinal training sample image set to different object examples by adopting an end-to-end example segmentation method Queryinst to obtain an intestinal training sample segmentation feature map, wherein the method specifically comprises the following steps of:

Inputting an intestinal training sample image with the size of 480 multiplied by 480 into a first stage, and obtaining a first intestinal training sample segmentation feature map, wherein the size of the first intestinal training sample segmentation feature map is 120 multiplied by 120, and the first intestinal training sample segmentation feature map comprises mucosa features in a bottom intestinal mucosa of the intestinal training sample image;

inputting the first intestinal training sample segmentation feature map into a second stage, and obtaining a second intestinal training sample segmentation feature map, wherein the size of the second intestinal training sample segmentation feature map is 60 multiplied by 60, and the second intestinal training sample segmentation feature map comprises highlighting of a light region in an intestinal tract of an intestinal training sample image;

inputting the second intestinal training sample segmentation feature map into a third stage and a fourth stage, and obtaining a third intestinal training sample segmentation feature map and a fourth intestinal training sample segmentation feature map, wherein the sizes of the third intestinal training sample segmentation feature map and the fourth intestinal training sample segmentation feature map are 30 multiplied by 30 and 15 multiplied by 15, and the segmentation region is formed by high-grade semantic feature target foreign matters in the intestinal tract of the intestinal training sample image;

neg uses an FPN feature pyramid to up-sample the fourth intestinal training sample segmentation feature map generated in the fourth stage, and then performs 1×1 convolution refinement on the up-sample segmentation feature map and the third intestinal training sample segmentation feature map in the third stage, and then performs residual connection to obtain a fifth intestinal training sample segmentation feature map f ₅ ；

Will f ₅ Up-sampling, performing 1×1 convolution refinement on the up-sampled and second intestinal training sample segmentation feature map in the second stage, and performing residual connection to integrate color and brightness features to obtain a sixth intestinal training sample segmentation feature map f ₆ ；

Will f ₆ Upsampling, performing 1×1 convolution refinement on the upsampled segmentation feature map of the first intestinal training sample in the first stage, and performing residual connection to integrate boundary information of foreign matters in the intestinal tract to obtain a segmentation feature map f of the seventh intestinal training sample ₇ ；

Extracting a plurality of candidate frames of the seventh intestinal training sample segmentation feature map, and initializing N anchor frames to obtain the intersection ratio of each anchor frame and the candidate frames;

setting a background separation preset threshold, wherein if the intersection ratio is larger than the background separation preset threshold, the region of the candidate frame is a foreground, otherwise, the region is a background;

inputting the candidate box into the region of interest and giving a query Q _t Will Q _t Input to Multi-head self-attention Module to get feature enhanced query Q _t ^* Will Q _t ^* And the candidate frame is used as a clue, fine tuning of the refinement feature is performed under dynamic convolution, and a segmentation area of the first foreign object is generated under the segmentation head as a clue guide of the next stage;

repeating the steps of inquiring and inputting the multi-head self-attention module to obtain a segmentation area of the second foreign object and a segmentation area of the third foreign object;

Fusing the first foreign body segmentation area, the second foreign body segmentation area and the third foreign body segmentation area to obtain an intestinal foreign body instance segmentation result;

and carrying out iterative training on the intestinal semantic segmentation network according to the intestinal foreign object example segmentation result to obtain an intestinal foreign object semantic segmentation model.

2. The method for detecting cleanliness based on intestinal tract segmentation according to claim 1, wherein,

the method for obtaining a plurality of intestinal tract detection sample images, performing pixel-level semantic annotation on the intestinal tract detection sample images to obtain an intestinal tract annotation sample image set, specifically comprises the following steps:

obtaining a plurality of intestinal tract detection sample images from colonoscopy historical video;

dividing the intestinal tract detection sample image into a background image and a foreign object image, wherein the background image comprises a clean intestinal tract mucosa area, a blurring area, a flushing area, a reflecting area and a pixel area of a surgical instrument, and the foreign object image comprises an intestinal tract inner excrement block, excrement liquid, a bulk bubble and a yellow mucosa range which causes visual interference due to excrement;

and selecting a pixel point area belonging to the intestinal foreign matter through a polygonal labeling tool frame, and carrying out pixel-level labeling on the image to obtain an intestinal labeling sample image set.

3. The method for detecting cleanliness based on intestinal tract segmentation according to claim 1, wherein,

the method comprises the steps of carrying out effective frame judgment pretreatment on the intestinal tract labeling sample image set, carrying out image standardization and data enhancement operation on an effective frame to obtain an intestinal tract training sample image set, and specifically comprises the following steps:

preprocessing the intestinal labeling sample image set, taking the dyed, flushed, reflected and blurred images in the intestinal labeling sample image set as invalid frame images, and taking the images containing intestinal foreign matters and intestinal mucosa as valid frame images;

according to the data volume of the effective frame image in the intestinal labeling sample image set, according to the ineffective frame image: valid frame image = 1:10, selecting part of invalid frame images as negative samples, and removing redundant invalid frame images;

performing image standardization and data enhancement operation on all the effective frame images to obtain an intestinal training sample image set;

the image normalization realizes the centering treatment by removing the mean value, and in the data enhancement, the enhancement of space class and color transformation class is carried out;

the spatial class comprises operations of translating, turning and rotating the image, and the color transformation class comprises operations of noise, brightness, contrast and blurring.

4. The method for detecting cleanliness based on intestinal tract segmentation according to claim 1, wherein,

performing semantic segmentation on the video of the intestinal canal to be detected according to the intestinal canal foreign matter semantic segmentation model to obtain a foreign matter identification result of an intestinal canal segmentation image to be detected, wherein the method specifically comprises the following steps:

when the condition that a back blind flap of an intestinal tract to be detected starts to a mirror-withdrawing process outside the body is identified, starting the intestinal foreign matter semantic segmentation model to carry out semantic segmentation on a video of the intestinal tract to be detected, and acquiring a mirror-withdrawing rate of the mirror-withdrawing process;

dividing the intestinal tract to be detected into a ascending colon area, a transverse colon area and a descending colon area, and endowing the cleanliness of the three areas with different confidence degrees a1, a2 and a3;

the ascending colon area is close to the ileocecal valve, the foreign matter exists most in the area, and the area comprises solid manure and manure blocks, so that the influence on intestinal examination is large, and a1=1.2;

the transverse colon area is mainly composed of manure residues, partial small manure blocks exist, and interference is reduced through repeated flushing, so that a2=1.0;

the descending colon area comprises a descending colon and a sigmoid colon, the foreign matters are small-area fecal residues and residual fecal liquid, the influence is minimal, and a3=0.8;

dividing image frame boundaries of a ascending colon region, a transverse colon region and a descending colon region of the intestinal tract to be detected according to the mirror withdrawal rate, and acquiring an image frame set of the three regions;

And carrying out semantic segmentation on the image frame set according to the intestinal foreign matter semantic segmentation model, and obtaining a foreign matter identification result of the intestinal segmented image to be detected through weighted calculation.

5. The method for detecting cleanliness based on intestinal tract segmentation according to claim 1, wherein,

the step of integrating the foreign matter identification result to obtain the foreign matter cleanliness of the intestinal tract to be detected specifically comprises the following steps:

calculating the foreign object area and the duty ratio of each image frame in the image frame set according to the foreign object identification result;

classifying the foreign matters in the intestinal tract to be detected according to the foreign matter area and the proportion;

wherein p represents the area ratio of the foreign matter of each image frame, and a represents the confidence of different areas of the intestinal tract;

classification as class I small foreign bodies when p x a < 0.15%;

when 0.15% < pa <1.5%, classifying as foreign matter in class II;

when 1.5% < pa <15%, classifying into class III large foreign matter;

when pa >15%, classifying into class IV extra-large foreign bodies;

when foreign substances are classified into class III and class IV in an image frame, the intestinal cleanliness of the image frame is poor;

on the basis of each foreign matter classification, grading the intestinal cleanliness of the whole image frame according to the sum of the proportion of all foreign matter examples of the whole image frame, and dividing the image frame into 4 grades from high to low by using a Boston intestinal grading scale, wherein the grades are respectively 3, 2, 1 and 0;

The score is proportional to the cleanliness of the image frame.

6. The method for detecting cleanliness based on intestinal tract segmentation according to claim 1, wherein,

selecting the section of the intestinal tract to be detected with the lowest foreign matter cleanliness according to the foreign matter cleanliness and the detection duration of the intestinal tract to be detected, and outputting to form a cleanliness detection report, wherein the method specifically comprises the following steps:

the score output standard takes the score with lowest intestinal preparation quality score of a time period as the score of the time period at fixed time intervals; the cumulative score duty cycle is output after the intestinal examination is completed.

7. A cleanliness detection system based on intestinal region segmentation, comprising:

the labeling module is used for acquiring a plurality of intestinal tract detection sample images, and performing pixel-level semantic labeling on the intestinal tract detection sample images to obtain an intestinal tract labeling sample image set;

the preprocessing module is used for carrying out effective frame judgment preprocessing on the intestinal tract labeling sample image set, and carrying out image standardization and data enhancement operation on the effective frames to obtain an intestinal tract training sample image set;

the training module inputs the intestinal training sample image set into an intestinal semantic segmentation network for iterative training to obtain an intestinal foreign object semantic segmentation model;

The semantic segmentation module performs semantic segmentation on the video of the intestinal canal to be detected according to the intestinal canal foreign matter semantic segmentation model to obtain a foreign matter identification result of the intestinal canal segmentation image to be detected;

the cleanliness analysis module is used for integrating the foreign matter identification result to obtain the cleanliness of the foreign matters in the intestinal tract to be detected;

the output module is used for selecting the section of the intestinal tract to be detected with the lowest foreign matter cleanliness to output according to the foreign matter cleanliness and the detection duration of the intestinal tract to be detected, so as to form a cleanliness detection report;

the training module specifically comprises:

using an end-to-end instance segmentation method Queryinst to distribute each pixel in the intestinal training sample image set to different object instances to obtain an intestinal training sample segmentation feature map;

adopting RegNetX-3.2GF as a backbone network backbone, inputting the intestinal training sample segmentation feature map into a candidate frame and a region of interest, wherein a region of interest head structure performs fine tuning refinement on the generated candidate frame, and completing candidate frame classification and target position detection;

the method comprises the steps of acquiring characteristics of each intestinal training sample segmentation characteristic diagram based on a multi-head self-attention module and dynamic convolution by adopting a query object detector, sharing mask heads of different branches by using three parallel dynamic mask heads, and mutually utilizing advanced semantic characteristics;

And distributing each pixel in the intestinal training sample image set to different object examples by adopting an end-to-end example segmentation method Queryinst to obtain an intestinal training sample segmentation feature map, wherein the method specifically comprises the following steps of:

inputting an intestinal training sample image with the size of 480 multiplied by 480 into a first stage, and obtaining a first intestinal training sample segmentation feature map, wherein the size of the first intestinal training sample segmentation feature map is 120 multiplied by 120, and the first intestinal training sample segmentation feature map comprises mucosa features in a bottom intestinal mucosa of the intestinal training sample image;

inputting the first intestinal training sample segmentation feature map into a second stage, and obtaining a second intestinal training sample segmentation feature map, wherein the size of the second intestinal training sample segmentation feature map is 60 multiplied by 60, and the second intestinal training sample segmentation feature map comprises highlighting of a light region in an intestinal tract of an intestinal training sample image;

inputting the second intestinal training sample segmentation feature map into a third stage and a fourth stage, and obtaining a third intestinal training sample segmentation feature map and a fourth intestinal training sample segmentation feature map, wherein the sizes of the third intestinal training sample segmentation feature map and the fourth intestinal training sample segmentation feature map are 30 multiplied by 30 and 15 multiplied by 15, and the segmentation region is formed by high-grade semantic feature target foreign matters in the intestinal tract of the intestinal training sample image;

neg uses an FPN feature pyramid to up-sample the fourth intestinal training sample segmentation feature map generated in the fourth stage, and then performs 1×1 convolution refinement on the up-sample segmentation feature map and the third intestinal training sample segmentation feature map in the third stage, and then performs residual connection to obtain a fifth intestinal training sample segmentation feature map f ₅ ；

Will f ₅ Upsampling with a secondPerforming 1×1 convolution refinement on the segmentation feature map of the second intestinal training sample in the stage, and performing residual connection to integrate color and brightness features to obtain a sixth segmentation feature map f of the intestinal training sample ₆ ；

Will f ₆ Upsampling, performing 1×1 convolution refinement on the upsampled segmentation feature map of the first intestinal training sample in the first stage, and performing residual connection to integrate boundary information of foreign matters in the intestinal tract to obtain a segmentation feature map f of the seventh intestinal training sample ₇ ；

Extracting a plurality of candidate frames of the seventh intestinal training sample segmentation feature map, and initializing N anchor frames to obtain the intersection ratio of each anchor frame and the candidate frames;

setting a background separation preset threshold, wherein if the intersection ratio is larger than the background separation preset threshold, the region of the candidate frame is a foreground, otherwise, the region is a background;

inputting the candidate box into the region of interest and giving a query Q _t Will Q _t Input to Multi-head self-attention Module to get feature enhanced query Q _t ^* Will Q _t ^* And the candidate frame is used as a clue, fine tuning of the refinement feature is performed under dynamic convolution, and a segmentation area of the first foreign object is generated under the segmentation head as a clue guide of the next stage;

repeating the steps of inquiring and inputting the multi-head self-attention module to obtain a segmentation area of the second foreign object and a segmentation area of the third foreign object;

fusing the first foreign body segmentation area, the second foreign body segmentation area and the third foreign body segmentation area to obtain an intestinal foreign body instance segmentation result;

and carrying out iterative training on the intestinal semantic segmentation network according to the intestinal foreign object example segmentation result to obtain an intestinal foreign object semantic segmentation model.

8. A cleanliness detection system based on segmentation of intestinal regions according to claim 7,

the semantic segmentation module specifically comprises:

when the condition that a back blind flap of an intestinal tract to be detected starts to a mirror-withdrawing process outside the body is identified, starting the intestinal foreign matter semantic segmentation model to carry out semantic segmentation on a video of the intestinal tract to be detected, and acquiring a mirror-withdrawing rate of the mirror-withdrawing process;

dividing the intestinal tract to be detected into a ascending colon area, a transverse colon area and a descending colon area, and endowing the cleanliness of the three areas with different confidence degrees a1, a2 and a3;

The ascending colon area is close to the ileocecal valve, the foreign matter exists most in the area, and the area comprises solid manure and manure blocks, so that the influence on intestinal examination is large, and a1=1.2;

the transverse colon area is mainly composed of manure residues, partial small manure blocks exist, and interference is reduced through repeated flushing, so that a2=1.0;

the descending colon area comprises a descending colon and a sigmoid colon, the foreign matters are small-area fecal residues and residual fecal liquid, the influence is minimal, and a3=0.8;

dividing image frame boundaries of a ascending colon region, a transverse colon region and a descending colon region of the intestinal tract to be detected according to the mirror withdrawal rate, and acquiring an image frame set of the three regions;

performing semantic segmentation on the image frame set according to the intestinal foreign matter semantic segmentation model, and performing weighted calculation to obtain a foreign matter identification result of the intestinal segmented image to be detected;

the cleanliness analysis module specifically comprises:

calculating the foreign object area and the duty ratio of each image frame in the image frame set according to the foreign object identification result;

classifying the foreign matters in the intestinal tract to be detected according to the foreign matter area and the proportion;

wherein p represents the area ratio of the foreign matter of each image frame, and a represents the confidence of different areas of the intestinal tract;

Classification as class I small foreign bodies when p x a < 0.15%;

when 0.15% < pa <1.5%, classifying as foreign matter in class II;

when 1.5% < pa <15%, classifying into class III large foreign matter;

when pa >15%, classifying into class IV extra-large foreign bodies;

when foreign substances are classified into class III and class IV in an image frame, the intestinal cleanliness of the image frame is poor;

on the basis of each foreign matter classification, grading the intestinal cleanliness of the whole image frame according to the sum of the proportion of all foreign matter examples of the whole image frame, and dividing the image frame into 4 grades from high to low by using a Boston intestinal grading scale, wherein the grades are respectively 3, 2, 1 and 0;

the score is proportional to the cleanliness of the image frame.