CN114511558B

CN114511558B - Method and device for detecting cleanliness of intestinal tract

Info

Publication number: CN114511558B
Application number: CN202210402593.9A
Authority: CN
Inventors: 李�昊; 于天成; 刘奇为
Original assignee: Wuhan Endoangel Medical Technology Co Ltd
Current assignee: Wuhan Endoangel Medical Technology Co Ltd
Priority date: 2022-04-18
Filing date: 2022-04-18
Publication date: 2022-07-19
Anticipated expiration: 2042-04-18
Also published as: CN114511558A

Abstract

The application provides a method and a device for detecting the cleanliness of an intestinal tract, wherein the method for detecting the cleanliness of the intestinal tract comprises the following steps: acquiring a plurality of intestinal tract endoscope withdrawal images and performing real-time three-dimensional reconstruction to obtain an intestinal tract three-dimensional geometric model; acquiring an intestinal tract center line of the intestinal tract three-dimensional geometric model; respectively determining a plurality of center line point positions on the center line of the intestinal tract as target point positions, and calculating the change included angle of the intestinal tract center line of each target point position; determining a target point position with an included angle of the intestinal center line change smaller than a preset angle as a segmentation point position; determining a plurality of intestinal tract endoscope withdrawal images corresponding to the plurality of segmentation point positions as a plurality of first segmentation images; dividing the first retained images when the intestinal tract is retroactively into a plurality of retained image sets based on the first segmented images; determining the cleanliness of the intestinal tract based on the plurality of retained image sets. The method and the device can improve the accuracy of the detection method of the cleanliness of the intestinal tract.

Description

Method and device for detecting cleanliness of intestinal tract

Technical Field

The application mainly relates to the technical field of image processing, in particular to a method and a device for detecting the cleanliness of an intestinal tract.

Background

Colorectal cancer is the third leading cause of cancer-related death, and enteroscopy can effectively reduce the incidence of colorectal cancer by detecting and resecting intestinal adenomas. However, the efficiency of enteroscopy for lesion detection depends to a large extent on the quality of the observation of the intestinal mucosa, and if the intestinal preparation of the patient is insufficient, part of the mucosa may be obscured by dirt such as feces, liquid feces, food residues, etc. and thus cannot be observed. Thus, patients with inadequate bowel preparation should be instructed to re-purge their bowel for an enteroscopy review. However, due to the limitations of subjectivity of doctors, different clinical experiences and the like, the conventional clinical intestinal tract preparation assessment accuracy is not sufficient, and the condition of focus detection is difficult to reflect. Therefore, an auxiliary tool capable of accurately evaluating the cleanliness of the intestinal tract is urgently needed to be developed clinically, so that an endoscope physician is assisted to identify patients with insufficient intestinal tract preparation, and focus missed diagnosis is avoided.

That is, the method for detecting the cleanliness of the intestinal tract in the prior art is inaccurate.

Disclosure of Invention

The application provides a method and a device for detecting the cleanliness of an intestinal tract, and aims to solve the problem that the detection method of the cleanliness of the intestinal tract in the prior art is inaccurate.

In a first aspect, the present application provides a method for detecting cleanliness of an intestinal tract, including:

acquiring a plurality of intestinal tract endoscope-withdrawal images shot by an endoscope when the intestinal tract endoscope is withdrawn, and performing real-time three-dimensional reconstruction to obtain an intestinal tract three-dimensional geometric model;

acquiring an intestinal center line of the intestinal three-dimensional geometric model, wherein the intestinal center line is a line segment formed by centroids of intestinal section areas in a plurality of intestinal cross section images of the intestinal three-dimensional geometric model;

respectively determining a plurality of center line point positions on the center line of the intestinal tract as target point positions, and calculating the change included angle of the intestinal tract center line of each target point position;

determining a target point position with an included angle of the intestinal center line change smaller than a preset angle as a segmentation point position;

determining a plurality of intestinal tract endoscope withdrawing images corresponding to the plurality of segmentation point locations as a plurality of first segmentation images, wherein the intestinal tract endoscope withdrawing images corresponding to the segmentation point locations are intestinal tract endoscope withdrawing images added when the segmentation point locations are generated by updating the intestinal tract three-dimensional geometric model;

dividing the plurality of first retained images when the intestinal tract is retroactively into a plurality of retained image sets based on the plurality of first segmented images;

determining the cleanliness of the intestinal tract based on the plurality of retained image sets.

Optionally, the determining, as target points, a plurality of centerline points on the centerline of the intestinal tract, and calculating a variation included angle of the centerline of the intestinal tract at each target point includes:

respectively determining a plurality of central line point positions on the central line of the intestinal tract as target point positions;

performing straight line fitting on the target point location and a plurality of central line locations before the target point location to obtain a first intestinal line;

performing straight line fitting on the target point location and a plurality of central line locations behind the target point location to obtain a second intestinal line;

and determining an included angle between the first intestinal line and the second intestinal line as an intestinal center line change included angle of the target point positions to obtain the intestinal center line change included angle of each target point position.

Optionally, the dividing, based on the first segmentation images, the first retained images during the intestinal tract withdrawal into a plurality of retained image sets includes:

acquiring a plurality of intestinal cross section images corresponding to a plurality of segmentation point locations and a plurality of central line location before and after the segmentation point locations on the intestinal three-dimensional geometric model, wherein the intestinal cross section images comprise intestinal cross section areas, and the segmentation point locations and the plurality of central line location before and after the segmentation point locations are centroids of the plurality of intestinal cross section images respectively;

judging whether the area of the section area of the intestinal canal in the plurality of intestinal canal cross section images is increased along with the increase of the distance from each central line point to the target point;

if the area of the intestinal section area in the plurality of intestinal section images is not increased along with the increase of the distance from each central line point to the target point, determining the segmentation point to be an abnormal target point;

removing the first segmented images corresponding to the abnormal target point positions from the plurality of first segmented images to obtain a plurality of second segmented images;

and dividing the plurality of first retained images when the intestinal tract is withdrawn into a plurality of retained image sets based on the plurality of second segmented images.

Optionally, the dividing, based on the second segmentation images, the first retention images during the intestinal tract withdrawal into a plurality of retention image sets includes:

acquiring a plurality of first retained image images;

removing one of two adjacent first retained images with the similarity higher than the preset similarity in the first retained images to obtain a plurality of second retained images;

acquiring shooting time of each second sectional image and image retention time of a plurality of second image retention images;

respectively inputting the plurality of second segmented images into a target detection model for target detection to obtain target detection results of the plurality of second segmented images, wherein the target detection results comprise spleen koji types and liver koji types;

and determining a second retained image between the second segmented image with the target detection result of the spleen curve type and the second segmented image with the target detection result of the liver curve type as a transverse colon retained image set, determining a second retained image after the second segmented image with the target detection result of the spleen curve type as a left half colon retained image set, and determining a second retained image before the second segmented image with the target detection result of the liver curve type as a right half colon retained image set.

Optionally, the removing one of two adjacent first retained images with similarity higher than a preset similarity in the plurality of first retained images to obtain a plurality of second retained images includes:

respectively determining one of two adjacent first retained images in the plurality of first retained images as a target retained image;

acquiring the image height, the image width, the pixel value of each pixel point and the domain mean value of each pixel point of a target image retaining image;

determining a brightness parameter based on the image height and the image width of the target image retaining image and the pixel value of each pixel point;

determining the frequency of occurrence of each pixel point based on the pixel value and the domain mean value of each pixel point, wherein the pixel points with the same pixel value and the same domain mean value are the same type of pixel points;

calculating an image entropy parameter based on the frequency of occurrence of each pixel point, the image height and the image width;

weighting and summing the brightness parameter and the image entropy parameter based on the brightness weight and the image entropy weight to obtain a similarity comparison parameter of the target retained image and obtain a similarity comparison parameter of two adjacent first retained images;

and if the change rate of the similarity comparison parameter of the two adjacent first retained images is smaller than the preset change rate, determining that the similarity of the two adjacent first retained images is higher than the preset similarity, and removing one of the two adjacent first retained images to obtain a plurality of second retained images.

Optionally, the determining of the cleanliness of the intestinal tract based on the plurality of retained image sets comprises:

determining a transverse colon cleanliness, a left colon cleanliness and a right colon cleanliness based on the transverse colon retention image set, the left colon retention image set and the right colon retention image set respectively;

and weighting and summing the transverse colon cleanliness, the left half colon cleanliness and the right half colon cleanliness based on the transverse colon weight coefficient, the left half colon weight coefficient and the right half colon weight coefficient to obtain the intestinal cleanliness.

Optionally, the determining of the transverse colon cleanliness, the left colon cleanliness and the right colon cleanliness based on the transverse colon retention image set, the left colon retention image set and the right colon retention image set, respectively, comprises:

determining each transverse colon retaining image in the transverse colon retaining image set as a target transverse colon retaining image;

identifying a plurality of unclean regions on the target transverse colon atlas image;

removing the non-clean areas with the areas smaller than the preset area from the plurality of non-clean areas, and calculating the areas of the non-clean areas of the plurality of non-clean areas after removal;

calculating the area ratio of the uncleaned region area to the target transverse colon retaining image area;

determining the cleanliness class of a target transverse colon retaining image based on the area ratio to obtain the cleanliness class of each transverse colon retaining image in the transverse colon retaining image set, wherein if the area ratio is smaller than a preset ratio, the cleanliness class of the target transverse colon retaining image is determined to be clean, and if the area ratio is not smaller than the preset ratio, the cleanliness class of the target transverse colon retaining image is determined to be not clean;

determining the cleanliness of the transverse colon retained image set based on the image quantity ratio of the unclean category in the transverse colon retained image set.

In a second aspect, the present application provides a device for detecting cleanliness of an intestinal tract, comprising:

the three-dimensional reconstruction unit is used for acquiring a plurality of intestinal tract endoscope withdrawal images shot by an endoscope when the intestinal tract endoscope is withdrawn and performing real-time three-dimensional reconstruction to obtain an intestinal tract three-dimensional geometric model;

the acquisition unit is used for acquiring an intestinal center line of the intestinal three-dimensional geometric model, wherein the intestinal center line is a line segment formed by centroids of intestinal section areas in a plurality of intestinal cross section images of the intestinal three-dimensional geometric model;

the included angle calculation unit is used for respectively determining a plurality of center line points on the center line of the intestinal tract as target point points and calculating the intestinal tract center line change included angle of each target point;

the first determining unit is used for determining a target point position of which the variation included angle of the intestinal center line is smaller than a preset angle as a segmentation point position;

the second determining unit is used for determining a plurality of intestinal tract endoscope withdrawing images corresponding to the plurality of segmentation point locations into a plurality of first segmentation images, wherein the intestinal tract endoscope withdrawing images corresponding to the segmentation point locations are intestinal tract endoscope withdrawing images added when the segmentation point locations are generated by updating the intestinal tract three-dimensional geometric model;

the set dividing unit is used for dividing the first retained images during the intestinal tract withdrawal into a plurality of retained image sets based on the first segmented images;

and a third determination unit for determining the cleanliness of the intestinal tract based on the plurality of reserved image sets.

Optionally, the included angle calculating unit is configured to:

performing straight line fitting on the target point location and a plurality of center line point locations behind the target point location to obtain a second intestinal line;

and determining an included angle between the first intestinal line and the second intestinal line as an intestinal centerline change included angle of the target point positions to obtain the intestinal centerline change included angle of each target point position.

Optionally, the set dividing unit is configured to:

removing the first segmented images corresponding to the abnormal target points from the plurality of first segmented images to obtain a plurality of second segmented images;

and dividing the first retained image images when the intestinal tract is withdrawn into a plurality of retained image sets based on the second segmented images.

Optionally, the set dividing unit is configured to:

acquiring a plurality of first retained image images;

acquiring the shooting time of each second sectional image and the image retention time of a plurality of second image retention images;

Optionally, the set dividing unit is configured to:

acquiring the image height, the image width, the pixel value of each pixel point and the domain average value of each pixel point of a target image retaining image;

weighting and summing the brightness parameters and the image entropy parameters based on the brightness weight and the image entropy weight to obtain similarity comparison parameters of the target retained image and similarity comparison parameters of two adjacent first retained images;

Optionally, the third determining unit is configured to:

calculating the area ratio of the area of the uncleaned region to the area of the target transverse colon retaining image;

In a third aspect, the present application provides a computer device comprising:

one or more processors;

a memory; and

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the method of detecting gut cleanliness of any one of the first aspect.

In a fourth aspect, the present application provides a computer-readable storage medium storing a plurality of instructions, the instructions being suitable for being loaded by a processor to execute the steps of the method for detecting intestinal cleanliness according to any one of the first aspect.

The application provides a method for detecting the cleanliness of an intestinal tract, which comprises the following steps: acquiring a plurality of intestinal tract endoscope-withdrawal images shot by an endoscope when the intestinal tract endoscope is withdrawn, and performing real-time three-dimensional reconstruction to obtain an intestinal tract three-dimensional geometric model; acquiring an intestinal center line of the intestinal three-dimensional geometric model, wherein the intestinal center line is a line segment formed by centroids of intestinal section areas in a plurality of intestinal cross section images of the intestinal three-dimensional geometric model; respectively determining a plurality of center line point positions on the center line of the intestinal tract as target point positions, and calculating the change included angle of the intestinal tract center line of each target point position; determining a target point position with an included angle of the intestinal center line change smaller than a preset angle as a segmentation point position; determining a plurality of intestinal tract endoscope withdrawing images corresponding to the plurality of segmentation point locations as a plurality of first segmentation images, wherein the intestinal tract endoscope withdrawing images corresponding to the segmentation point locations are intestinal tract endoscope withdrawing images added when the segmentation point locations are generated by updating the intestinal tract three-dimensional geometric model; dividing the first retained images when the intestinal tract is retroactively into a plurality of retained image sets based on the first segmented images; determining the cleanliness of the intestinal tract based on the plurality of retained image sets. The application can improve the accuracy of the detection method of the intestinal cleanliness.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic view of a scene of a detection system for intestinal cleanliness provided in an embodiment of the present application;

fig. 2 is a schematic flow chart of an embodiment of a method for detecting the cleanliness of the intestinal tract in the embodiment of the present application;

FIG. 3 is a schematic diagram of a three-dimensional geometric model of the intestine according to an embodiment of the present application;

FIG. 4 is a schematic representation of the generation of an intestinal centerline in an embodiment of the present application;

fig. 5 is a schematic structural diagram of an embodiment of the intestinal cleanliness detection device provided in the embodiment of the present application;

fig. 6 is a schematic structural diagram of an embodiment of a computer device provided in the embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In the description of the present application, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like indicate orientations or positional relationships based on those shown in the drawings, and are used merely for convenience of description and for simplicity of description, and do not indicate or imply that the referenced device or element must have a particular orientation, be constructed in a particular orientation, and be operated, and thus should not be considered as limiting the present application. Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.

In this application, the word "exemplary" is used to mean "serving as an example, instance, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the application. In the following description, details are set forth for the purpose of explanation. It will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. In other instances, well-known structures and processes are not set forth in detail in order to avoid obscuring the description of the present application with unnecessary detail. Thus, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

The embodiments of the present application provide a method and a device for detecting cleanliness of an intestinal tract, which are described in detail below.

Referring to fig. 1, fig. 1 is a schematic view of a system for detecting intestinal cleanliness according to an embodiment of the present application, where the system for detecting intestinal cleanliness may include a computer device 100, and a device for detecting intestinal cleanliness is integrated in the computer device 100.

In this embodiment, the computer device 100 may be an independent server, or may be a server network or a server cluster composed of servers, for example, the computer device 100 described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a plurality of network server sets, or a cloud server composed of a plurality of servers. Among them, the Cloud server is constituted by a large number of computers or web servers based on Cloud Computing (Cloud Computing).

In the embodiment of the present application, the computer device 100 may be a general-purpose computer device or a special-purpose computer device. In a specific implementation, the computer device 100 may be a desktop computer, a portable computer, a web server, a Personal Digital Assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, an embedded device, and the like, and the embodiment does not limit the type of the computer device 100.

Those skilled in the art will appreciate that the application environment shown in fig. 1 is only one application scenario of the present application, and does not constitute a limitation to the application scenario of the present application, and that other application environments may further include more or less computer devices than those shown in fig. 1, for example, only 1 computer device is shown in fig. 1, and it is understood that the intestinal cleanliness detection system may further include one or more other computer devices capable of processing data, and is not limited herein.

In addition, as shown in fig. 1, the intestinal cleanliness detection system may further include a memory 200 for storing data.

It should be noted that the scene schematic diagram of the detection system of the intestinal cleanliness shown in fig. 1 is merely an example, and the detection system of the intestinal cleanliness and the scene described in the embodiment of the present application are for more clearly illustrating the technical solution of the embodiment of the present application, and do not constitute a limitation to the technical solution provided in the embodiment of the present application.

First, an embodiment of the present application provides a method for detecting cleanliness of an intestinal tract, where the method for detecting cleanliness of an intestinal tract includes: acquiring a plurality of intestinal tract endoscope-withdrawal images shot by an endoscope when the intestinal tract endoscope is withdrawn, and performing real-time three-dimensional reconstruction to obtain an intestinal tract three-dimensional geometric model; acquiring an intestinal center line of the intestinal three-dimensional geometric model, wherein the intestinal center line is a line segment formed by centroids of intestinal section areas in a plurality of intestinal cross section images of the intestinal three-dimensional geometric model; respectively determining a plurality of center line point positions on the center line of the intestinal tract as target point positions, and calculating the change included angle of the intestinal tract center line of each target point position; determining a target point position with an included angle of the intestinal center line change smaller than a preset angle as a segmentation point position; determining a plurality of intestinal tract endoscope withdrawing images corresponding to the plurality of segmentation point locations as a plurality of first segmentation images, wherein the intestinal tract endoscope withdrawing images corresponding to the segmentation point locations are intestinal tract endoscope withdrawing images added when the segmentation point locations are generated by updating the intestinal tract three-dimensional geometric model; dividing the first retained images when the intestinal tract is retroactively into a plurality of retained image sets based on the first segmented images; determining the cleanliness of the intestinal tract based on the plurality of retained image sets.

As shown in fig. 2, fig. 2 is a schematic flow chart of an embodiment of a method for detecting cleanliness of an intestinal tract in an embodiment of the present application, where the method for detecting cleanliness of an intestinal tract includes the following steps S201 to S207:

s201, obtaining a plurality of intestinal tract endoscope-withdrawal images shot by an endoscope when the intestinal tract endoscope is withdrawn, and carrying out real-time three-dimensional reconstruction to obtain a three-dimensional intestinal tract geometric model.

Referring to fig. 3, fig. 3 is a schematic view of a three-dimensional geometric model of an intestinal tract in an embodiment of the present application, specifically, an endoscope captures a plurality of images of the intestinal tract after endoscope withdrawal according to a preset frequency when the intestinal tract withdraws from the endoscope. And directly inputting the obtained intestinal tract endoscope-withdrawing image into three-dimensional modeling software for real-time three-dimensional reconstruction, updating the intestinal tract three-dimensional geometric model once every time one intestinal tract endoscope-withdrawing image is input, and updating the intestinal tract central line of the intestinal tract three-dimensional geometric model. The intestinal tract center line is a line segment formed by centroids of intestinal tract interface regions of a plurality of intestinal tract cross section images of the intestinal tract three-dimensional geometric model. When a new intestinal tract endoscope withdrawal image is input, a new central line point position is generated on the updated intestinal tract central line and corresponds to the input intestinal tract endoscope withdrawal image, and the generation time of the new central line point position is the shooting time of the input intestinal tract endoscope withdrawal image. Wherein, the endoscope passes through the right half colon, the transverse colon and the left half colon in sequence when the endoscope is withdrawn.

S202, obtaining an intestinal tract central line of the intestinal tract three-dimensional geometric model.

Referring to fig. 4, fig. 4 is a schematic diagram illustrating generation of an intestinal centerline in an embodiment of the present application, where the intestinal centerline is a line segment formed by centroids of intestinal cross-sectional areas in a plurality of intestinal cross-sectional images of a three-dimensional geometric model of an intestinal tract. Specifically, an intestinal cross-sectional image of the intestinal three-dimensional geometric model is made every 2 pixel distances along the endoscope retracting direction, the intestinal cross-sectional area in the intestinal cross-sectional image is set to be white, and the outside of the intestinal cross-sectional area is set to be black, as shown in fig. 3. And calculating the centroids of the sections of the intestinal tracts through the cv2.connected components within () function packet of the opencv, and sequentially connecting the centroids of the sections of the intestinal tracts to form the intestinal tract center line of the three-dimensional geometric model of the intestinal tract, as shown in fig. 4.

S203, respectively determining a plurality of centerline points on the centerline of the intestinal tract as target points, and calculating the variation included angle of the intestinal tract centerline of each target point.

In a specific embodiment, determining a plurality of centerline points on the centerline of the intestinal tract as target points, and calculating the variation included angle of the intestinal tract centerline of each target point, includes: and acquiring an included angle between two adjacent line segments of the target point location, and determining the included angle between the two adjacent line segments of the target point location as the intestinal tract center line change included angle of the target point location.

In another specific embodiment, determining a plurality of centerline points on the centerline of the intestinal tract as target points, and calculating the variation included angle of the intestinal tract centerline of each target point may include:

(1) and respectively determining a plurality of central line points on the central line of the intestinal tract as target points.

(2) And performing straight line fitting on the target point location and a plurality of central line locations before the target point location to obtain a first intestinal line.

Specifically, a least square method is used for performing straight line fitting on the target point location and 5 central line point locations before the target point location to obtain a first intestinal line. The number of the center line points before the target point may be set according to specific conditions.

(3) And performing straight line fitting on the target point location and a plurality of central line locations behind the target point location to obtain a second intestinal line.

Specifically, a least square method is used for performing straight line fitting on the target point location and 5 central line point locations behind the target point location to obtain a second intestinal line. The number of the plurality of center line points behind the target point can be set according to specific conditions.

(4) And determining an included angle between the first intestinal line and the second intestinal line as an intestinal center line change included angle of the target point positions to obtain the intestinal center line change included angle of each target point position.

And S204, determining the target point position with the changed included angle of the intestinal center line smaller than the preset angle as a segmentation point position.

In a specific embodiment, the predetermined angle is 100 degrees. Generally, the intestinal tract is divided into a right half colon, a transverse colon and a left half colon which are connected in sequence. There is a large corner between the right and transverse colon and a large corner between the left and transverse colon, which allows for differentiation of different bowel segments. If the central line of the intestinal tract does not change, the change included angle of the central line of the intestinal tract is 180 degrees, and if the central line of the intestinal tract changes greatly, the change included angle of the central line of the intestinal tract is smaller. And determining a target point position with the intestinal center line variation included angle smaller than a preset angle as a segmentation point position, wherein the segmentation point position is a segmentation node of different intestinal segments.

Further, if at least three target point locations with the intestinal centerline variation included angle smaller than the preset angle exist, whether a plurality of target point locations with the distance smaller than the preset distance exist is judged, and if the plurality of target point locations with the distance smaller than the preset distance exist, only the target point location with the intestinal centerline variation included angle smallest among the plurality of target point locations with the distance smaller than the preset distance is determined as the segmentation point location. Because the intestinal tract central line change included angles of a plurality of target point positions near the segmentation point position are possibly smaller than the preset angle, only the intestinal tract central line change included angle in the target point positions near the segmentation point position is reserved with the minimum change included angle, and the boundary can be accurately divided. Further, if the number of the target point positions with the changed included angles of the central line of the intestinal tract smaller than the preset angle is at least three, clustering the plurality of target point positions to obtain a plurality of point position clusters, and determining the target point position corresponding to the mass center of each point position cluster as a segmented point position.

For example, the centroid of each target point and the centroids of the front 5 intestinal section areas and the centroid of the rear 5 intestinal section areas are respectively fitted by a least square method to obtain two intersecting straight lines, and the included angle between the two straight lines is calculated. The 7 points form 7 included angles, and the centroid of the intestinal section area corresponding to the minimum value of the included angles is the segmented point position of the right colon and the transverse colon.

S205, determining a plurality of intestinal tract endoscope withdrawing images corresponding to the segmented point locations into a plurality of first segmented images, wherein the intestinal tract endoscope withdrawing images corresponding to the segmented point locations are intestinal tract endoscope withdrawing images added when the segmented point locations are generated for updating the intestinal tract three-dimensional geometric model.

As each intestinal tract endoscope withdrawal image is added, the three-dimensional reconstruction is carried out in real time to obtain an updated intestinal tract three-dimensional geometric model. When one intestinal tract endoscope withdrawing image is added to generate a segmentation point position, the newly generated segmentation point position corresponds to the added intestinal tract endoscope withdrawing image, and the generation time of the newly generated segmentation point position is the shooting time of the added intestinal tract endoscope withdrawing image.

S206, dividing the first retained images during intestinal tract endoscope withdrawal into a plurality of retained image sets based on the first segmented images.

In a specific embodiment, dividing the first retained images during the intestinal endoscopy withdrawal into a plurality of retained image sets based on the first segmented images may include:

(1) the method comprises the steps of obtaining a plurality of intestinal cross section images corresponding to a plurality of segmentation point positions and a plurality of central line position positions in front of and behind the segmentation point positions on an intestinal three-dimensional geometric model, wherein the intestinal cross section images comprise intestinal section areas, and the segmentation point positions and the plurality of central line position positions in front of and behind the segmentation point positions are centroids of the plurality of intestinal cross section images respectively.

(2) And judging whether the area of the intestinal tract section area in the plurality of intestinal tract cross section images is increased along with the increase of the distance from each central line point to the segmentation point.

(3) And if the areas of the intestinal section areas in the plurality of intestinal section images are not increased along with the increase of the distances from the corresponding centerline points to the segmentation points, determining the segmentation points as abnormal target points.

If the areas of the intestinal section areas in the multiple intestinal cross section images are increased along with the increase of the distances from the corresponding central line points to the segmentation point, the intestinal three-dimensional geometric model accords with the characteristic of the intestinal angle inflection point at the target point. If the area of the section area of the intestinal canal in the plurality of intestinal canal cross section images does not increase along with the increase of the distance between each central line point and the target point, the segmented point is determined to be an abnormal target point when the segmented point does not accord with the characteristic of the angle inflection point of the intestinal canal in the segmented point.

(4) And removing the first segmented images corresponding to the abnormal target points from the plurality of first segmented images to obtain a plurality of second segmented images.

And removing the first segmented image corresponding to the abnormal target point position from the plurality of first segmented images to obtain a plurality of second segmented images. The retained second segmented image not only meets the characteristic of an angle inflection point of the intestinal tract, but also meets the characteristic of an area inflection point, namely the characteristic that the section area is the minimum at the inflection point.

(5) And dividing the first retained image images when the intestinal tract is withdrawn into a plurality of retained image sets based on the second segmented images.

In a specific embodiment, the dividing the first retained image sets into a plurality of retained image sets when the intestinal tract is retroactively based on the second segmented images comprises:

(1) a plurality of first retained image images are acquired.

The first image-keeping images are obtained by manual screenshot of a doctor during enteroscopy.

In one specific embodiment, the process of the demagnifying operation is as follows:

the doctor starts to withdraw the endoscope from the ileocecal part, triggers the left pedal for the 1 st time, the endoscope enters the right colon, triggers the right pedal when the doctor needs to collect and leave the picture, acquires the collection instruction sent by the right pedal, and collects the first picture image.

After receiving the prompt of finding the segmented point location, the doctor triggers the left pedal 2 nd time to enter the transverse colon, triggers the right pedal when the doctor needs to collect the retained image, acquires the collection instruction sent by the right pedal, and collects the first retained image.

(2) And removing one of two adjacent first retained images with the similarity higher than the preset similarity in the plurality of first retained images to obtain a plurality of second retained images.

(3) And acquiring the shooting time of each second sectional image and the figure retention time of the plurality of second figure retention images.

(4) And respectively inputting the second segmented images into a target detection model for target detection to obtain target detection results of the second segmented images, wherein the target detection results comprise spleen koji types and liver koji types.

Besides a large angular inflection point and a large area inflection point, hepatic flexure exists between the right half colon and the transverse colon, and splenic flexure exists between the left half colon and the transverse colon, so that different intestinal sections can be distinguished. Wherein, the endoscope passes through the right half colon, the transverse colon and the left half colon in sequence when the endoscope is withdrawn.

(5) And determining a second retained image between the second segmented image with the target detection result of the spleen curve type and the second segmented image with the target detection result of the liver curve type as a transverse colon retained image set, determining a second retained image after the second segmented image with the target detection result of the spleen curve type as a left half colon retained image set, and determining a second retained image before the second segmented image with the target detection result of the liver curve type as a right half colon retained image set.

In addition to the conventional similarity calculation, in order to improve the accuracy, the similarity between two adjacent first retained images may further be calculated by removing one of the two adjacent first retained images with a similarity higher than a preset similarity in the plurality of first retained images to obtain a plurality of second retained images, where:

(1) and respectively determining one of two adjacent first retained images in the plurality of first retained images as a target retained image.

(2) And acquiring the image height, the image width, the pixel value of each pixel point and the domain average value of each pixel point of the target image-keeping image.

The neighborhood average method is to set the gray value of each pixel point as the average (or weighted average) of the gray values of all pixel points in a certain neighborhood window of the point, the window size adopted is generally 3 × 3, 5 × 5, 7 × 7 … …, and in the actual application, the flexible window size is adopted according to the trend of stripes, so that an ideal result can be obtained.

(3) And determining a brightness parameter based on the image height and the image width of the target image-keeping image and the pixel value of each pixel point.

Specifically, the luminance parameter L satisfies the following formula,

。

wherein, W and H respectively represent the image width and the image height of the image; m is_ijIs the pixel value of the pixel ij, i and j are the coordinate values of the pixel ij.

(4) And determining the frequency of the occurrence of each pixel point based on the pixel value and the domain mean value of each pixel point, wherein the pixel points with the same pixel value and the same domain mean value are the same type of pixel points.

The neighborhood gray level mean value of the image is used as the spatial characteristic quantity of gray level distribution, and the spatial characteristic quantity and the pixel gray level of the image form a characteristic binary group which is recorded as

Characteristic doublet

In the formula, i represents the gray value of a pixel, j represents the mean value of the gray levels of the neighborhood, and the frequency of the occurrence of each pixel point is

，

Is a characteristic binary group

The frequency of occurrence. The pixel points with the same pixel value and the same domain mean value are determined to be the same type of pixel points, and the number of the same type of pixel points is counted to be used as the frequency of each pixel point in the same type of pixel points.

(5) And calculating an image entropy parameter based on the frequency of occurrence of each pixel point, the image height and the image width.

Specifically, the image entropy parameter S satisfies the following formula,

。

n is the size of the image, and N =512 in the invention, and N is determined according to the height of the image and the width of the image.

(6) And weighting and summing the brightness parameters and the image entropy parameters based on the brightness weight and the image entropy weight to obtain similarity comparison parameters of the target retained image and similarity comparison parameters of two adjacent first retained images.

Specifically, the brightness weight is a, the image entropy weight is b, and the similarity comparison parameter

In the present invention

。

(7) And if the change rate of the similarity comparison parameter of the two adjacent first retained images is smaller than the preset change rate, determining that the similarity of the two adjacent first retained images is higher than the preset similarity, and removing one of the two adjacent first retained images to obtain a plurality of second retained images.

Wherein the change rate is 0.1, and the change rate of the similarity comparison parameter of two adjacent first retained images

And if the similarity of the two adjacent first retained images is higher than the preset similarity, the two adjacent first retained images are similar, and one of the two adjacent first retained images is removed.

And removing one of two adjacent first retained images in the plurality of first retained images to obtain a plurality of second retained images.

And S207, determining the cleanliness of the intestinal tract based on the multiple retained image sets.

Specifically, the plurality of chart image sets are respectively a transverse colon chart image set, a left half colon chart image set and a right half colon chart image set, and transverse colon cleanliness, left half colon cleanliness and right half colon cleanliness are respectively determined based on the transverse colon chart image set, the left half colon chart image set and the right half colon chart image set; and weighting and summing the transverse colon cleanliness, the left half colon cleanliness and the right half colon cleanliness based on the transverse colon weight coefficient, the left half colon weight coefficient and the right half colon weight coefficient to obtain the intestinal cleanliness. The incidence of polyps was found to be 11.91%, 16.47% and 41.23% for the right, transverse and left half-colon from large epidemiological databases, respectively, and thus the right, transverse and left half-colon weight coefficients were determined to be 11.91%, 16.47% and 41.23%, respectively.

Further, determining the cleanliness of the transverse colon, the cleanliness of the left half colon and the cleanliness of the right half colon based on the transverse colon retention image set, the left colon retention image set and the right colon retention image set respectively comprises:

(1) and determining each transverse colon retaining image in the transverse colon retaining image set as a target transverse colon retaining image.

(2) A plurality of unclean regions on the target transverse colon atlas image are identified.

Specifically, an unclean region segmentation model is trained, and a Unet + + network model is preferentially selected. And loading the trained unclean region segmentation model, and segmenting a plurality of mask images of unclean regions from the target transverse colon retained image.

(3) And removing the non-clean areas with the areas smaller than the preset area from the plurality of non-clean areas, and calculating the areas of the non-clean areas of the plurality of removed non-clean areas.

The area of each unclean region is calculated separately. Obtaining a target transverse colon retention map image j₀Area S of each unclean region_i0E.g. a predetermined area of 4, when

The noise is filtered, and then the sum of the areas of the plurality of uncleaned areas after the noise is removed is calculated and used as the area of the uncleaned area

In which K is_j0Image j representing a target transverse colon rendering₀And the number of the residual unclean areas after the noise is eliminated.

(4) The ratio of the area of the uncleaned region to the area of the target transverse colon retaining image is calculated.

Then calculating a target transverse colon retention image j₀Area ratio of unclean region to target transverse colon mapping image

Wherein W is_j0,H_j0Respectively retaining image j for target transverse colon₀Width and height of (a).

(5) And determining the cleanliness class of the target transverse colon retaining image based on the area ratio to obtain the cleanliness class of each transverse colon retaining image in the transverse colon retaining image set, wherein if the area ratio is smaller than a preset ratio, the cleanliness class of the target transverse colon retaining image is determined to be clean, and if the area ratio is not smaller than the preset ratio, the cleanliness class of the target transverse colon retaining image is determined to be unclean.

The preset ratio can be 10%, and the like, and can be set according to specific conditions. And if the area ratio is less than the preset ratio, determining the cleanliness class of the target transverse colon retention map image as clean, and if the area ratio is not less than the preset ratio, determining the cleanliness class of the target transverse colon retention map image as unclean.

(6) Determining the cleanliness of the transverse colon retained image set based on the image quantity ratio of the unclean category in the transverse colon retained image set.

Further, if the image quantity proportion of the unclean category in the transverse colon retained image set is less than 15% of the preset quantity proportion, the transverse colon cleanliness of the transverse colon retained image set is determined to be clean. And if the image number proportion of the unclean category in the transverse colon retained image set is not less than 15% of the preset number proportion, determining the transverse colon cleanliness of the transverse colon retained image set as unclean. Of course, transverse colonic cleanliness may also be scored, for example, transverse colonic cleanliness ofScore ₂。

In the same manner, the cleanliness of the left half colon was calculatedScore ₃And right-half colonic cleanlinessScore ₁。

Further, based on the transverse colon weight coefficient, the left half colon weight coefficient and the right half colon weight coefficient, the transverse colon cleanliness, the left half colon cleanliness, the right half colon cleanliness and the transverse colon weight coefficient are subjected to weighted summation to obtain the intestinal cleanliness. For example, when the transverse colon weight coefficient, the left half colon weight coefficient and the right half colon weight coefficient are all 1, the intestinal cleanliness is calculatedScoreI.e. overall intestinal cleanliness score:Score= Score ₁₊ Score ₂₊ Score ₃；

judging whether the cleanliness of the whole intestinal tract is qualified or not:

。

further, in the above-mentioned case,

step S1.1: the device is provided with two pedals, wherein the pedals are divided into a left pedal and a right pedal, the left pedal is connected into a USB induction left pedal in the computer equipment and is used as a intestine section trigger. Each time the doctor steps on the left pedal, the doctor transmits a trigger signal to the receiving module a of the computer device, the response module of which then takes the appointed follow-up action. The specific using process is as follows:

triggering the left pedal for the 1 st time, marking the pedal as the starting point of the mirror-down process, namely the starting point of the right colon, prompting the start of the right colon by an app interface of the computer device, and automatically creating a 'right colon' image-retaining folder by the image-retaining system. And placing a second retained image before the second segmented image with the target detection result of the hepatic curve type into a right colon' retained image folder.

Triggering the left pedal at the 2 nd time, marking as the starting point of the transverse colon, prompting the transverse colon to start by an app interface of the computer device, automatically creating a transverse colon image retaining folder by the image retaining system, and putting the transverse colon image retaining image set into the transverse colon image retaining folder. And gives a score for the right colon in the app interface of the computer device.

Triggering a left pedal for 3 times, marking the left pedal as a left half colon starting point, prompting the start of the left half colon by an app interface of the computer device, automatically creating a left half colon image retaining folder by the image retaining system, putting the left half colon image retaining image set into a transverse colon image retaining folder, and giving a transverse colon score on the app interface of the computer device;

triggering a left pedal for the 4 th time, marking the case to be ended, and giving a score of a right colon on an app interface of the computer device;

triggering the left pedal in the 5 th time, giving a score of the whole intestinal cleanliness on an app interface of the computer equipment, and giving a judgment result of whether the intestinal cleanliness is qualified.

Step S1.2: and a USB induction right pedal is connected into the computer equipment and is used as a drawing trigger. Each time the doctor steps on the right pedal, the doctor transmits a trigger signal to the receiving module B of the computer device, the response module of which then takes the appointed follow-up action. The receiving module B receives the signal of the intestine section trigger, automatically identifies the currently examined intestine section, collects a picture of a reserved image once a doctor steps on the right pedal once, and reserves the picture to a file of the reserved image corresponding to the current intestine section.

Step S2: training a scoring system model:

step S2.1: based on a training set of intestinal white light images and staining images in a typical training set, a white light image and staining image 2 classification recognition model DCNN1 is trained, and Resnet50 is preferably selected and labeled as white light and staining.

Step S2.2: based on the white light images of the intestinal tract in a typical training set, a clean image and unclean image 2 classification recognition model DCNN2 is trained, and Resnet50 is preferably selected and labeled as clean and unclean.

Step S2.3: based on the unclean white light images of the intestinal tract in the step S2.2, training an image 2 classification recognition model DCNN3 with scores of 0 and 1, preferentially selecting Resnet50 with labels of 0 and 1.

Step S2.4: based on the white-light images of the intestinal tract cleaned in step S2.2, the 2-class recognition model DCNN4 with scores of 2-point images and 3-point images is trained, and Resnet50 is preferentially selected and labeled with 2-point and 3-point labels.

Step S2.5: and (3) training an unclean region segmentation model based on the unclean intestinal white light image in the step (S2.2), and preferentially selecting a Unet + + network model.

Step S3: the operation process of mirror removal:

s3.1, the doctor withdraws the endoscope from the ileocecal part, triggers the left pedal for the 1 st time to enter the right colon, triggers the right pedal when the doctor needs to collect the retained images, and retains the retained images to a 'right colon' retained image folder;

step S3.2: when a doctor starts to withdraw the endoscope, the segmented point of the intestinal segment is found in the following way:

1) and (3) creating a three-dimensional intestinal tract geometric model by using a picture captured by the endoscope in the endoscope withdrawal process through three-dimensional modeling software VKT.

2) And (3) making intestinal tract cross section images of the intestinal tract three-dimensional geometric model at intervals of 2 pixel distances along the endoscope withdrawing direction, setting the intestinal tract cross section area in the intestinal tract cross section images to be white, and setting the outer part of the intestinal tract cross section area to be black, as shown in fig. 3.

3) On the basis of the connected domain, the centroids of all the intestinal section areas are calculated through the cv2.connected Components WithStats () function packet carried by opencv, and the centroids of all the intestinal section areas are sequentially connected to form the intestinal centerline of the intestinal three-dimensional geometric model, as shown in FIG. 4.

4) And calculating Euclidean distances from points on the boundary of each intestinal section area to the centroid of the intestinal section area on the basis of the connected domain, and then calculating the mean value of all the Euclidean distances to be the equivalent radius of the intestinal section area.

5) In the process of endoscope withdrawal, the equivalent radius of the section of the three-dimensional geometric model is reduced firstly, and then when the equivalent radius of continuous three sections is increased gradually, 7 centroids of the intestinal section area corresponding to the minimum value of the radius and the centroids of the three intestinal section areas in front of and behind the intestinal section area are found.

6) 5) connecting each point with the centroids of the front 5 intestinal section areas and the centroids of the back 5 intestinal section areas respectively by least square fittingCombining to obtain two intersecting straight lines, calculating the included angle of the two straight lines to obtain 7 included angles in total, wherein the centroid of the intestinal section region corresponding to the minimum value of the included angles is the segmentation point location of the right colon and the transverse colon, and recording the generation time of the segmentation point locationt ₁。

7) After receiving the prompt of finding the segmented point location, the doctor triggers the left pedal 2 times to enter the transverse colon, triggers the right pedal when the doctor needs to collect the retained image, retains the retained image in the transverse colon retaining folder, and retains the image in the right colon retaining folder later than the transverse colon retaining folder

The time-generated image is moved to the "transverse colon" mapping folder.

Step S3.3: in the process of transverse colon examination, a scoring system is started to calculate the cleanliness of the right colon, and the method is specifically realized by the following steps:

1) starting the DCNN1 model trained in the step S2.1 to filter out the dyeing images in the "right-half colon" image-keeping folder processed in the step S3.2, step 7), and leaving the dyeing images in the "right-half colon" image-keeping folderN _wOpening a white light image, and filtering out the white light image with higher similarity in the following way:

2) loading white light images through the DCNN2 model trained in the step S2.2, and recognizing the number N of unclean images in the right colon_1WBNumber N of clean images_1WQThe ratio of the right unclean image to the right unclean image

。

3) If it is

And the right colon is unclean, and N is loaded through the DCNN3 model trained in step S2.3_1WBUnclean white light image, identify image N which scores 0_1WB01-point image N_1WB1The right colon cleanliness score is then assessed as follows:

loading the Unet + + model trained in step S2.5, and dividingCutting out 0 part of mask image of unclean region in the original image, and obtaining j (th) on the basis of connected domain₀Area S of each connected domain in Zhan 0 min image_i0When is coming into contact with

The noise is filtered out, and the sum of the areas of the residual connected domains is calculated

，

In which K is_j0Denotes the jth₀And (5) removing noise from the 0-point image to obtain the number of residual unclean areas. Then calculate the j₀Area ratio of unclean region of 0 min image

Wherein W is_j0，H_j0Are respectively the jth₀The width and height of the 0-part image are spread. Finally, calculating the area ratio of all the 0-min image average unclean areas

；

Secondly, loading the Unet + + model trained in the step S2.5, segmenting a mask map of an uncleaned area in the enteron retroscopic image with 1 score, and obtaining the jth mask map on the basis of a connected domain₁Area S of each connected domain in 1-inch-expanded image_i1When is coming into contact with

Wherein K_j1Denotes the j (th)₁And (4) removing noise from the 1-inch image to obtain the number of residual unclean areas. Then calculate the j₁Area ratio of unclean region of 1-division-by-one image

Wherein W is_j1，H_j1Are respectively the jth₁1 part of the image width and height is obtained. Finally, calculating the area ratio of the average unclean area of all the 1-minute images

；

If it is

The cleanliness of the right colonScore ₁If not =0

The cleanliness of the right colon

。

If it is

The right colon is cleaned, at which time N is loaded via the DCNN4 model trained in step S2.4_1WQOpening a clean white light image, and identifying an image N with 2 points_1WQ2Image N with score of 3_1WQ3And (5) opening the paper. The right colon cleanliness score is then assessed as follows:

loading the Unet + + model trained in the step S2.5, segmenting mask images of uncleaned areas in all clean white light images, and obtaining the first image on the basis of connected domainsj _QArea of each connected region in sheet imageS _iQWhen is coming into contact with

，

WhereinK _jQIs shown asj _QAnd (5) removing noise from the 2-piece images, and then remaining the number of the regions which are not cleaned. Then calculate the firstj _QArea ratio of uncleaned area of image

Wherein W is_jQ，H_jQAre respectively the firstj _QWidth and height of the sheet image. Finally, calculating the area ratio of the average unclean area of all the images

Average number of unclean areas for all images

。

② if

The cleanliness of the right colonScore ₁=2, if

The cleanliness of the right colonScore ₁=3。

The score for the right colon is given at the app interface of the computer device.

Step S3.4: after entering the transverse colon, with the continuous process of endoscope withdrawal, the method of step S3.2 is adopted to find the second segmented point of the intestine section and record the section generating time of the second segmented pointt ₂The doctor triggers the left pedal 3 times to enter the left half colon, triggers the right pedal when needing to collect and reserve images, reserves the images to the left half colon image reserving folder, and reserves the transverse colon image later than the transverse colon image reserving foldert ₂The generated image is moved to the "left half colon" mapping folder. In the process of examining the left half colon, the cleanliness of the transverse colon is calculated by adopting the method for the cleanliness of the right half colon in the step S3.3Score ₂And the score of the transverse colon is given at the app interface of the computer device.

Step S3.5: triggering the left pedal for 4 th time, finishing enteroscopy, and calculating the cleanliness of the left half colon by adopting the method for the cleanliness of the right half colon in the step S3.3Score ₃And a score is given for the left half colon at the app interface of the computer device.

Step S3.6: and triggering the left pedal at the 5 th time, giving a score of the whole intestinal cleanliness on an app interface of the computer equipment, and giving a judgment result of whether the intestinal cleanliness is qualified. The method comprises the following concrete steps:

calculating the overall intestinal cleanliness score:Score=Score ₁ +Score ₂ +Score ₃；

。

in order to better implement the method for detecting cleanliness of an intestinal tract in the embodiment of the present application, on the basis of the method for detecting cleanliness of an intestinal tract, an apparatus for detecting cleanliness of an intestinal tract is further provided in the embodiment of the present application, as shown in fig. 5, the apparatus 300 for detecting cleanliness of an intestinal tract includes:

the three-dimensional reconstruction unit 301 is used for obtaining a plurality of intestinal tract endoscope-withdrawal images shot by an endoscope when the intestinal tract endoscope is withdrawn and performing real-time three-dimensional reconstruction to obtain an intestinal tract three-dimensional geometric model;

the acquiring unit 302 is configured to acquire an intestinal centerline of the intestinal three-dimensional geometric model, where the intestinal centerline is a line segment formed by centroids of intestinal cross-sectional areas in a plurality of intestinal cross-sectional images of the intestinal three-dimensional geometric model;

an included angle calculation unit 303, configured to determine a plurality of center line point locations on the center line of the intestinal tract as target point locations, respectively, and calculate a variation included angle of the intestinal tract center line of each target point location;

a first determining unit 304, configured to determine a target point where an included angle of the change of the intestinal centerline is smaller than a preset angle as a segmentation point;

a second determining unit 305, configured to determine multiple intestinal tract endoscope-withdrawing images corresponding to multiple segmented point locations as multiple first segmented images, where the intestinal tract endoscope-withdrawing images corresponding to the segmented point locations are intestinal tract endoscope-withdrawing images added when the three-dimensional geometric model of the intestinal tract is updated and generated;

a set dividing unit 306, configured to divide the first retained images during the intestinal tract withdrawal into a plurality of retained image sets based on the first segmented images;

a third determination unit 307 for determining the cleanliness of the intestinal tract based on the plurality of keep-alive image sets.

Optionally, the included angle calculating unit 303 is configured to:

respectively determining a plurality of central line points on the central line of the intestinal tract as target points;

performing straight line fitting on the target point location and a plurality of center line point locations before the target point location to obtain a first intestinal line;

Optionally, the set dividing unit 306 is configured to:

acquiring a plurality of intestinal cross section images corresponding to a plurality of segmentation point positions and a plurality of central line positions before and after the segmentation point positions on an intestinal three-dimensional geometric model, wherein the intestinal cross section images comprise intestinal section areas, and the segmentation point positions and the plurality of central line positions before and after the segmentation point positions are centroids of the plurality of intestinal cross section images respectively;

judging whether the area of the intestinal tract section area in the plurality of intestinal tract cross section images is increased along with the increase of the distance from each central line point to the target point;

if the area of the intestinal section area in the plurality of intestinal section images is not increased along with the increase of the distance between each central line point and the target point, determining the segmentation point to be an abnormal target point;

Optionally, the set dividing unit 306 is configured to:

acquiring a plurality of first retained image images;

and determining a second retained image between the second segmented image with the target detection result of the splenic flexure type and the second segmented image with the target detection result of the hepatic flexure type as a transverse colon retained image set, determining a second retained image after the second segmented image with the target detection result of the splenic flexure type as a left half colon retained image set, and determining a second retained image before the second segmented image with the target detection result of the hepatic flexure type as a right half colon retained image set.

Optionally, the set dividing unit 306 is configured to:

Optionally, a third determining unit 307, configured to:

Optionally, the third determining unit 307 is configured to:

determining the cleanliness class of the target transverse colon retaining image based on the area ratio to obtain the cleanliness class of each transverse colon retaining image in the transverse colon retaining image set, wherein if the area ratio is smaller than a preset ratio, the cleanliness class of the target transverse colon retaining image is determined to be clean, and if the area ratio is not smaller than the preset ratio, the cleanliness class of the target transverse colon retaining image is determined to be unclean;

and determining the transverse colon cleanliness of the transverse colon chart image set based on the image quantity ratio of the unclean category in the transverse colon chart image set.

The embodiment of the present application further provides a computer device, which integrates the device for detecting cleanliness of any kind of intestinal tract provided by the embodiment of the present application, and the computer device includes:

one or more processors;

a memory; and

one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor for the steps of the method for detecting intestinal cleanliness in any one of the above embodiments of the method for detecting intestinal cleanliness.

As shown in fig. 6, it shows a schematic structural diagram of a computer device according to an embodiment of the present application, and specifically:

the computer device may include components such as a processor 401 of one or more processing cores, memory 402 of one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that the computer device configurations illustrated in the figures are not meant to be limiting of computer devices and may include more or fewer components than those illustrated, or some components may be combined, or a different arrangement of components. Wherein:

the processor 401 is a control center of the computer device, connects various parts of the entire computer device using various interfaces and lines, and performs various functions of the computer device and processes data by running or executing software programs and/or modules stored in the memory 402 and calling data stored in the memory 402, thereby monitoring the computer device as a whole. Optionally, processor 401 may include one or more processing cores; the Processor 401 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, preferably the processor 401 may integrate an application processor, which handles primarily the operating system, user interfaces, application programs, etc., and a modem processor, which handles primarily wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 401.

The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by operating the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to use of the computer device, and the like. Further, the memory 402 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 access to the memory 402.

The computer device further comprises a power supply 403 for supplying power to the various components, and preferably, the power supply 403 is logically connected to the processor 401 via a power management system, so that functions of managing charging, discharging, and power consumption are implemented via the power management system. The power supply 403 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.

The computer device may also include an input unit 404, and the input unit 404 may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

Although not shown, the computer device may further include a display unit and the like, which are not described in detail herein. Specifically, in this embodiment, the processor 401 in the computer device loads the executable file corresponding to the process of one or more application programs into the memory 402 according to the following instructions, and the processor 401 runs the application programs stored in the memory 402, thereby implementing various functions as follows:

acquiring a plurality of intestinal tract endoscope-withdrawal images shot by an endoscope when the intestinal tract endoscope is withdrawn, and performing real-time three-dimensional reconstruction to obtain an intestinal tract three-dimensional geometric model; acquiring an intestinal center line of the intestinal three-dimensional geometric model, wherein the intestinal center line is a line segment formed by centroids of intestinal section areas in a plurality of intestinal cross section images of the intestinal three-dimensional geometric model; respectively determining a plurality of center line point positions on the center line of the intestinal tract as target point positions, and calculating the change included angle of the intestinal tract center line of each target point position; determining a target point position with an included angle of the intestinal center line change smaller than a preset angle as a segmentation point position; determining a plurality of intestinal tract endoscope withdrawing images corresponding to the plurality of segmentation point locations as a plurality of first segmentation images, wherein the intestinal tract endoscope withdrawing images corresponding to the segmentation point locations are intestinal tract endoscope withdrawing images added when the segmentation point locations are generated by updating the intestinal tract three-dimensional geometric model; dividing the plurality of first retained images when the intestinal tract is retroactively into a plurality of retained image sets based on the plurality of first segmented images; determining the cleanliness of the intestinal tract based on the plurality of retained image sets.

It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.

To this end, an embodiment of the present application provides a computer-readable storage medium, which may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like. The computer program is loaded by the processor to execute the steps of any one of the methods for detecting the cleanliness of the intestinal tract provided by the embodiments of the application. For example, the computer program may be loaded by a processor to perform the steps of:

acquiring a plurality of intestinal tract endoscope-withdrawal images shot by an endoscope when the intestinal tract endoscope is withdrawn, and performing real-time three-dimensional reconstruction to obtain an intestinal tract three-dimensional geometric model; acquiring an intestinal center line of the intestinal three-dimensional geometric model, wherein the intestinal center line is a line segment formed by centroids of intestinal section areas in a plurality of intestinal cross section images of the intestinal three-dimensional geometric model; respectively determining a plurality of central line point positions on the central line of the intestinal tract as target point positions, and calculating the variation included angle of the central line of the intestinal tract of each target point position; determining a target point position with an included angle of the intestinal center line change smaller than a preset angle as a segmentation point position; determining a plurality of intestinal tract endoscope withdrawing images corresponding to the plurality of segmentation point locations as a plurality of first segmentation images, wherein the intestinal tract endoscope withdrawing images corresponding to the segmentation point locations are intestinal tract endoscope withdrawing images added when the segmentation point locations are generated by updating the intestinal tract three-dimensional geometric model; dividing the first retained images when the intestinal tract is retroactively into a plurality of retained image sets based on the first segmented images; determining the cleanliness of the intestinal tract based on the plurality of retained image sets.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and parts that are not described in detail in a certain embodiment may refer to the above detailed descriptions of other embodiments, and are not described herein again.

In a specific implementation, each unit or structure may be implemented as an independent entity, or may be combined arbitrarily to be implemented as one or several entities, and the specific implementation of each unit or structure may refer to the foregoing method embodiment, which is not described herein again.

The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.

The method and the device for detecting the cleanliness of the intestinal tract provided by the embodiment of the application are described in detail, a specific example is applied in the description to explain the principle and the implementation mode of the application, and the description of the embodiment is only used for helping to understand the method and the core idea of the application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A method for detecting the cleanliness of an intestinal tract, comprising:

determining a target point position with the changed included angle of the intestinal tract central line smaller than a preset angle as a segmentation point position;

dividing the first retained images when the intestinal tract is retroactively into a plurality of retained image sets based on the first segmented images; acquiring a plurality of intestinal cross section images corresponding to a plurality of segmentation point locations and a plurality of central line locations before and after the segmentation point locations on the intestinal three-dimensional geometric model, wherein the intestinal cross section images comprise intestinal cross section areas, and the segmentation point locations and the plurality of central line locations before and after the segmentation point locations are centroids of the plurality of intestinal cross section images respectively; judging whether the area of the intestinal tract section area in the plurality of intestinal tract cross section images is increased along with the increase of the distance from each central line point to the target point; if the area of the section area of the intestinal canal in the plurality of intestinal canal cross section images is not increased along with the increase of the distance from each central line point to the target point, determining the segmentation point to be an abnormal target point; removing the first segmented images corresponding to the abnormal target points from the plurality of first segmented images to obtain a plurality of second segmented images; dividing the first retained image images when the intestinal tract is withdrawn into a plurality of retained image sets based on the second segmented images;

2. The method for detecting the cleanliness of the intestinal tract according to claim 1, wherein the step of determining a plurality of centerline points on the central line of the intestinal tract as target points respectively and calculating the variation included angle of the central line of the intestinal tract of each target point comprises the steps of:

3. The method for detecting cleanliness of intestinal tract according to claim 1, wherein the dividing the plurality of first retained images during the withdrawal of the intestinal tract into a plurality of retained image sets based on the plurality of second segmented images comprises:

acquiring a plurality of first retained image images;

4. The method for detecting the cleanliness of the intestinal tract according to claim 3, wherein the step of removing one of two adjacent first retained images with the similarity higher than a preset similarity in the plurality of first retained images to obtain a plurality of second retained images comprises the steps of:

5. The method for detecting intestinal cleanliness according to claim 4, wherein the determining intestinal cleanliness based on a plurality of the retained image sets comprises:

6. The method for detecting intestinal cleanliness according to claim 5, wherein the determining of the transverse colon cleanliness, the left colon cleanliness and the right colon cleanliness based on the transverse colon retention map image set, the left colon retention map image set and the right colon retention map image set respectively comprises:

identifying a plurality of unclean regions on the target transverse colon retention map image;

removing the unclean regions with the areas smaller than the preset area from the plurality of unclean regions, and calculating the areas of the unclean regions after removal;

7. A device for detecting cleanliness of an intestinal tract, comprising:

the first determining unit is used for determining a target point position with an intestinal center line change included angle smaller than a preset angle as a segmentation point position;

the set dividing unit is used for dividing the first retained image images when the intestinal tract is retroactively zoomed into a plurality of retained image sets based on the first segmented images; acquiring a plurality of intestinal cross section images corresponding to a plurality of segmentation point locations and a plurality of central line locations before and after the segmentation point locations on the intestinal three-dimensional geometric model, wherein the intestinal cross section images comprise intestinal cross section areas, and the segmentation point locations and the plurality of central line locations before and after the segmentation point locations are centroids of the plurality of intestinal cross section images respectively; judging whether the area of the intestinal tract section area in the plurality of intestinal tract cross section images is increased along with the increase of the distance from each central line point to the target point; if the area of the intestinal section area in the plurality of intestinal section images is not increased along with the increase of the distance from each central line point to the target point, determining the segmentation point to be an abnormal target point; removing the first segmented images corresponding to the abnormal target point positions from the plurality of first segmented images to obtain a plurality of second segmented images; dividing the first retained image images when the intestinal tract is withdrawn into a plurality of retained image sets based on the second segmented images;

a third determination unit for determining the cleanliness of the intestinal tract based on the plurality of retained image sets.

8. A computer device, characterized in that the computer device comprises:

one or more processors;

a memory; and

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the method of detecting intestinal cleanliness of any one of claims 1 to 6.

9. A computer-readable storage medium, having stored thereon a computer program which is loaded by a processor to execute the steps of the method for detecting intestinal cleanliness of any one of claims 1 to 6.