CN113643291A

CN113643291A - Method and device for determining esophagus marker infiltration depth grade and readable storage medium

Info

Publication number: CN113643291A
Application number: CN202111195227.2A
Authority: CN
Inventors: 于红刚; 张丽辉; 姚理文; 罗任权; 卢姿桦
Original assignee: Wuhan University WHU
Current assignee: Wuhan Endoangel Medical Technology Co Ltd
Priority date: 2021-10-14
Filing date: 2021-10-14
Publication date: 2021-11-12
Anticipated expiration: 2041-10-14
Also published as: CN113643291B

Abstract

The application provides a method, a device and a readable storage medium for determining the level of the infiltration depth of an esophageal marker, wherein the method comprises the following steps: acquiring a first esophageal mucosa image to be analyzed; performing first image preprocessing on the first esophageal mucosa image to obtain a preprocessed target esophageal mucosa image; determining representative color characteristics of the target esophageal mucosa image; and determining the esophagus marker infiltration depth grade corresponding to the first esophagus mucosa image based on the representative color feature of the target esophagus mucosa image and a preset standard color feature library. According to the embodiment of the application, the automatic identification of the esophagus marker infiltration depth grade is realized, and the identification efficiency and the identification accuracy are improved.

Description

Method and device for determining esophagus marker infiltration depth grade and readable storage medium

Technical Field

The application relates to the technical field of image processing, in particular to a method and a device for determining the level of the infiltration depth of an esophageal marker and a readable storage medium.

Background

Esophageal cancer is the eighth most common cancer type worldwide and is also the sixth leading cause of cancer death. China is a high-risk country of Esophageal Cancer (EC) and accounts for about 50% of the global esophageal cancer burden.

Compared with the traditional surgical operation, the endoscopic treatment of early esophageal cancer has the advantages of small wound, good prognosis, few complications and the like, and is widely applied to the treatment of early esophageal cancer. According to the guidelines of the japanese and european associations, the risk of tumor metastasis is relatively low (< 10%) in Epithelial (EP), intrinsic Muscularis (LPM), and Mucosal Muscularis (MM) and submucosal cancer infiltrates to a depth of 200 μm or less (SM 1), and endoscopic resection can be performed. The surgical resection of the esophageal tumor mainly aims at the esophageal cancer with the submucosal cancer infiltration depth of more than 200 mu m (SM 2 and deeper), the tumor metastasis risk is increased to more than 25 percent, and the surgical resection and the lymph node cleaning are required to realize the radical cure of the tumor. Therefore, an endoscopist needs to accurately detect the infiltration depth of the esophageal lesion under an endoscope so as to select a correct operation formula to treat a patient, reduce the health burden and obtain a better prognosis.

However, accurate infiltration depth detection of esophageal lesions has certain requirements on the seniors of endoscopic physicians, and under the condition of insufficient medical resources, how to effectively improve the efficiency and accuracy of accurate infiltration depth detection of esophageal lesions under endoscopy is a technical problem which needs to be solved at present.

Disclosure of Invention

The application provides a method and a device for determining the grade of the infiltration depth of an esophageal marker and a readable storage medium, and aims to effectively improve the efficiency and the accuracy of accurate infiltration depth detection of esophageal lesions under an endoscope.

In one aspect, the present application provides a method for determining an esophageal marker infiltration depth level, the method comprising:

acquiring a first esophageal mucosa image to be analyzed;

performing first image preprocessing on the first esophageal mucosa image to obtain a preprocessed target esophageal mucosa image;

determining a representative color characteristic of the target esophageal mucosa image;

and determining the esophagus marker infiltration depth grade corresponding to the first esophagus mucosa image based on the representative color feature of the target esophagus mucosa image and a preset standard color feature library.

In one possible implementation manner of the present application, the preset standard color feature library includes preset individual nuclear color features corresponding to preset determined esophageal marker infiltration depth levels;

determining an esophageal marker infiltration depth grade corresponding to the first esophageal mucosa image based on the representative color feature of the target esophageal mucosa image and a preset standard color feature library, wherein the determining includes:

respectively calculating the Euclidean distance between each kernel color feature in the preset kernel color features and the three-channel pixel corresponding to the representative color feature to obtain a preset Euclidean distance;

selecting a target Euclidean distance with the minimum distance from the preset Euclidean distances;

determining a target kernel color feature corresponding to the target Euclidean distance;

and determining the esophagus marker infiltration depth grade corresponding to the target nuclear color feature as the esophagus marker infiltration depth grade corresponding to the first esophagus mucosa image.

In one possible implementation manner of the present application, before determining the esophageal marker infiltration depth level corresponding to the first esophageal mucosa image based on the representative color feature of the target esophageal mucosa image and a preset standard color feature library, the method further includes:

acquiring a first esophageal marker image sample set which comprises a first esophageal marker image sample set corresponding to a preset determined esophageal marker infiltration depth grade in a first time period;

classifying the first esophageal marker image sample set into a preset first esophageal marker image sub-sample set corresponding to the preset determined esophageal marker infiltration depth grade;

respectively calculating the nuclear color characteristics corresponding to each esophageal marker image subsample set in the preset first esophageal marker image subsample set to obtain preset nuclear color characteristics corresponding to preset determined esophageal marker infiltration depth levels;

and constructing the preset standard color feature library based on the preset individual core color features.

In one possible implementation manner of the present application, after determining the esophageal marker infiltration depth level corresponding to the first esophageal mucosa image based on the representative color feature of the target esophageal mucosa image and a preset standard color feature library, the method further includes:

acquiring a second esophageal marker image sample set which comprises a second esophageal marker image sample set corresponding to a preset determined esophageal marker infiltration depth grade in a second time period;

classifying the second esophageal marker image sample set into a preset second esophageal marker image sub-sample set corresponding to the preset determined esophageal marker infiltration depth grade;

calibrating each nuclear color feature in the preset standard color feature library based on the first esophageal marker image sample set, the second esophageal marker image subsample set and a preset calibration model.

In one possible implementation manner of the present application, the determining a representative color feature of the target esophageal mucosa image includes:

acquiring a primary color feature set in the target esophageal mucosa image;

removing black color features in the primary color feature set to obtain a middle-level color feature set;

calculating the mean value of three-channel pixels corresponding to each color feature in the middle-level color feature set;

and selecting a median value in the mean value set of three-channel pixels corresponding to all the color features in the middle-level color feature set as a representative color feature of the target esophageal mucosa image.

In a possible implementation manner of the present application, the performing a first image preprocessing on the first esophageal mucosa image to obtain a preprocessed target esophageal mucosa image includes:

judging whether the first esophageal mucosa image meets the preset image specification requirement or not;

if the size of the first esophageal mucosa image does not meet the preset image specification requirement, adjusting the size of the first esophageal mucosa image to a target size meeting the preset image specification requirement to obtain an adjusted second esophageal mucosa image;

performing second image preprocessing on the second esophageal mucosa image to obtain a preprocessed third esophageal mucosa image;

and adjusting the number of the correspondingly displayed colors of the third esophageal mucosa image to obtain a target esophageal mucosa image.

In a possible implementation manner of the present application, the performing second image preprocessing on the second esophageal mucosa image to obtain a preprocessed third esophageal mucosa image includes:

converting the second esophageal mucosa image from an RGB color mode to a Lab color mode to obtain a first mode conversion diagram;

filtering the L channel of the first mode conversion diagram to obtain a filtered first mode conversion diagram;

and converting the filtered first mode conversion diagram from a Lab color mode to an RGB color mode to obtain a third esophageal mucosa image.

In another aspect, the present application provides an esophageal marker infiltration depth level determination device, comprising:

the first acquisition unit is used for acquiring a first esophageal mucosa image to be analyzed;

the first image preprocessing unit is used for performing first image preprocessing on the first esophageal mucosa image to obtain a preprocessed target esophageal mucosa image;

a first determination unit for determining a representative color feature of the target esophageal mucosa image;

and the second determination unit is used for determining the esophagus marker infiltration depth grade corresponding to the first esophagus mucosa image based on the representative color feature of the target esophagus mucosa image and a preset standard color feature library.

the second determining unit is specifically configured to:

In one possible implementation manner of the present application, the apparatus is further configured to:

In a possible implementation manner of the present application, the first determining unit is specifically configured to:

acquiring a primary color feature set in the target esophageal mucosa image;

In a possible implementation manner of the present application, the first image preprocessing unit is specifically configured to:

In another aspect, the present application further provides a computer device, including:

one or more processors;

a memory; and

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the esophageal marker infiltration depth level determination method.

In another aspect, the present application further provides a computer readable storage medium, on which a computer program is stored, the computer program being loaded by a processor to perform the steps of the method for determining the level of esophageal marker infiltration depth.

The method comprises the steps of firstly obtaining a first esophageal mucosa image to be analyzed; then, carrying out first image preprocessing on the first esophageal mucosa image to obtain a preprocessed target esophageal mucosa image; then determining the representative color characteristic of the target esophageal mucosa image; and finally, determining the esophagus marker infiltration depth grade corresponding to the first esophagus mucosa image based on the representative color feature of the target esophagus mucosa image and a preset standard color feature library, so that the automatic identification of the esophagus marker infiltration depth grade is realized, and the identification efficiency and the identification accuracy are improved by quantifying the color feature of the image and the infiltration depth of the esophagus marker.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic view of a scene of an esophageal marker infiltration depth level determination system provided in an embodiment of the present application;

fig. 2 is a schematic flowchart of an embodiment of a method for determining an esophageal marker infiltration depth level provided in an embodiment of the present application;

FIG. 3 is a flowchart illustrating an embodiment of step 202 in the present application;

FIG. 4 is a schematic flowchart of an embodiment of step 303 in the present application;

FIG. 5 is a schematic flow chart diagram illustrating an embodiment of step 203 in the present application;

fig. 6 is a schematic flowchart of another embodiment of a method for determining an esophageal marker infiltration depth level provided in an embodiment of the present application;

FIG. 7 is a flowchart illustrating an embodiment of step 204 in the present application;

FIG. 8 is a schematic flow chart diagram illustrating a method for determining a level of esophageal marker infiltration depth provided in an embodiment of the present application;

fig. 9 is a schematic structural diagram of an embodiment of an esophageal marker infiltration depth grade determination device provided in an embodiment of the present application;

FIG. 10 is a schematic structural diagram of an embodiment of a computer device provided by an embodiment of the present application;

FIG. 11 is a graph of the effect of edge-filling image scaling provided in the embodiments of the present application;

FIG. 12 is a diagram illustrating the effect of palette control provided in an embodiment of the present application;

fig. 13 is a schematic view of color principal component analysis provided in the embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In the description of the present application, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like indicate orientations or positional relationships based on those shown in the drawings, and are used merely for convenience of description and for simplicity of description, and do not indicate or imply that the referenced device or element must have a particular orientation, be constructed in a particular orientation, and be operated, and thus should not be considered as limiting the present application. Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", may explicitly or implicitly include one or more of the described features. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.

In this application, the word "exemplary" is used to mean "serving as an example, instance, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the application. In the following description, details are set forth for the purpose of explanation. It will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. In other instances, well-known structures and processes are not set forth in detail in order to avoid obscuring the description of the present application with unnecessary detail. Thus, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

The embodiment of the application provides a method and a device for determining the level of the infiltration depth of an esophageal marker, and a readable storage medium, which are respectively described in detail below.

As shown in fig. 1, fig. 1 is a scene schematic diagram of an esophageal marker infiltration depth level determination system provided in an embodiment of the present application, where the esophageal marker infiltration depth level determination system may include a plurality of terminals 100 and a server 200, the terminals 100 and the server 200 are connected by a network, and an esophageal marker infiltration depth level determination device is integrated in the server 200, such as the server in fig. 1, and the terminals 100 may access the server 200.

In the embodiment of the present application, the server 200 is mainly used for acquiring a first esophageal mucosa image to be analyzed; performing first image preprocessing on the first esophageal mucosa image to obtain a preprocessed target esophageal mucosa image; determining a representative color characteristic of the target esophageal mucosa image; and determining the esophagus marker infiltration depth grade corresponding to the first esophagus mucosa image based on the representative color feature of the target esophagus mucosa image and a preset standard color feature library.

In this embodiment, the server 200 may be an independent server, or may be a server network or a server cluster composed of servers, for example, the server 200 described in this embodiment includes, but is not limited to, a computer, a network terminal, a single network server, a plurality of network server sets, or a cloud server composed of a plurality of servers. Among them, the Cloud server is constituted by a large number of computers or web servers based on Cloud Computing (Cloud Computing). In the embodiment of the present application, the server and the terminal may implement communication through any communication manner, including but not limited to mobile communication based on the third Generation Partnership Project (3 GPP), Long Term Evolution (LTE), Worldwide Interoperability for Microwave Access (WiMAX), or computer network communication based on the TCP/IP Protocol Suite (TCP/IP), User Datagram Protocol (UDP), and the like.

It is to be understood that the terminal 100 used in the embodiments of the present application may be a device that includes both receiving and transmitting hardware, as well as a device that has both receiving and transmitting hardware capable of performing two-way communication over a two-way communication link. Such a terminal may include: a cellular or other communication device having a single line display or a multi-line display or a cellular or other communication device without a multi-line display. The terminal 100 may specifically be a desktop terminal or a mobile terminal, and the terminal 100 may also specifically be one of a mobile phone, a tablet computer, a notebook computer, and the like.

Those skilled in the art will understand that the application environment shown in fig. 1 is only one application scenario of the present application, and does not constitute a limitation to the application scenario of the present application, and other application environments may also include more or fewer terminals than those shown in fig. 1, or a server network connection relationship, for example, only 1 server and 2 terminals are shown in fig. 1. It is understood that the esophageal marker infiltration depth level determination system may further include one or more other servers, or/and one or more terminals connected to a server network, and is not limited herein.

In addition, as shown in fig. 1, the esophageal marker infiltration depth level determination system may further include a memory 300 for storing data, such as an esophageal mucosa image of the user and esophageal marker infiltration depth level determination data, for example, esophageal marker infiltration depth level determination data when the esophageal marker infiltration depth level determination system is in operation.

It should be noted that the scene schematic diagram of the esophageal marker infiltration depth level determination system shown in fig. 1 is only an example, and the esophageal marker infiltration depth level determination system and the scene described in the embodiment of the present application are for more clearly illustrating the technical solution of the embodiment of the present application, and do not form a limitation on the technical solution provided in the embodiment of the present application.

Next, a method for determining the level of the esophageal marker infiltration depth provided by the embodiment of the present application will be described.

In an embodiment of the method for determining an esophageal marker infiltration depth level according to the embodiment of the present application, an esophageal marker infiltration depth level determining apparatus is used as an execution subject, and for simplicity and convenience of description, the execution subject is omitted in subsequent method embodiments, and the esophageal marker infiltration depth level determining apparatus is applied to a computer device, and the method includes: acquiring a first esophageal mucosa image to be analyzed; performing first image preprocessing on the first esophageal mucosa image to obtain a preprocessed target esophageal mucosa image; determining representative color characteristics of the target esophageal mucosa image; and determining the esophagus marker infiltration depth grade corresponding to the first esophagus mucosa image based on the representative color feature of the target esophagus mucosa image and a preset standard color feature library.

Referring to fig. 2 to 13, fig. 2 is a schematic flowchart illustrating an embodiment of a method for determining an esophageal marker infiltration depth level according to an embodiment of the present disclosure, where the method for determining the esophageal marker infiltration depth level includes steps 201 to 204:

201. a first esophageal mucosa image to be analyzed is acquired.

The first esophageal mucosa image may be acquired by Narrow Band Imaging (NBI), and specifically, the first esophageal mucosa image may include a background mucosa color. NBI is a technique that filters out a broadband spectrum of red, blue, green light waves emitted by an endoscopic light source using a filter, leaving only a narrow band spectrum for obtaining the conditions of the digestive tract. The main advantages of the NBI endoscope technology are: not only can accurately observe the epithelial form of the alimentary tract mucosa, such as the epithelial glandular structure, but also can observe the form of an epithelial vascular network. The new technology can better help an endoscopist to distinguish gastrointestinal epithelium, such as intestinal metaplasia epithelium in Barrett esophagus, change of blood vessel morphology in gastrointestinal inflammation and irregular change of early tumor fovea of gastrointestinal tract, thereby improving the accuracy rate of endoscopic detection.

202. And carrying out first image preprocessing on the first esophageal mucosa image to obtain a preprocessed target esophageal mucosa image.

In general, the size of the original image without the first image preprocessing may cause a great influence on the subsequent analysis of the color feature information included in the image, and may seriously reduce the accuracy and efficiency of the detection result. Therefore, before the first esophageal mucosa image is analyzed, the first image preprocessing needs to be performed on the first esophageal mucosa image to improve the identification accuracy and efficiency of the image.

For a specific image preprocessing scheme, please refer to the following embodiments, which are not described herein.

203. Representative color characteristics of the target esophageal mucosa image are determined.

204. And determining the esophagus marker infiltration depth grade corresponding to the first esophagus mucosa image based on the representative color feature of the target esophagus mucosa image and a preset standard color feature library.

Wherein the esophageal marker may be esophageal tumor, inflammatory material, polyp, esophageal foreign body (such as external swallow). The application mainly exemplifies esophageal tumors (or esophageal cancer), namely esophageal markers, namely esophageal cancer, and esophageal marker infiltration depth grades, namely esophageal marker infiltration depth grades.

Specifically, esophageal (superficial) tumors often appear as brown regions under NBI light source endoscopy. The brown region under the non-amplified light source of NBI consists mainly of irregularly distended intra-epithelial papillary capillary loops (IPCLs). However, background coloring areas, also called background color (BGC), exist between areas where IPCL can be clearly seen under NBI amplified light source.

Different from endoscopic imaging under a white light source, the NBI narrow-band imaging technology enhances the image of the alimentary canal mucous membrane blood vessel and highlights the change of the mucous membrane surface capillary by filtering broadband spectrums in red, blue and green light waves, so that the NBI amplification light source is the primary basis for judging the infiltration depth of the esophageal tumor.

The preset standard color feature library comprises preset nuclear color features corresponding to preset determined esophageal marker infiltration depth levels, the target nuclear color features are determined from the preset nuclear color features by judging the relationship between the representative color features of the target esophageal mucosa image and each nuclear color feature in the preset standard color feature library, and then the esophageal marker infiltration depth levels corresponding to the first esophageal mucosa image are determined based on the corresponding relationship between the target color features and the determined esophageal marker infiltration depth levels.

For a specific determination scheme, please refer to the following specific embodiments, which are not described herein.

The method comprises the steps of firstly obtaining a first esophageal mucosa image to be analyzed; then, carrying out first image preprocessing on the first esophageal mucosa image to obtain a preprocessed target esophageal mucosa image; then determining representative color characteristics of the target esophageal mucosa image; and finally, determining the esophagus marker infiltration depth grade corresponding to the first esophagus mucosa image based on the representative color feature of the target esophagus mucosa image and a preset standard color feature library, so that the automatic identification of the esophagus marker infiltration depth grade is realized, and the identification efficiency and the identification accuracy are improved by quantifying the color feature of the image and the infiltration depth of the esophagus marker.

In this embodiment of the application, as shown in fig. 3, step 202, performing first image preprocessing on the first esophageal mucosa image to obtain a preprocessed target esophageal mucosa image, includes steps 301 to 304:

301. and judging whether the first esophageal mucosa image meets the preset image specification requirement or not.

302. And if the size of the first esophageal mucosa image does not meet the preset image specification requirement, adjusting the size of the first esophageal mucosa image to a target size meeting the preset image specification requirement, and obtaining an adjusted second esophageal mucosa image.

The target size required by the preset image planning may be adjusted according to actual conditions, and in the next step 303, before performing the second image preprocessing on the second esophageal mucosa image, the target size is required to be (w, h), whereas if the initial size of the first esophageal mucosa image is (w 0, h 0), the initial first esophageal mucosa image size needs to be adjusted to the target size.

Specifically, the present application may use image boundary filling, for example, the initial size (w 0, h 0) of the first esophageal mucosa image, and set the target size (w, h), specifically, the target size in the present application may be (480 ).

First, a scaling factor is determined,

then the picture size after scaling is

(ii) a The original image is centered during the border filling, and the black border is filled at the edge. Width of broadside filling:

the filling width of the long side is as follows:

thereby, the initial size of the first esophageal mucosa image is converted from (w 0, h 0) to the target size (w x, h).

Further, since there are at least two specific cases as described above, as shown in fig. 11, (it is determined that the width of the image is in the x direction and the height of the image is in the y direction) the width of the a1 image is larger than the height, and the height of the a2 image is larger than the width. Therefore, when the first is the tube mucosa image as the a1 image, i.e., w0 of the initial size of the first esophageal mucosa image>h0, fill the pixel value of 0 and height above and below the a1 image

The a1 image zoom effect is as shown for the a2 image.

When the first esophageal mucosa image is the b1 image, i.e., w0 of the initial size of the first esophageal mucosa image<h0, the left and right sides of the image are filled with 0 and width

The b2 image zoom effect is shown as the b2 image.

303. And carrying out second image preprocessing on the second esophageal mucosa image to obtain a preprocessed third esophageal mucosa image.

In order to further improve the detection accuracy and efficiency of the image to be analyzed, second image preprocessing can be performed, and the specific scheme can be adjusted according to actual requirements. To achieve the purpose, other schemes under the core idea of the present application should fall within the protection scope of the present application.

304. And adjusting the number of the correspondingly displayed colors of the third esophageal mucosa image to obtain the target esophageal mucosa image.

Specifically, the third esophageal mucosa image is converted from an RGB color mode to a P color mode, the number of colors in the color palette is controlled after the color dithering is increased, and the third esophageal mucosa image is expressed by a certain number of color features, so that the target esophageal mucosa image expressed by a certain number of color features is obtained.

The P color mode is understood to be a palette mode, in which the palette is a preset table capable of storing up to 256 different color schemes, and the color schemes can be defined according to RGB values. Color dithering is the process whereby the two colors demarcate significantly and robustly as rich colors go to less colors, and if dithering is used, transitional pixels are filled in these places.

As shown in fig. 12, the c1 image is an original image of the third esophageal mucosa map, and the c2, c3 and c4 images are images of the third esophageal mucosa map with the number of palette control colors of n =10, n =5 and n =2 in the P color mode, respectively. Based on practical tests and research findings, the picture sets used in the present application for the esophageal marker infiltration depth are best judged for the infiltration depths of multiple levels when the number of colors n =10, and therefore the present application includes, but is not limited to, selecting the number of colors n = 10.

As shown in fig. 13, the right side bar graphs e1 and e2 each contain 10 colors, that is, when the original image is in the P color mode when the original image palette control number is 10, the 10 colors are expressed.

"0-9" in the bar graphs e1 and e2 indicates the order of the 10 colors in which "0" corresponds to the most frequent color and "9" corresponds to the least frequent color.

For example, the d1 image in fig. 13 corresponds to a "deep saturation type" picture, and when n =10, the corresponding color pixel values and the corresponding color appearance times of the color tones "0-9" in the bar graph e1 are as follows:

[0-(40127,(159,131,108)),1-(39260,(0,0,0)),2-(35653,(145,107,88)),3-(33753,(125,97,78)),4-(32174,(110,75,61)),5-(30981,(83,52,42)),6-(24150,(195,179,155)),7-(23724,(53,36,30)),8-(1776,(4,0,3)),9-(546,(0,0,3))]。

in the above-mentioned collection, each unit may be represented by x- (a, (R, G, B)), where x represents the number of color bars in the picture, (R, G, B) represents the corresponding color pixel value, and a represents the number of times the color corresponding to the (R, G, B) pixel value appears.

In further contrast, the d2 image in fig. 13 corresponds to a "non-deep infiltration type" picture, and when n =10, the color pixel values and the corresponding color appearance times of the color bars "0-9" in the corresponding bar graph e2 are as follows:

(39689,(158,154,126)),1-(39551,(0,0,0)),2-(35624,(138,138,111)),3-(35409,(114,107,84)),4-(32515,(137,120,96)),5-(31906,(154,141,115)),6-(27931,(184,179,150)),7-(16430,(85,71,57)),8-(2408,(3,0,4)),9-(681,(0,0,3))]。

in the embodiment of the present application, as shown in fig. 4, step 303, performing second image preprocessing on the second esophageal mucosa image to obtain a preprocessed third esophageal mucosa image, including steps 401 to 403:

401. and converting the second esophageal mucosa image from an RGB color mode to a Lab color mode to obtain a first mode conversion map.

Wherein, the Lab color mode is a mode simulating human eyes to recognize colors. There are three types of photoreceptor cells in the human eye, one that recognizes lightness (L), one that distinguishes between red and green, and one that distinguishes between yellow and blue. In the Lab color mode, there are three channels, a luminance channel (L channel), an a channel, and a b channel.

402. And carrying out filtering processing on an L channel of the first mode conversion diagram to obtain a filtered first mode conversion diagram.

The filtering process may specifically be bilateral filtering, where bilateral filtering is a nonlinear filtering method, and is a compromise process combining spatial proximity and pixel value similarity of an image, and simultaneously considers spatial domain information and gray level similarity to achieve the purpose of edge-preserving and denoising. The reason that the bilateral filter can achieve smooth denoising and well preserve edges is that the kernel of the filter is generated by two functions: one function determines the coefficients of the filter template from the euclidean distance of the pixels and the other function determines the coefficients of the filter from the difference in gray levels of the pixels.

The method comprises the following specific steps:

；

wherein the content of the first and second substances,

represents an output point;

by finger and

(2N +1) centered size range;

representing input point(s);

，

for the spatial proximity of the gaussian function,

，

is a gaussian function of the similarity of pixel values,

。

the bilateral filter is controlled by 3 parameters: filter half width N, parameter

And

. The larger N is, the stronger the smoothing effect is;

and

respectively controlling the spatial proximity factor

And a brightness similarity factor

The degree of attenuation of. Included in this application but not limited to taking N =5,

。

403. and converting the filtered first mode conversion diagram from a Lab color mode to an RGB color mode to obtain a third esophageal mucosa image.

In the embodiment of the present application, as shown in fig. 5, step 203 of determining a representative color feature of the image of the target esophageal mucosa includes steps 501 to 504:

501. a set of primary color features in an image of the target esophageal mucosa is acquired.

Wherein the primary set of color features may be extracted from the target esophageal mucosa image. Specifically, all color features in the target esophageal mucosa Image can be obtained by a gettools () method carried by a preset Image processing kit (Python Image Library, PIL):

。

502. and eliminating the black color features in the primary color feature set to obtain a middle-level color feature set.

After eliminating the black color feature in the primary color feature set, obtaining a middle-level color feature set

。

503. And calculating the mean value of the three-channel pixels corresponding to each color feature in the middle-level color feature set.

504. And selecting a median value in the mean value set of three channel pixels corresponding to all the color features in the medium-level color feature set as a representative color feature of the target esophageal mucosa image.

Namely, the representative color feature of the target esophageal mucosa image is the median value in the mean value set of three-channel pixels corresponding to all the color features in the medium-level color feature set

。

In this embodiment of the application, as shown in fig. 6, before determining, in step 204, an esophageal marker infiltration depth level corresponding to a first esophageal mucosa image based on a representative color feature of a target esophageal mucosa image and a preset standard color feature library, the method further includes steps 601 to 604:

601. and acquiring a first esophageal marker image sample set which comprises a first esophageal marker image sample set corresponding to a preset determined esophageal marker infiltration depth grade in a first time period.

The preset determined esophageal marker infiltration depth grades are preset grades which are obtained by classifying determined esophageal marker images after the pathological results of the esophageal marker infiltration depth are corrected according to a preset standard rule (wherein the preset standard rule is set by multiple specialist physicians), and the preset grades include but are not limited to three grades, and the application performs distance description according to the three grades, and the three grades sequentially include: a non-deep infiltration rating of d0, a shallow infiltration rating of d1, and a deep infiltration rating of d 2. Based on this, a first esophageal marker image sample set including images corresponding to a preset determined esophageal marker infiltration depth level within a first time period is obtained.

602. And classifying the first esophageal marker image sample set into a preset first esophageal marker image sub-sample set corresponding to a preset determined esophageal marker infiltration depth grade.

That is, each picture in the first esophageal marker image sample set is classified into a preset first esophageal marker image sub-sample set corresponding to a preset determined esophageal marker infiltration depth level.

For example, the total number of the first esophageal marker image sample sets is 10 ten thousand pictures, each of the pictures is classified into three levels in step 601 according to preset standard rules, for example, 34250 sample sets corresponding to the non-deep infiltration level d0 classification, 53197 sample sets corresponding to the shallow infiltration level d1 classification, and 12553 sample sets corresponding to the deep infiltration level d2 classification.

603. And respectively calculating the nuclear color characteristics corresponding to each esophageal marker image subsample set in the preset first esophageal marker image subsample set to obtain the preset nuclear color characteristics corresponding to the preset determined esophageal marker infiltration depth grade.

Based on the illustration in step 601 and step 603, the color features of the corresponding core of the three levels in the embodiment of the present application are calculated as follows:

the core color characteristics of the non-deep infiltration grade were:

where k is the number of esophageal marker image samples contained in the present grade, e.g., k = 34250;

the core color characteristics of the shallow infiltration rating are:

wherein l is the number of esophageal marker image samples contained in the grade, such as l =53197 sheets;

the core color characteristics of the deep infiltration grade are:

where p is the number of esophageal marker image samples contained in the present grade, e.g., p = 12553.

604. And constructing a preset standard color feature library based on the preset core color features.

In this embodiment of the application, as shown in fig. 7, in step 204, determining an esophageal marker infiltration depth level corresponding to a first esophageal mucosa image based on a representative color feature of a target esophageal mucosa image and a preset standard color feature library, includes steps 701 to 704:

701. and respectively calculating the Euclidean distance between each kernel color feature in the preset kernel color features and the three-channel pixel corresponding to the representative color feature to obtain the Euclidean distance including the preset kernel color features.

Euclidean metric (also known as euclidean distance) is a commonly used definition of distance, referring to the true distance between two points in an m-dimensional space, or the natural length of a vector (i.e., the distance of the point from the origin). The euclidean distance in two and three dimensions is the actual distance between two points.

In connection with the above embodiments, the representative color characteristics of the three-channel pixel are

And each of the three kernel color features is: the core color of the non-deep infiltration level is characterized by

To do so

And also comprises

，

，

) The core color of the shallow saturation grade is characterized by

To do so

And also comprises

，

，

) The deep infiltration level is characterized by a core color

To do so

And also comprises

，

，

）。

The Euclidean distance between each kernel color feature in the preset kernel color features and a three-channel pixel corresponding to the representative color feature is as follows:

(ii) a Wherein the content of the first and second substances,

、

and

sequentially comprises the kernel color characteristic of the non-deep infiltration level, the kernel color characteristic of the shallow infiltration level and the Euclidean distance of three-channel pixels corresponding to the kernel color characteristic of the deep infiltration level and the representative color characteristic.

702. And selecting a target Euclidean distance with the minimum distance from preset Euclidean distances.

Combining with step 701, comparing

、

And

and selecting the minimum value as the target Euclidean distance. For example,

and minimum.

703. And determining the target kernel color characteristics corresponding to the target Euclidean distance.

In connection with step 702, when determining

At a minimum, then shallow may be determinedAnd the nuclear color feature of the layer infiltration level is a target nuclear color feature corresponding to the target Euclidean distance.

704. And determining the esophagus marker infiltration depth grade corresponding to the target nuclear color feature as the esophagus marker infiltration depth grade corresponding to the first esophagus mucosa image.

In combination with step 703, when it is determined that the nuclear color feature of the shallow infiltration level is the target nuclear color feature corresponding to the target euclidean distance, it may be determined that the esophageal marker shallow infiltration depth level is the esophageal marker infiltration depth level corresponding to the first esophageal mucosa image.

In this embodiment of the application, as shown in fig. 8, after determining, in step 204, an esophageal marker infiltration depth level corresponding to a first esophageal mucosa image based on a representative color feature of a target esophageal mucosa image and a preset standard color feature library, the method further includes steps 801 to 803:

801. and acquiring a second esophageal marker image sample set which comprises a second esophageal marker image sample set corresponding to the preset determined esophageal marker infiltration depth grade in a second time period.

Wherein the second time period is later than the first time period. That is, after the above embodiment is implemented for a period of time, the kernel color features in the standard color feature library are optimized and calibrated based on the actual usage effect.

802. And classifying the second esophageal marker image sample set into a preset second esophageal marker image sub-sample set corresponding to the preset determined esophageal marker infiltration depth grade.

803. And calibrating each nuclear color feature in a preset standard color feature library based on the first esophageal marker image sample set, the second esophageal marker image subsample set and a preset calibration model.

With reference to the foregoing embodiments, the preset calibration model in the embodiment of the present application is as follows:

；

wherein i represents three infiltration depth grades, and i takes values of 0, 1 and 2 in the application;

m represents the number of esophageal marker image samples contained in a certain infiltration depth grade when the construction of the standard color feature library of the previous round (corresponding to the first time period) is completed;

n is the number of esophageal marker image samples newly added to a certain infiltration depth level when the round (corresponding to the second time period) is optimized;

newly adding a convergence coefficient of the average pixel error of the nuclear color feature of the infiltration depth of a certain level before and after the esophageal marker image, and when the error is smaller than the convergence coefficient, indicating that the calibration (or construction) of the standard color feature library of the level is finished

It should be noted that, the specific values of the above parameters can be adaptively adjusted according to actual requirements, and the adjustment based on the concept of the present application is all within the protection scope of the present application.

In order to better implement the method for determining the level of the esophageal marker infiltration depth in the embodiment of the present application, on the basis of the method for determining the level of the esophageal marker infiltration depth, an apparatus for determining the level of the esophageal marker infiltration depth is further provided in the embodiment of the present application, as shown in fig. 9, the apparatus 900 for determining the level of the esophageal marker infiltration depth includes a first obtaining unit 901, a first image preprocessing unit 902, a first determining unit 903, and a second determining unit 904:

a first acquiring unit 901, configured to acquire a first esophageal mucosa image to be analyzed.

The first image preprocessing unit 902 is configured to perform first image preprocessing on the first esophageal mucosa image to obtain a preprocessed target esophageal mucosa image.

A first determining unit 903 for determining a representative color feature of the image of the target esophageal mucosa.

And a second determining unit 904, configured to determine, based on the representative color feature of the target esophageal mucosa image and a preset standard color feature library, an esophageal marker infiltration depth level corresponding to the first esophageal mucosa image.

the second determining unit 904 is specifically configured to:

and respectively calculating the Euclidean distance between each kernel color feature in the preset kernel color features and the three-channel pixel corresponding to the representative color feature to obtain the Euclidean distance including the preset kernel color features.

And selecting a target Euclidean distance with the minimum distance from preset Euclidean distances.

And determining the target kernel color characteristics corresponding to the target Euclidean distance.

and acquiring a first esophageal marker image sample set which comprises a first esophageal marker image sample set corresponding to a preset determined esophageal marker infiltration depth grade in a first time period.

And classifying the first esophageal marker image sample set into a preset first esophageal marker image sub-sample set corresponding to a preset determined esophageal marker infiltration depth grade.

And respectively calculating the nuclear color characteristics corresponding to each esophageal marker image subsample set in the preset first esophageal marker image subsample set to obtain the preset nuclear color characteristics corresponding to the preset determined esophageal marker infiltration depth grade.

And constructing a preset standard color feature library based on the preset core color features.

classifying the second esophageal marker image sample set into a preset second esophageal marker image sub-sample set corresponding to a preset determined esophageal marker infiltration depth grade;

and calibrating each nuclear color feature in a preset standard color feature library based on the first esophageal marker image sample set, the second esophageal marker image subsample set and a preset calibration model.

In a possible implementation manner of the present application, the first determining unit 903 is specifically configured to:

a set of primary color features in an image of the target esophageal mucosa is acquired.

And eliminating the black color features in the primary color feature set to obtain a middle-level color feature set.

And calculating the mean value of the three-channel pixels corresponding to each color feature in the middle-level color feature set.

And selecting a median value in the mean value set of three channel pixels corresponding to all the color features in the medium-level color feature set as a representative color feature of the target esophageal mucosa image.

In a possible implementation manner of the present application, the first image preprocessing unit 902 is specifically configured to:

and judging whether the first esophageal mucosa image meets the preset image specification requirement or not.

And if the size of the first esophageal mucosa image does not meet the preset image specification requirement, adjusting the size of the first esophageal mucosa image to a target size meeting the preset image specification requirement, and obtaining an adjusted second esophageal mucosa image.

And carrying out second image preprocessing on the second esophageal mucosa image to obtain a preprocessed third esophageal mucosa image.

And adjusting the number of the correspondingly displayed colors of the third esophageal mucosa image to obtain the target esophageal mucosa image.

and converting the second esophageal mucosa image from an RGB color mode to a Lab color mode to obtain a first mode conversion map.

And carrying out filtering processing on an L channel of the first mode conversion diagram to obtain a filtered first mode conversion diagram.

The method comprises the steps of firstly acquiring a first esophageal mucosa image to be analyzed through a first acquisition unit 901; then, a first image preprocessing unit 902 performs first image preprocessing on the first esophageal mucosa image to obtain a preprocessed target esophageal mucosa image; then the representative color feature of the target esophageal mucosa image is determined by the first determination unit 903; finally, the second determining unit 904 determines the esophageal marker infiltration depth grade corresponding to the first esophageal mucosa image based on the representative color feature of the target esophageal mucosa image and the preset standard color feature library, so that the automatic identification of the esophageal marker infiltration depth grade is realized, and the identification efficiency and the identification accuracy are improved by quantifying the color feature of the image and the infiltration depth of the esophageal marker.

In addition to the above-mentioned method and apparatus for determining the level of the esophageal marker infiltration depth, an embodiment of the present application further provides a computer device, which integrates any one of the apparatuses for determining the level of the esophageal marker infiltration depth provided by the embodiments of the present application, where the computer device includes:

one or more processors;

a memory; and

one or more applications, wherein the one or more applications are stored in the memory and configured to perform, by the processor, the operations of any of the methods in any of the above-described esophageal marker infiltration depth level determination method embodiments.

The embodiment of the present application further provides a computer device, which integrates any one of the esophageal marker infiltration depth grade determination devices provided by the embodiments of the present application. Referring to fig. 10, fig. 10 is a schematic structural diagram of an embodiment of a computer device according to an embodiment of the present application.

Fig. 10 is a schematic structural diagram of an esophageal marker infiltration depth level determination device designed according to an embodiment of the present application, specifically:

the esophageal marker infiltration depth level determination device may include one or more processing cores of a processor 1001, one or more computer-readable storage media of a storage unit 1002, a power source 1003 and an input unit 1004. It will be appreciated by those skilled in the art that the esophageal marker infiltration depth level determination device structure shown in fig. 10 does not constitute a definition of an esophageal marker infiltration depth level determination device, and may include more or fewer components than shown, or some components in combination, or a different arrangement of components. Wherein:

the processor 1001 is a control center of the esophageal marker infiltration depth level determination device, connects each part of the entire esophageal marker infiltration depth level determination device by using various interfaces and lines, and executes various functions and processing data of the esophageal marker infiltration depth level determination device by running or executing software programs and/or modules stored in the storage unit 1002 and calling data stored in the storage unit 1002, thereby performing overall monitoring on the esophageal marker infiltration depth level determination device. Optionally, processor 1001 may include one or more processing cores; preferably, the processor 1001 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 1001.

The storage unit 1002 may be used to store software programs and modules, and the processor 1001 executes various functional applications and data processing by operating the software programs and modules stored in the storage unit 1002. The storage unit 1002 may mainly include a storage program area and a storage data area, where the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required by at least one function, and the like; the stored data area may store data created from use of the esophageal marker infiltration depth level determination device, and the like. In addition, the storage unit 1002 may include a high-speed random access memory, and may further include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory unit 1002 may further include a memory controller to provide the processor 1001 with access to the memory unit 1002.

The device for determining the level of the infiltration depth of the esophageal marker further comprises a power source 1003 for supplying power to each component, preferably, the power source 1003 can be logically connected with the processor 1001 through a power management system, so that functions of charging, discharging, power consumption management and the like can be managed through the power management system. The power source 1003 may also include any component including one or more of a dc or ac power source, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.

The esophageal marker infiltration depth level determination device may further comprise an input unit 1004, the input unit 1004 being operable to receive entered numerical or character information and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

Although not shown, the esophageal marker infiltration depth level determination device may further include a display unit and the like, which are not described in detail herein. Specifically, in this embodiment of the present application, the processor 1001 in the device for determining the level of the esophageal marker infiltration depth loads an executable file corresponding to one or more processes of an application program into the storage unit 1002 according to the following instructions, and the processor 1001 runs the application program stored in the storage unit 1002, so as to implement various functions as follows:

acquiring a first esophageal mucosa image to be analyzed; performing first image preprocessing on the first esophageal mucosa image to obtain a preprocessed target esophageal mucosa image; determining a representative color characteristic of the target esophageal mucosa image; and determining the esophagus marker infiltration depth grade corresponding to the first esophagus mucosa image based on the representative color feature of the target esophagus mucosa image and a preset standard color feature library.

To this end, an embodiment of the present application provides a computer-readable storage medium, which may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like. The computer readable storage medium has stored therein a plurality of instructions that can be loaded by a processor to perform the steps of any of the methods for determining an esophageal marker infiltration depth level provided by the embodiments of the present application. For example, the instructions may perform the steps of:

In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

The method, the device and the readable storage medium for determining the level of the infiltration depth of the esophageal marker provided by the embodiment of the present application are introduced in detail, and a specific example is applied to illustrate the principle and the implementation manner of the present application, and the description of the embodiment is only used to help understanding the method and the core concept of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A method for determining an esophageal marker infiltration depth level, the method comprising:

acquiring a first esophageal mucosa image to be analyzed;

2. The method for determining the grade of the esophageal marker infiltration depth according to claim 1, wherein the preset standard color feature library comprises preset nuclear color features corresponding to preset determined grade of the esophageal marker infiltration depth;

3. The method for determining the esophageal marker infiltration depth level according to claim 1, wherein before determining the esophageal marker infiltration depth level corresponding to the first esophageal mucosa image based on the representative color feature of the target esophageal mucosa image and a preset standard color feature library, the method further comprises:

4. The method for determining the esophageal marker infiltration depth level according to claim 3, wherein after determining the esophageal marker infiltration depth level corresponding to the first esophageal mucosa image based on the representative color feature of the target esophageal mucosa image and a preset standard color feature library, the method further comprises:

5. The method for determining the level of esophageal marker infiltration depth according to claim 1, wherein the determining the representative color feature of the target esophageal mucosa image comprises:

acquiring a primary color feature set in the target esophageal mucosa image;

6. The method for determining the esophageal marker infiltration depth level according to claim 1, wherein the performing of the first image preprocessing on the first esophageal mucosa image to obtain a preprocessed target esophageal mucosa image comprises:

7. The method for determining the esophageal marker infiltration depth level according to claim 6, wherein the second image preprocessing is performed on the second esophageal mucosa image to obtain a preprocessed third esophageal mucosa image, and the method comprises:

8. An esophageal marker infiltration depth level determination device, comprising:

9. A computer device, characterized in that the computer device comprises:

one or more processors;

a memory; and

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the esophageal marker infiltration depth level determination method of any of claims 1-7.

10. A computer-readable storage medium, having stored thereon a computer program which is loaded by a processor to perform the steps of the method of determining an esophageal marker infiltration depth level according to any one of claims 1-7.