CN110993099B - Ulcerative colitis severity evaluation method and system based on deep learning - Google Patents
Ulcerative colitis severity evaluation method and system based on deep learning Download PDFInfo
- Publication number
- CN110993099B CN110993099B CN201911312322.9A CN201911312322A CN110993099B CN 110993099 B CN110993099 B CN 110993099B CN 201911312322 A CN201911312322 A CN 201911312322A CN 110993099 B CN110993099 B CN 110993099B
- Authority
- CN
- China
- Prior art keywords
- ulcerative colitis
- score
- blood vessel
- severity
- scoring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10068—Endoscopic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30028—Colon; Small intestine
Abstract
The invention provides a method and a system for evaluating the severity of ulcerative colitis based on deep learning. The method comprises the steps of marking a white light colonoscope image for Mayo endoscopic scoring and ulcerative colitis endoscopic activity index scoring to form a sample set; constructing an ulcerative colitis severity evaluation model, and training by using the marked white light colonoscope image samples in the sample set; the ulcerative colitis severity evaluation model is an SPPNet network with characteristic pyramid pooling, and the output of the model comprises 4 softmax functions; and receiving the white light colonoscope image in real time, outputting score prediction results of characteristics of the Mayo endoscope, such as score, blood vessel typing, spontaneous hemorrhage and erosive ulcer by using the ulcerative colitis severity evaluation model, and accumulating the score prediction results of the characteristics of the blood vessel typing, the spontaneous hemorrhage and the erosive ulcer to obtain an activity index score of the ulcerative colitis endoscope. The method can automatically score inflammation, simultaneously meet the requirements of two clinical most common UC severity scoring, and has high result accuracy and good repeatability.
Description
Technical Field
The invention belongs to the field of ulcerative colitis image processing, and particularly relates to a method and a system for evaluating the severity of ulcerative colitis based on deep learning.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
Ulcerative Colitis (UC) is a chronic nonspecific inflammation mainly manifested by repeated abdominal pain, diarrhea, mucopurulent bloody stool, and the lesions are continuously distributed and may involve multiple intestinal segments of the rectum, sigmoid colon or the whole colon, and severe patients may have systemic infection and poisoning symptoms, even endanger life. Under the condition of a typical UC endoscope, the ulcer is characterized by continuous and diffuse erosive ulcer, mucous membrane is full of blood and is crisp, and spontaneous bleeding and purulent secretion adhesion are often accompanied. The endoscopic severity grading is closely related to the clinical symptoms, treatment prognosis and maintenance of remission stage of patients, and is a main means for judging the response of the patients to treatment drugs and guiding the next diagnosis and treatment. Because the disease course of UC is prolonged and repeated, and relapse and remission are alternated, the accurate evaluation of the severity degree of the UC has important significance for realizing the individualized and standardized diagnosis and treatment of the UC.
In the existing clinical guidelines, various UC endoscopic severity scores have been established and used for guiding diagnosis and treatment decisions, and the Mayo endoscopic score (Mayo ES), and Ulcerative colitis endoscopic activity index score (UCEIS index of severity, UCEIS) are most widely used. The inventor finds that all the endoscopic scores are artificial scores, and an endoscopy physician needs to judge the degree of the relevant characteristics of the colonic mucosa at the UC lesion by combining subjective experience, and then takes the part where the lesion is affected and the inflammation of the intestinal segment is the most serious to evaluate. At present, no objective scoring index under endoscope exists for quantitatively judging the severity of UC lesion. Due to the lack of uniform and objective scoring standards, the existing artificial UC endoscope severity scoring has the following inherent defects: first, there is a significant difference in scoring accuracy between experienced physicians and inexperienced physicians. The existing scoring is mainly based on subjective judgment of an operating physician on the severity of lesion features, and low-age physicians with little experience are often difficult to make accurate judgment, and particularly cannot demarcate mild and moderate lesions, so that scoring is inaccurate. Secondly, consistency between individuals with the same scorer is difficult to guarantee. The whole colonoscopy retrospective observation process is as long as 6-7 minutes, UC lesion is accumulated on a plurality of intestinal segments and is in diffuse distribution, and an operator often has difficulty in clearly recalling the most serious characteristics of inflammation to make accurate evaluation; moreover, the scoring is fine, so that more items are available, the manual scoring is time-consuming and labor-consuming, and the working efficiency of clinical colonoscopy is greatly affected. In addition, the heterogeneity among different scorers is large, and the repeatability is poor. It is often difficult for multiple evaluation physicians to obtain consistent scoring results, and the past scoring is often difficult to trace back, which is not favorable for long-term management of UC.
Disclosure of Invention
In order to solve the problems, the invention provides a method and a system for evaluating the severity of ulcerative colitis based on deep learning, which can automatically perform scoring calculation on the severity of inflammation of a white-light colonoscope image of ulcerative colitis, simultaneously meet the two most common clinical scoring requirements of MayoES and UCEIS, obtain more accurate and good repeatability results, are suitable for different endoscope center system platforms, are convenient to access and record, and can more effectively guide clinical diagnosis and treatment.
In order to achieve the purpose, the invention adopts the following technical scheme:
a first aspect of the present invention provides a method for deep learning-based assessment of the severity of ulcerative colitis, comprising:
labeling a Mayo endoscopic score and an ulcerative colitis endoscopic activity index score on the white light colonoscope image to form a sample set; wherein, the activity index score under the endoscope of the ulcerative colitis is respectively and correspondingly marked with scores according to three characteristics of blood vessel typing, spontaneous hemorrhage and erosive ulcer;
constructing an ulcerative colitis severity evaluation model, and training by using the marked white light colonoscope image samples in the sample set; the ulcerative colitis severity evaluation model is an SPPNet network with characteristic pyramid pooling, and outputs score prediction results which comprise 4 softmax functions and respectively correspond to features of Mayo endoscopic scoring, UCEIS blood vessel typing, UCEIS spontaneous bleeding and UCEIS erosive ulcer of a single frame image;
the method comprises the steps of receiving a white light colonoscope image in real time, extracting relevant scoring characteristics under a Mayo endoscope and relevant blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics of an activity index under the ulcerative colitis endoscope by using an ulcerative colitis severity evaluation model, outputting scoring prediction results of the Mayo endoscope scoring, blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics, and accumulating the scoring prediction results of the blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics to obtain the activity index score under the ulcerative colitis endoscope.
A second aspect of the present invention provides a deep learning-based ulcerative colitis severity assessment system, comprising:
the sample set forming module is used for carrying out labeling Mayo endoscopic scoring and ulcerative colitis endoscopic activity index scoring on the white light colonoscope image to form a sample set; wherein, the activity index score under the endoscope of the ulcerative colitis is respectively and correspondingly marked with scores according to three characteristics of blood vessel typing, spontaneous hemorrhage and erosive ulcer;
the evaluation model building and training module is used for building an ulcerative colitis severity evaluation model and training by using the marked white light colonoscope image samples in the sample set; the ulcerative colitis severity evaluation model is an SPPNet network with characteristic pyramid pooling, and outputs score prediction results which comprise 4 softmax functions and respectively correspond to features of Mayo endoscopic scoring, UCEIS blood vessel typing, UCEIS spontaneous bleeding and UCEIS erosive ulcer of a single frame image;
and the severity evaluation output module is used for receiving the white light colonoscope image in real time, extracting the relevant scoring characteristics under the Mayo endoscope and the relevant blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics of the activity index under the ulcerative colitis endoscope by using the ulcerative colitis severity evaluation model, outputting scoring prediction results of the scoring, blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics under the Mayo endoscope, and accumulating the scoring prediction results of the blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics to obtain the activity index score under the ulcerative colitis endoscope.
A third aspect of the present invention provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps in the method for deep learning-based ulcerative colitis severity assessment as described above.
A fourth aspect of the invention provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method for deep learning based ulcerative colitis severity assessment as described above when executing the program.
The invention has the beneficial effects that:
(1) according to the invention, by utilizing the SPPNet network with characteristic pyramid pooling, two clinical most common scoring prediction results of the score under the Mayo endoscope and the activity index under the ulcerative colitis endoscope of a single-frame image can be simultaneously output, and the severity of the ulcerative colitis can be automatically scored, deduced and predicted according to probability.
(2) The method is based on the single-frame image scoring result, automatically compares and identifies the most serious lesion, and can perform indiscriminate and standard-identical continuous evaluation on the UC lesions of multiple intestine segments and diffuse type, so that the evaluation of the UC severity degree under the endoscope is more accurate and reliable.
(3) The invention can be well adapted to different endoscope center computer system environments, is convenient to store records and guides the long-term standardized diagnosis and treatment of UC patients.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention.
Fig. 1 is a flowchart of a method for evaluating the severity of ulcerative colitis based on deep learning according to an embodiment of the present invention;
FIG. 2 is a white light colonoscope image with different scores according to the Mayo endoscopic scoring provided by the present invention;
FIG. 3 is a white light colonoscope image with different scores corresponding to the score of the endoscopic activity index score for ulcerative colitis according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of labeling Mayo endoscopic scoring and ulcerative colitis endoscopic activity index scoring for a single frame of colonoscope images according to an embodiment of the present invention;
FIG. 5 is a schematic structural diagram of a model for evaluating the severity of ulcerative colitis according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a system for evaluating severity of ulcerative colitis based on deep learning according to an embodiment of the present invention.
Detailed Description
The invention is further described with reference to the following figures and examples.
It is to be understood that the following detailed description is exemplary and is intended to provide further explanation of the invention as claimed. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
Example 1
Fig. 1 is a flowchart of a method for evaluating the severity of ulcerative colitis based on deep learning according to this embodiment.
The specific implementation process of the deep learning-based ulcerative colitis severity assessment method of this embodiment is described in detail below with reference to fig. 1.
As shown in fig. 1, the present embodiment provides a method for evaluating the severity of ulcerative colitis based on deep learning, which includes:
step S101: marking the white-light colonoscope image for Mayo endoscopic scoring and ulcerative colitis endoscopic activity index scoring to form a sample set; wherein, the activity index score under the ulcerative colitis endoscope is respectively and correspondingly marked according to three characteristics of UCEIS blood vessel typing, UCEIS spontaneous bleeding and UCEIS erosive ulcer.
In the implementation process, before labeling the white light colonoscope image, the method further comprises:
and reserving the region of interest for each white light colonoscope image by utilizing a de-blackening algorithm, and then performing data enhancement, random overturning and preset-size scaling on the white light colonoscope images.
As shown in FIG. 2, the score under Mayo endoscope is selected from 0-3 points and the corresponding score is labeled, 0 represents normal or inactive lesion; 1 indicates mild activity with edema, reduced vessel image and mild fragility; 2, moderate activity with marked edema, disappearing blood vessel image, mucosal fragility and erosion; 3 indicates that severe activity has continued bleeding and ulceration.
As shown in fig. 3, the endoscopic activity index score of ulcerative colitis is labeled by the following three characteristics:
blood vessel typing: 0 indicates a clearly visible vessel image, or a blurred and slightly missing edge vessel; 1 represents a blood vessel image with a sheet-like disappearance; 2, the vessel image disappears completely;
spontaneous hemorrhage: 0 indicates no visible bleeding; 1 indicates that bleeding from visible bleeding spots or banding coagulation of the mucosa can be washed away; 2 indicates mild active bleeding in the cavity; 3 represents severe active hemorrhage or persistent mucosal bleeding in the cavity;
and (3) erosive ulcer: 0 indicates normal mucosa, no obvious erosion and ulcer; 1 represents mucosal defect with erosion less than 5mm, white or yellowish with flat edge; 2 superficial ulcer with mucosa defect less than 5mm, with whitish fur; 3 indicates a deep ulcer with mucosal defect greater than 5mm, possibly with raised edges.
Fig. 4 shows an example of labeling a white-light colonoscope image, i.e., a single-frame colonoscope image, corresponding to a Mayo endoscopic score of 1, a blood vessel typing of 2, a spontaneous bleeding of 1, and an erosive ulcer of 1.
The features related to MayoES scoring include, but are not limited to, colonic mucosal edema, erosion degree and fragility feature of a lesion area.
It is understood that in other embodiments, the score under the Mayo endoscope and the score of the vascularity, spontaneous bleeding and erosive ulceration characteristics of the endoscopic activity index of ulcerative colitis may be specifically set by one skilled in the art according to the actual circumstances.
Step S102: constructing an ulcerative colitis severity evaluation model, and training by using the marked white light colonoscope image samples in the sample set; the ulcerative colitis severity evaluation model is an SPPNet network with characteristic pyramid pooling, and outputs score prediction results which comprise 4 softmax functions and respectively correspond to the features of the single-frame image under Mayo endoscope scoring, blood vessel typing, spontaneous hemorrhage and erosive ulcer.
In order to accurately infer the UC image Mayo ES (Mayo endoscopic scoring) and UCEIS (ulcerative colitis endoscopic activity index scoring) at the same time, the embodiment of the invention innovatively uses a spatial pyramid network (SPPNet) and modifies a prediction layer of the network to support multi-label prediction.
SPPNet can be applied to arbitrary sizes, and in order to solve the training problem of different image sizes, two preset sizes are adopted: 180 x 180,224 x 224 (which have processed the image accordingly in steps 1 and 2). The aforementioned region image of 224 is changed to 180 size using scaling instead of cropping. Thus, the regions of different dimensions are simply different in resolution, not in content and layout, and the 180 network and 224 network have the same parameters. During training, sppnets of different input sizes are implemented by using two fixed-size networks that share parameters. To reduce the overhead of switching from one network (e.g., 224) to another network (e.g., 180), a complete epoch is trained on each network and then switched to the other network (weight preservation) at the next completed epoch. The main purpose of multi-scale training is to simulate different input sizes while ensuring that the fixed-scale network which is better optimized at present is fully utilized, so as to better adapt to various endoscopic image requirements. In addition to the two-scale implementation described above, different SXS inputs were also tested in each epoch, with S being chosen uniformly from 180 to 224.
ZFNET is adopted by the Backbone of the SPPNet, and for realizing Mayo ES and UCEIS scoring related feature identification of a UC single-frame image, feature pyramid pooling and multi-label prediction are integrated, and the model is improved as follows:
characteristic pyramid pooling: the pooling of the original SPPNet before the fully connected layer is replaced by feature pyramid pooling, and the structure of the feature pyramid is shown in fig. 5. The use of the characteristic pyramid can ensure that the image is input in any scale and finally converted into a uniform dimension input to the full connection layer. In addition, the characteristic pyramid can extract various characteristics of UC mucous membrane edema, blood vessel definition, spontaneous hemorrhage, erosion ulcer and the like on different dimensions, find small differences among images and promote the learning of UC lesion image characteristics.
Multi-label classification: the invention innovatively modifies the SPPNet network to support multi-label classification prediction. 4 labels of MayoES, blood vessel typing, spontaneous bleeding and erosive ulcer are defined. Wherein Mayo ES comprises 4 categories of 0-3, the blood vessel type comprises 3 categories of 0-2, and the spontaneous hemorrhage and anabrotic ulcer each comprise 4 categories of 0-3. And adjusting the number of the nodes output by the last full-connection layer of the ZFNET to be 4, and respectively scoring corresponding to the 4 labels. And 4 fully-connected nodes are respectively connected with 4 softmax functions finally, each softmax function corresponds to each label and is classified, 4 classifications {0,1,2,3} and corresponding probabilities thereof are output, and because the blood vessel typing only comprises 3 classifications, the output values {0,1} of the softmax corresponding to the blood vessel typing are mapped to {0}, {2} is mapped to {1}, and {3} is mapped to {2 }.
The SPPNet finally outputs a score prediction result containing 4 softmax functions, wherein the 4 functions respectively correspond to Mayo ES, blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics of a single-frame image. And respectively performing cross entropy loss on each softmax output result, wherein the final loss function is the average value of 4 cross entropy loss functions. And evaluating the error between the predicted value and the actual value by using a loss function, and guiding the adjustment of the model parameters. In this embodiment, the training network uses the SGD as the optimizer for training.
Step S103: the method comprises the steps of receiving a white light colonoscope image in real time, extracting relevant scoring characteristics and relevant blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics under a Mayo endoscope by using an ulcerative colitis severity evaluation model, outputting scoring prediction results of the relevant scoring, blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics under the Mayo endoscope, and accumulating the scoring prediction results of the blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics to obtain an activity index score under the ulcerative colitis endoscope.
Specifically, the white light colonoscope image received in real time is subjected to black edge removing processing, additional data enhancement and scaling processing are not needed to be performed on the image, the processed image is input into a trained ulcerative colitis severity evaluation model, and finally, 4 scoring categories output by softmax and corresponding probability values can be obtained, according to the training sequence, the first output category of softmax is the scoring result of Mayo ES of a single-frame image, the categories output by 2-4 softmax are blood vessel typing scoring, spontaneous hemorrhage scoring and erosive ulcer scoring, and the three characteristics are accumulated, namely UCEIS scoring of a single-frame image.
It is understood that in other embodiments, the weighting values of the blood vessel type, spontaneous bleeding and erosive ulcer characteristics can be specifically set by a person skilled in the art according to actual situations.
In another embodiment, the method for evaluating the severity of ulcerative colitis based on deep learning further comprises:
receiving a plurality of continuous frames of white light colonoscope images, and if the probabilities corresponding to the Mayo endoscopic score and the ulcerative colitis endoscopic activity index score are both greater than 0.5, and the Mayo endoscopic score and the ulcerative colitis endoscopic activity index score are both greater than or equal to corresponding preset scores (for example, the Mayo endoscopic score preset threshold score is 2, the ulcerative colitis endoscopic activity index score preset threshold score is 6), and the continuous occurrence times are not lower than preset times (for example, 5 times), determining the positions corresponding to the white light colonoscope images as the most serious positions of the ulcerative colitis; the probability corresponding to the corresponding score is output by the softmax function, and the probability corresponding to the activity index score under the ulcerative colitis endoscope is the mean value of the probabilities corresponding to the relevant characteristics of blood vessel typing, spontaneous hemorrhage and erosive ulcer.
Wherein a higher score indicates more severe ulcerative colitis.
In the embodiment, the SPPNet network with the characteristic pyramid pooling is used for outputting the scoring of the single-frame image under the Mayo endoscope and the scoring prediction results of the blood vessel typing, spontaneous bleeding and erosive ulcer characteristics of the activity index under the ulcerative colitis endoscope, and the severity of the ulcerative colitis is subjected to automatic scoring inference and probability prediction.
The embodiment is based on the single-frame image scoring result, automatically compares and identifies the most serious lesion, and can perform indiscriminate and standard-identical continuous evaluation on the UC lesions of multiple intestine segments and diffuse type, so that the evaluation of the UC severity degree under the endoscope is more accurate and reliable.
The embodiment can be well suitable for different endoscope center computer system environments, is convenient for storing records and guides long-term standardized diagnosis and treatment of UC patients.
Example 2
Fig. 6 is a schematic structural diagram of a system for evaluating the severity of ulcerative colitis based on deep learning according to this embodiment.
The structural composition of the deep learning-based ulcerative colitis severity evaluation system of this embodiment is described in detail below with reference to fig. 6:
as shown in fig. 6, the present embodiment provides a system for evaluating severity of ulcerative colitis based on deep learning, which includes:
(1) the sample set forming module is used for carrying out labeling Mayo endoscopic scoring and ulcerative colitis endoscopic activity index scoring on the white light colonoscope image to form a sample set; wherein, the activity index score under the ulcerative colitis endoscope is respectively and correspondingly marked according to three characteristics of UCEIS blood vessel typing, UCEIS spontaneous bleeding and UCEIS erosive ulcer.
In the implementation process, before labeling the white light colonoscope image, the method further comprises:
and reserving the region of interest for each white light colonoscope image by utilizing a de-blackening algorithm, and then performing data enhancement, random overturning and preset-size scaling on the white light colonoscope images.
As shown in FIG. 2, the score under Mayo endoscope is selected from 0-3 points and the corresponding score is labeled, 0 represents normal or inactive lesion; 1 indicates mild activity with edema, reduced vessel image and mild fragility; 2, moderate activity with marked edema, disappearing blood vessel image, mucosal fragility and erosion; 3 indicates that severe activity has continued bleeding and ulceration.
As shown in fig. 3, the endoscopic activity index score of ulcerative colitis is labeled by the following three characteristics:
blood vessel typing: 0 indicates a clearly visible vessel image, or a blurred and slightly missing edge vessel; 1 represents a blood vessel image with a sheet-like disappearance; 2, the vessel image disappears completely;
spontaneous hemorrhage: 0 indicates no visible bleeding; 1 indicates that bleeding from visible bleeding spots or banding coagulation of the mucosa can be washed away; 2 indicates mild active bleeding in the cavity; 3 represents severe active hemorrhage or persistent mucosal bleeding in the cavity;
and (3) erosive ulcer: 0 indicates normal mucosa, no obvious erosion and ulcer; 1 represents mucosal defect with erosion less than 5mm, white or yellowish with flat edge; 2 superficial ulcer with mucosa defect less than 5mm, with whitish fur; 3 indicates a deep ulcer with mucosal defect greater than 5mm, possibly with raised edges.
Fig. 4 shows white light colonoscope images corresponding to Mayo endoscopic score 1, blood vessel typing 2, spontaneous bleeding 1, and erosive ulcer 1.
The features related to Mayo ES scoring include but are not limited to colonic mucosal edema, erosion degree and fragility features of lesion areas.
It is understood that in other embodiments, the Mayo endoscopic scoring and the scoring of the characteristics of the blood vessel type, spontaneous bleeding and erosive ulcer can be specifically set by one skilled in the art according to actual circumstances.
(2) The evaluation model building and training module is used for building an ulcerative colitis severity evaluation model and training by using the marked white light colonoscope image samples in the sample set; the ulcerative colitis severity evaluation model is an SPPNet network with characteristic pyramid pooling, and outputs score prediction results which comprise 4 softmax functions and respectively correspond to the features of the single-frame image under Mayo endoscope scoring, blood vessel typing, spontaneous hemorrhage and erosive ulcer.
In order to accurately infer the UC image Mayo ES (Mayo endoscopic scoring) and UCEIS (ulcerative colitis endoscopic activity index scoring) at the same time, the embodiment of the invention innovatively uses a spatial pyramid network (SPPNet) and modifies a prediction layer of the network to support multi-label prediction.
SPPNet can be applied to arbitrary sizes, and in order to solve the training problem of different image sizes, two preset sizes are adopted: 180 x 180,224 x 224 (which have processed the image accordingly in steps 1 and 2). The aforementioned region image of 224 is changed to 180 size using scaling instead of cropping. Thus, the regions of different dimensions are simply different in resolution, not in content and layout, and the 180 network and 224 network have the same parameters. During training, sppnets of different input sizes are implemented by using two fixed-size networks that share parameters. To reduce the overhead of switching from one network (e.g., 224) to another network (e.g., 180), a complete epoch is trained on each network and then switched to the other network (weight preservation) at the next completed epoch. The main purpose of multi-scale training is to simulate different input sizes while ensuring that the fixed-scale network which is better optimized at present is fully utilized, so as to better adapt to various endoscopic image requirements. In addition to the two-scale implementation described above, different SXS inputs were also tested in each epoch, with S being chosen uniformly from 180 to 224.
ZFNET is adopted by the Backbone of the SPPNet, and for realizing Mayo ES and UCEIS scoring related feature identification of a UC single-frame image, feature pyramid pooling and multi-label prediction are integrated, and the model is improved as follows:
characteristic pyramid pooling: the pooling of the original SPPNet before the fully connected layer is replaced by feature pyramid pooling, and the structure of the feature pyramid is shown in fig. 5. The use of the characteristic pyramid can ensure that the image is input in any scale and finally converted into a uniform dimension input to the full connection layer. In addition, the characteristic pyramid can extract various characteristics of UC mucous membrane edema, blood vessel definition, spontaneous hemorrhage, erosion ulcer and the like on different dimensions, find small differences among images and promote the learning of UC lesion image characteristics.
Multi-label classification: the invention innovatively modifies the SPPNet network to support multi-label classification prediction. 4 labels of MayoES, blood vessel typing, spontaneous bleeding and erosive ulcer are defined. Wherein Mayo ES comprises 4 categories of 0-3, the blood vessel type comprises 3 categories of 0-2, and the spontaneous hemorrhage and anabrotic ulcer each comprise 4 categories of 0-3. And adjusting the number of the nodes output by the last full-connection layer of the ZFNET to be 4, and respectively scoring corresponding to the 4 labels. And 4 fully-connected nodes are respectively connected with 4 softmax functions finally, each softmax function corresponds to each label and is classified, 4 classifications {0,1,2,3} and corresponding probabilities thereof are output, and because the blood vessel typing only comprises 3 classifications, the output values {0,1} of the softmax corresponding to the blood vessel typing are mapped to {0}, {2} is mapped to {1}, and {3} is mapped to {2 }.
The SPPNet finally outputs a score prediction result containing 4 softmax functions, wherein the 4 functions respectively correspond to MayoES, blood vessel typing, spontaneous bleeding and erosive ulcer characteristics of a single-frame image. And respectively performing cross entropy loss on each softmax output result, wherein the final loss function is the average value of 4 cross entropy loss functions. And evaluating the error between the predicted value and the actual value by using a loss function, and guiding the adjustment of the model parameters. In this embodiment, the training network uses the SGD as the optimizer for training.
(3) And the severity evaluation output module is used for receiving the white light colonoscope image in real time, extracting the relevant scoring characteristics and the relevant blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics under the Mayo endoscope by using the ulcerative colitis severity evaluation model, outputting scoring prediction results of the relevant scoring, blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics under the Mayo endoscope, and accumulating the scoring prediction results of the blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics to obtain the activity index score under the ulcerative colitis endoscope.
Specifically, the white light colonoscope image received in real time is subjected to black edge removing processing, additional data enhancement and scaling processing are not needed to be performed on the image, the processed image is input into a trained ulcerative colitis severity evaluation model, and finally, 4 scoring categories output by softmax and corresponding probability values can be obtained, according to the training sequence, the first output category of softmax is the scoring result of Mayo ES of a single-frame image, the categories output by 2-4 softmax are blood vessel typing scoring, spontaneous hemorrhage scoring and erosive ulcer scoring, and the three characteristics are accumulated, namely UCEIS scoring of a single-frame image.
It is understood that in other embodiments, the weighting values of the blood vessel type, spontaneous bleeding and erosive ulcer characteristics can be specifically set by a person skilled in the art according to actual situations.
In another embodiment, the system for evaluating the severity of ulcerative colitis based on deep learning further comprises:
a most severe location determination module to: receiving a plurality of continuous frames of white light colonoscope images, and if the probabilities corresponding to the Mayo endoscopic score and the ulcerative colitis endoscopic activity index score are both greater than 0.5, and the Mayo endoscopic score and the ulcerative colitis endoscopic activity index score are both greater than or equal to corresponding preset values, and the continuous occurrence times are not lower than preset times, determining the positions corresponding to the white light colonoscope images as the most serious positions of the ulcerative colitis; the probability corresponding to the corresponding score is output by the softmax function, and the probability corresponding to the activity index score under the ulcerative colitis endoscope is the mean value of the probabilities corresponding to the relevant characteristics of blood vessel typing, spontaneous hemorrhage and erosive ulcer.
In the embodiment, the SPPNet network with the characteristic pyramid pooling is used for outputting the scoring prediction results of the features of scoring, blood vessel typing, spontaneous hemorrhage and erosive ulcer under the Mayo endoscope of a single-frame image, and the severity of ulcerative colitis is automatically scored and deduced and probability predicted.
The embodiment is based on the single-frame image scoring result, automatically compares and identifies the most serious lesion, and can perform indiscriminate and standard-identical continuous evaluation on the UC lesions of multiple intestine segments and diffuse type, so that the evaluation of the UC severity degree under the endoscope is more accurate and reliable.
The embodiment can be well suitable for different endoscope center computer system environments, is convenient for storing records and guides long-term standardized diagnosis and treatment of UC patients.
Example 3
The present embodiment provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the steps of the method for evaluating the severity of ulcerative colitis based on deep learning as shown in fig. 1.
In the embodiment, the SPPNet network with the characteristic pyramid pooling is used for outputting the scoring prediction results of the features of scoring, blood vessel typing, spontaneous hemorrhage and erosive ulcer under the Mayo endoscope of a single-frame image, and the severity of ulcerative colitis is automatically scored and deduced and probability predicted.
The embodiment is based on the single-frame image scoring result, automatically compares and identifies the most serious lesion, and can perform indiscriminate and standard-identical continuous evaluation on the UC lesions of multiple intestine segments and diffuse type, so that the evaluation of the UC severity degree under the endoscope is more accurate and reliable.
The embodiment can be well suitable for different endoscope center computer system environments, is convenient for storing records and guides long-term standardized diagnosis and treatment of UC patients.
Example 4
The embodiment provides a computer device, which comprises a memory, a processor and a computer program stored on the memory and running on the processor, wherein the processor executes the program to realize the steps of the method for evaluating the severity of ulcerative colitis based on deep learning shown in fig. 1.
In the embodiment, the SPPNet network with the characteristic pyramid pooling is used for outputting the scoring prediction results of the features of scoring, blood vessel typing, spontaneous hemorrhage and erosive ulcer under the Mayo endoscope of a single-frame image, and the severity of ulcerative colitis is automatically scored and deduced and probability predicted.
The embodiment is based on the single-frame image scoring result, automatically compares and identifies the most serious lesion, and can perform indiscriminate and standard-identical continuous evaluation on the UC lesions of multiple intestine segments and diffuse type, so that the evaluation of the UC severity degree under the endoscope is more accurate and reliable.
The embodiment can be well suitable for different endoscope center computer system environments, is convenient for storing records and guides long-term standardized diagnosis and treatment of UC patients.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (8)
1. A method for evaluating the severity of ulcerative colitis based on deep learning, comprising:
labeling a Mayo endoscopic score and an ulcerative colitis endoscopic activity index score on the white light colonoscope image to form a sample set; wherein, the activity index score under the ulcerative colitis endoscope is respectively and correspondingly marked with scores according to three characteristics of UCEIS blood vessel typing, UCEIS spontaneous bleeding and UCEIS erosive ulcer;
constructing an ulcerative colitis severity evaluation model, and training by using the marked white light colonoscope image samples in the sample set; the ulcerative colitis severity evaluation model is an SPPNet network with characteristic pyramid pooling, and outputs score prediction results which comprise 4 softmax functions and respectively correspond to the features of scoring, blood vessel typing, spontaneous bleeding and erosive ulcer under a Mayo endoscope of a single-frame image; outputting two clinical most common score prediction results of the Mayo endoscopic score and the ulcerative colitis endoscopic activity index of a single frame image, and performing automatic score deduction and probability prediction on the severity of the ulcerative colitis;
receiving a white light colonoscope image in real time, extracting relevant scoring characteristics under a Mayo endoscope and relevant blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics of an activity index under the ulcerative colitis endoscope by using an ulcerative colitis severity evaluation model, outputting scoring prediction results of the Mayo endoscope scoring, blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics, and accumulating the scoring prediction results of the blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics to obtain an activity index score under the ulcerative colitis endoscope;
the method for evaluating the severity of ulcerative colitis based on deep learning further comprises the following steps:
receiving a plurality of continuous frames of white light colonoscope images, and if the probabilities corresponding to the Mayo endoscopic score and the ulcerative colitis endoscopic activity index score are both greater than 0.5, and the Mayo endoscopic score and the ulcerative colitis endoscopic activity index score are both greater than or equal to corresponding preset values, and the continuous occurrence times are not lower than preset times, determining the positions corresponding to the white light colonoscope images as the most serious positions of the ulcerative colitis; the probability corresponding to the corresponding score is output by the softmax function, and the probability corresponding to the activity index score under the ulcerative colitis endoscope is the mean value of the probabilities corresponding to the relevant characteristics of blood vessel typing, spontaneous hemorrhage and erosive ulcer.
2. The method of claim 1, wherein the deep learning based assessment of the severity of ulcerative colitis is made by selecting a score correspondence score between 0 and 3 for a Mayo endoscopic score, 0 indicating a normal or inactive lesion; 1 indicates mild activity with edema, reduced vessel image and mild fragility; 2, moderate activity with marked edema, disappearing blood vessel image, mucosal fragility and erosion; 3 indicates severe activity with persistent bleeding and ulceration;
or the activity index score under the endoscope of the ulcerative colitis is respectively and correspondingly marked as the score according to the three characteristics of blood vessel typing, spontaneous hemorrhage and erosive ulcer:
blood vessel typing: 0 indicates a clearly visible vessel image, or a blurred and slightly missing edge vessel; 1 represents a blood vessel image with a sheet-like disappearance; 2, the vessel image disappears completely;
spontaneous hemorrhage: 0 indicates no visible bleeding; 1 indicates that bleeding from visible bleeding spots or banding coagulation of the mucosa can be washed away; 2 indicates mild active bleeding in the cavity; 3 represents severe active hemorrhage or persistent mucosal bleeding in the cavity;
and (3) erosive ulcer: 0 indicates normal mucosa, no obvious erosion and ulcer; 1 represents mucosal defect with erosion less than 5mm, white or yellowish with flat edge; 2 superficial ulcer with mucosa defect less than 5mm, with whitish fur; 3 indicates a deep ulcer with mucosal defect greater than 5mm, possibly with raised edges.
3. The deep learning-based ulcerative colitis severity assessment method of claim 1, wherein in training the ulcerative colitis severity assessment model, further comprising:
respectively performing cross entropy loss on each softmax function output result, and taking the average value of cross entropy losses corresponding to the four softmax functions to obtain a final loss function; when the final loss function reaches the minimum value, training to complete the ulcerative colitis severity assessment model;
or
Before labeling the white light colonoscope image, the method further comprises the following steps:
and reserving the region of interest for each white light colonoscope image by utilizing a de-blackening algorithm, and then performing data enhancement, random overturning and preset-size scaling on the white light colonoscope images.
4. A deep learning based ulcerative colitis severity assessment system comprising:
the sample set forming module is used for carrying out labeling Mayo endoscopic scoring and ulcerative colitis endoscopic activity index scoring on the white light colonoscope image to form a sample set; wherein, the activity index score under the ulcerative colitis endoscope is respectively and correspondingly marked with scores according to three characteristics of UCEIS blood vessel typing, UCEIS spontaneous bleeding and UCEIS erosive ulcer;
the evaluation model building and training module is used for building an ulcerative colitis severity evaluation model and training by using the marked white light colonoscope image samples in the sample set; the ulcerative colitis severity evaluation model is an SPPNet network with characteristic pyramid pooling, and outputs score prediction results which comprise 4 softmax functions and respectively correspond to the features of scoring, blood vessel typing, spontaneous bleeding and erosive ulcer under a Mayo endoscope of a single-frame image; outputting two clinical most common score prediction results of the Mayo endoscopic score and the ulcerative colitis endoscopic activity index of a single frame image, and performing automatic score deduction and probability prediction on the severity of the ulcerative colitis;
the severity evaluation output module is used for receiving the white-light colonoscope image in real time, extracting relevant scoring characteristics under the Mayo endoscope and relevant blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics of the activity index under the ulcerative colitis endoscope by using the ulcerative colitis severity evaluation model, outputting scoring prediction results of the Mayo endoscope scoring, blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics, and accumulating the scoring prediction results of the blood vessel typing, spontaneous hemorrhage and erosive ulcer characteristics to obtain the activity index score under the ulcerative colitis endoscope;
the system for evaluating the severity of ulcerative colitis based on deep learning further comprises:
a most severe location determination module to: receiving a plurality of continuous frames of white light colonoscope images, and if the probabilities corresponding to the Mayo endoscopic score and the ulcerative colitis endoscopic activity index score are both greater than 0.5, and the Mayo endoscopic score and the ulcerative colitis endoscopic activity index score are both greater than or equal to corresponding preset values, and the continuous occurrence times are not lower than preset times, determining the positions corresponding to the white light colonoscope images as the most serious positions of the ulcerative colitis; the probability corresponding to the corresponding score is output by the softmax function, and the probability corresponding to the activity index score under the ulcerative colitis endoscope is the mean value of the probabilities corresponding to the relevant characteristics of blood vessel typing, spontaneous hemorrhage and erosive ulcer.
5. The deep learning-based ulcerative colitis severity assessment system of claim 4, wherein a Mayo intra-endoscopic score selects a corresponding score label between 0 and 3, 0 indicating normal or inactive lesions; 1 indicates mild activity with edema, reduced vessel image and mild fragility; 2, moderate activity with marked edema, disappearing blood vessel image, mucosal fragility and erosion; 3 indicates severe activity with persistent bleeding and ulceration;
or the activity index score under the endoscope of the ulcerative colitis is respectively and correspondingly marked as the score according to the three characteristics of blood vessel typing, spontaneous hemorrhage and erosive ulcer:
blood vessel typing: 0 indicates a clearly visible vessel image, or a blurred and slightly missing edge vessel; 1 represents a blood vessel image with a sheet-like disappearance; 2, the vessel image disappears completely;
spontaneous hemorrhage: 0 indicates no visible bleeding; 1 indicates that bleeding from visible bleeding spots or banding coagulation of the mucosa can be washed away; 2 indicates mild active bleeding in the cavity; 3 represents severe active hemorrhage or persistent mucosal bleeding in the cavity;
and (3) erosive ulcer: 0 indicates normal mucosa, no obvious erosion and ulcer; 1 represents mucosal defect with erosion less than 5mm, white or yellowish with flat edge; 2 superficial ulcer with mucosa defect less than 5mm, with whitish fur; 3 indicates a deep ulcer with mucosal defect greater than 5mm, possibly with raised edges.
6. The deep learning-based ulcerative colitis severity assessment system according to claim 4, wherein in the assessment model construction and training module, in the process of training the ulcerative colitis severity assessment model, further comprising:
respectively performing cross entropy loss on each softmax function output result, and taking the average value of cross entropy losses corresponding to the four softmax functions to obtain a final loss function; when the final loss function reaches a minimum value, training is completed to evaluate the severity of ulcerative colitis,
or in the sample set forming module, before labeling the white light colonoscope image, further comprising:
and reserving the region of interest for each white light colonoscope image by utilizing a de-blackening algorithm, and then performing data enhancement, random overturning and preset-size scaling on the white light colonoscope images.
7. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method for deep learning-based ulcerative colitis severity assessment according to any one of claims 1-3.
8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program performs the steps in the method for deep learning based ulcerative colitis severity assessment according to any one of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911312322.9A CN110993099B (en) | 2019-12-18 | 2019-12-18 | Ulcerative colitis severity evaluation method and system based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911312322.9A CN110993099B (en) | 2019-12-18 | 2019-12-18 | Ulcerative colitis severity evaluation method and system based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110993099A CN110993099A (en) | 2020-04-10 |
| CN110993099B true CN110993099B (en) | 2020-11-03 |
Family
ID=70095735
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911312322.9A Active CN110993099B (en) | 2019-12-18 | 2019-12-18 | Ulcerative colitis severity evaluation method and system based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110993099B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113192634B (en) * | 2021-05-25 | 2023-04-07 | 中原工学院 | Automatic evaluation system for liver mechanical source injury |
| CN114445406B (en) * | 2022-04-07 | 2022-08-09 | 武汉大学 | Enteroscopy image analysis method and device and medical image processing equipment |
| CN115132355B (en) * | 2022-07-13 | 2024-05-03 | 山东大学齐鲁医院 | Intelligent data auxiliary diagnosis system for inflammatory bowel disease |
| CN115497630B (en) * | 2022-08-24 | 2023-11-03 | 中国医学科学院北京协和医院 | Method and system for processing acute severe ulcerative colitis data |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107341798A (en) * | 2017-07-06 | 2017-11-10 | 西安电子科技大学 | High Resolution SAR image change detection method based on global local SPP Net |
| CN108648825A (en) * | 2018-05-30 | 2018-10-12 | 江苏大学附属医院 | A kind of leucoderma hickie appraisal procedure based on image recognition |
| CN108960265A (en) * | 2017-05-22 | 2018-12-07 | 阿里巴巴集团控股有限公司 | Optimization method, image classification method, the apparatus and system of image classification process |
| CN109447987A (en) * | 2018-11-28 | 2019-03-08 | 武汉大学人民医院（湖北省人民医院） | Ulcerative colitis assistant diagnosis system and method under colonoscopy based on deep learning |
-
2019
- 2019-12-18 CN CN201911312322.9A patent/CN110993099B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108960265A (en) * | 2017-05-22 | 2018-12-07 | 阿里巴巴集团控股有限公司 | Optimization method, image classification method, the apparatus and system of image classification process |
| CN107341798A (en) * | 2017-07-06 | 2017-11-10 | 西安电子科技大学 | High Resolution SAR image change detection method based on global local SPP Net |
| CN108648825A (en) * | 2018-05-30 | 2018-10-12 | 江苏大学附属医院 | A kind of leucoderma hickie appraisal procedure based on image recognition |
| CN109447987A (en) * | 2018-11-28 | 2019-03-08 | 武汉大学人民医院（湖北省人民医院） | Ulcerative colitis assistant diagnosis system and method under colonoscopy based on deep learning |
Non-Patent Citations (3)
| Title |
|---|
| Novel computer-assisted diagnosis system for endoscopic disease activity in patients with ulcerative colitis;Tsuyoshi Ozawa;《A Gastrointestinal Endoscopy》;20190228;第89卷(第2期);正文第416-421页 * |
| Recent advances in the endoscopic assessment of ulcerative colitis;Marietta Iacucci;《Techniques in Gastrointestinal Endoscopy》;20161231;正文第116-122页 * |
| 粪便钙卫蛋白与溃疡性结肠炎肠道病变程度及范围的相关性;杨荣萍;《广东医学》;20170131;第38卷(第2期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110993099A (en) | 2020-04-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110993099B (en) | Ulcerative colitis severity evaluation method and system based on deep learning | |
| US20190311479A1 (en) | Method and device for identifying pathological picture | |
| CN108268870B (en) | Multi-scale feature fusion ultrasonic image semantic segmentation method based on counterstudy | |
| CN107945173B (en) | Skin disease detection method and system based on deep learning | |
| CN111242242B (en) | Cervical tissue pathology whole-slide image automatic classification method based on confidence degree selection | |
| TWI696145B (en) | Colonoscopy image computer-aided recognition system and method | |
| Liu et al. | A framework of wound segmentation based on deep convolutional networks | |
| CN110689089A (en) | Active incremental training method for deep learning of multi-class medical image classification | |
| CN111127426B (en) | Gastric mucosa cleanliness evaluation method and system based on deep learning | |
| CN113658179B (en) | Method and device for detecting cleanliness of intestinal tract | |
| CN113724842B (en) | Cervical tissue pathology auxiliary diagnosis method based on attention mechanism | |
| CN112001921B (en) | New coronary pneumonia CT image focus segmentation image processing method based on focus weighting loss function | |
| CN114140651A (en) | Stomach focus recognition model training method and stomach focus recognition method | |
| Salsabili et al. | Automated segmentation of villi in histopathology images of placenta | |
| CN110974179A (en) | Auxiliary diagnosis system for stomach precancer under electronic staining endoscope based on deep learning | |
| CN112819768A (en) | DCNN-based cancer full-field digital pathological section survival analysis method | |
| CN113610118A (en) | Fundus image classification method, device, equipment and medium based on multitask course learning | |
| CN114445406B (en) | Enteroscopy image analysis method and device and medical image processing equipment | |
| CN113554668B (en) | Skin mirror image melanoma segmentation method, device and related components | |
| Tsai et al. | Deep learning techniques for colorectal cancer tissue classification | |
| CN117036288A (en) | Tumor subtype diagnosis method for full-slice pathological image | |
| KR102304609B1 (en) | Method for refining tissue specimen image, and computing system performing the same | |
| CN115587967A (en) | Fundus image optic disk detection method based on HA-UNet network | |
| CN114078137A (en) | Colposcope image screening method and device based on deep learning and electronic equipment | |
| CN110992309A (en) | Fundus image segmentation method based on deep information transfer network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CB03 | Change of inventor or designer information |
Inventor after: Zuo Xiuli Inventor after: Ji Chaoran Inventor after: Feng Jian Inventor after: Li Yanqing Inventor after: Li Zhen Inventor after: Shao Xuejun Inventor after: Yang Xiaoyun Inventor after: Xin Wei Inventor before: Zuo Xiuli Inventor before: Ji Chaoran Inventor before: Feng Jian Inventor before: Li Yanqing Inventor before: Li Zhen Inventor before: Shao Xuejun Inventor before: Yang Xiaoyun Inventor before: Xin Wei |
|
| CB03 | Change of inventor or designer information |