CN110427994A - Digestive endoscope image processing method, device, storage medium, equipment and system - Google Patents

Digestive endoscope image processing method, device, storage medium, equipment and system Download PDF

Info

Publication number
CN110427994A
CN110427994A CN201910670467.XA CN201910670467A CN110427994A CN 110427994 A CN110427994 A CN 110427994A CN 201910670467 A CN201910670467 A CN 201910670467A CN 110427994 A CN110427994 A CN 110427994A
Authority
CN
China
Prior art keywords
model
training
data collection
sample image
mark
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910670467.XA
Other languages
Chinese (zh)
Inventor
王晓宁
孙钟前
付星辉
尚鸿
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Healthcare Shenzhen Co Ltd
Original Assignee
Tencent Healthcare Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Healthcare Shenzhen Co Ltd filed Critical Tencent Healthcare Shenzhen Co Ltd
Priority to CN201910670467.XA priority Critical patent/CN110427994A/en
Publication of CN110427994A publication Critical patent/CN110427994A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/19—Recognition using electronic means
    • G06V30/192—Recognition using electronic means using simultaneous comparisons or correlations of the image signals with a plurality of references
    • G06V30/194—References adjustable by an adaptive method, e.g. learning
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30028—Colon; Small intestine
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Databases & Information Systems (AREA)
  • Public Health (AREA)
  • Evolutionary Biology (AREA)
  • Epidemiology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Primary Health Care (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Pathology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Quality & Reliability (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Multimedia (AREA)
  • Endoscopes (AREA)
  • Image Analysis (AREA)

Abstract

This application discloses a kind of digestive endoscope image processing method, device, storage medium, equipment and systems, belong to field of artificial intelligence, are related to computer vision technique and machine learning techniques.It include: to obtain digestive endoscope image to be detected；Classified based on the first model to digestive endoscope image to be detected, first model is to be obtained under the constraint of the second model based on the training of the first training dataset, first training dataset includes clean data collection and noise data collection, and the second model is to be obtained before the first model of training based on the training of the second training dataset；Clean data collection includes marking consistent sample image, and noise data collection includes the inconsistent sample image of mark, and the second training dataset is the subset of the first training dataset and including clean data collection, and sample image is digestive endoscope image.The application realizes while increasing amount of training data, and can reduce the influence to malfunction by mark label to model prediction precision.

Description

Digestive endoscope image processing method, device, storage medium, equipment and system
Technical field
This application involves field of artificial intelligence, in particular to a kind of digestive endoscope image processing method, is deposited device Storage media, equipment and system.
Background technique
The core of computer vision technique and machine learning techniques as artificial intelligence, application range have been spread at present Every field, such as medical field are one of.In the medical field, computer vision technique and machine learning skill are utilized Art is handled medical imaging, it can be achieved that identifying to whether patient suffers from certain disease.For example, passing through machine learning mould Type assists doctor to carry out disease of digestive tract detection.
The relevant technologies are when carrying out disease of digestive tract detection by digestive endoscope image procossing, it is common practice to: it obtains A training dataset is taken, it includes the digestive endoscope image (also referred to as sample image) marked which, which concentrates,；Root Some deep neural network is trained according to the training dataset, obtains a machine learning model；Later, disappear to be detected Change road endoscopic image to input in the machine learning model, the prediction result of machine learning model output can be obtained.
For above-mentioned digestive endoscope image procossing mode, it is normally based on such a in training process it is assumed that marking Note personnel are entirely correct to the mark label of sample image.However, digestive endoscope image is showed in some cases Lesion characteristics out, even some doctors by professional training, it is also possible to more difficult its lesion nature of resolution, i.e. mark label Have certain error probability, for this kind of situation, based on above-mentioned training method train come machine learning model precision it is non- It is often limited, the detection precision in subsequent image detection process can be seriously affected.
Summary of the invention
The embodiment of the present application provides a kind of digestive endoscope image processing method, device, storage medium, equipment and is System, it is poor to solve model accuracy existing for the relevant technologies, and then causes the detection precision in image-detection process poor Problem.The technical solution is as follows:
On the one hand, a kind of digestive endoscope image processing method is provided, which comprises
Obtain digestive endoscope image to be detected；
Classified based on the first model to the digestive endoscope image to be detected, first model is in the second mould It is obtained under the constraint of type based on the training of the first training dataset, first training dataset includes clean data collection and noise Data set, second model are to be obtained before training first model based on the training of the second training dataset；
Wherein, the clean data collection includes marking consistent sample image, and the noise data collection includes that mark is different The sample image of cause, second training dataset be first training dataset subset and including the clean data Collection, the sample image are digestive endoscope image.
On the other hand, a kind of digestive endoscope image processing apparatus is provided, described device includes:
Module is obtained, for obtaining digestive endoscope image to be detected；
Processing module, for being classified based on the first model to the digestive endoscope image to be detected, described first Model is to be obtained under the constraint of the second model based on the training of the first training dataset, and first training dataset includes pure Net data set and noise data collection, second model are to be assembled for training before training first model based on the second training data It gets；
Wherein, the clean data collection includes marking consistent sample image, and the noise data collection includes that mark is different The sample image of cause, second training dataset be first training dataset subset and including the clean data Collection, the sample image are digestive endoscope image.
In one possible implementation, for the total number that the noise data integrates as n-1, n is just whole not less than 2 Number；
Wherein, it includes that n-1 mark personnel mark consistent sample image that the (n-1)th noise data, which is concentrated,；N-th -2 noise number It include that n-2 mark personnel mark consistent sample image according to concentrating；And so on, it includes n mark that the first noise data, which is concentrated, Note personnel mark inconsistent sample image.
In one possible implementation, the training module is also used to obtain the mark personnel to the sample The mark label of image；
Second model is obtained to the prediction label of the sample image；
The prediction label of mark label and second model output based on the sample image, generates the sample graph The physical tags of picture.
In one possible implementation, using following formula, the physical tags of the sample image are generated:
Wherein,Refer to the physical tags of the sample image；λ refers to adjustable coefficient, and value is a constant；y Refer to the mark label of the sample image；S refers to model obtained in previous step training process to the pre- of the sample image Mark label.
In one possible implementation, the calculation formula of the loss function are as follows:
L(yi,f(xi))=l (λ yi+(1-λ)si,f(xi))
Wherein, λ refers to adjustable coefficient, and value is a constant；The value of i is positive integer, xiRefer to i-th of sample Image, yiIt is the mark personnel to sample image xiMark label, f (xi) currently trained model is referred to sample image xiPrediction label, siModel obtained in previous step training process is referred to sample image xiPrediction label.
On the other hand, provide a kind of storage medium, be stored at least one instruction in the storage medium, it is described at least One instruction is loaded by processor and is executed to realize above-mentioned digestive endoscope image processing method.
On the other hand, a kind of image processing equipment is provided, the equipment includes processor and memory, the memory In be stored at least one instruction, at least one instruction is loaded by the processor and is executed to realize above-mentioned alimentary canal Endoscopic image processing method.
On the other hand, a kind of image processing system is provided, the system comprises: model training equipment and image procossing are set Standby, described image processing equipment includes display screen；
The model training equipment, for being based on the first training dataset the first mould of training under the constraint of the second model Type, first training dataset include clean data collection and noise data collection, and second model is in training described first It is obtained before model based on the training of the second training dataset；Wherein, the clean data collection includes marking consistent sample graph Picture, the noise data collection include the inconsistent sample image of mark, and second training dataset is the first training number According to the subset of collection and including the clean data collection；
Described image processing equipment includes processor and memory, and at least one instruction, institute are stored in the memory It states at least one instruction to be loaded by the processor and executed to realize: digestive endoscope image to be detected is obtained, based on described First model classifies to the digestive endoscope image to be detected；
The display screen is for showing the prediction classification results of output.
Technical solution provided by the embodiments of the present application has the benefit that
Training dataset has been divided into clean data collection and noise data collection by the embodiment of the present application, wherein clean data collection In include marking consistent sample image, it includes marking inconsistent sample image that noise data, which is concentrated, wherein sample image is equal For the digestive endoscope image marked, inconsistent data adverse effect caused by model training is marked in order to evade, it is real Now effective use marks inconsistent data, includes multistep model training during model training, such as to be detected The first model that digestive endoscope image is classified is trained under the constraint of the second model obtains, wherein the second mould The training process of type is before the first model of training namely model training is a knowledge distillation process, and training obtains before Model can instruct current model training, wherein and it includes clean data collection that the training data that each step training uses, which is concentrated, and The data set that training uses before is the subset for the data set that current training uses, it ensures that knowledge distillation process is one Step up the process of the prediction precision of model.
A kind of expression way is changed, the embodiment of the present application is other than it can utilize the consistent data of mark, additionally it is possible to effectively It using inconsistent data are marked, realizes while increasing the data volume of training dataset, and can reduce because mark is marked The wrong and influence to model prediction precision is checked out, so as to have on the basis of effective use marks inconsistent data Effect improves the predictablity rate of model.The precision for training the machine learning model come namely based on above-mentioned training method is preferable, In After getting digestive endoscope image to be detected, the first model can be directly based upon, digestive endoscope image to be detected is divided Class, it is ensured that the detection precision in image-detection process.
Detailed description of the invention
In order to more clearly explain the technical solutions in the embodiments of the present application, make required in being described below to embodiment Attached drawing is briefly described, it should be apparent that, the drawings in the following description are only some examples of the present application, for For those of ordinary skill in the art, without creative efforts, it can also be obtained according to these attached drawings other Attached drawing.
Fig. 1 is a kind of signal of implementation environment that digestive endoscope image processing method is related to provided by the embodiments of the present application Figure；
Fig. 2 is a kind of schematic diagram of digestive endoscope image provided by the embodiments of the present application；
Fig. 3 is a kind of flow chart of digestive endoscope image processing method provided by the embodiments of the present application；
Fig. 4 is the schematic diagram of knowledge distillation frame during a kind of model training provided by the embodiments of the present application；
Fig. 5 is a kind of flow chart of digestive endoscope image processing method provided by the embodiments of the present application；
Fig. 6 is a kind of structural schematic diagram of digestive endoscope image processing apparatus provided by the embodiments of the present application；
Fig. 7 is a kind of structural schematic diagram of digestive endoscope image processing apparatus provided by the embodiments of the present application；
Fig. 8 is a kind of structural schematic diagram of model training equipment provided by the embodiments of the present application；
Fig. 9 is a kind of structural schematic diagram of image processing equipment provided by the embodiments of the present application.
Specific embodiment
To keep the purposes, technical schemes and advantages of the application clearer, below in conjunction with attached drawing to the application embodiment party Formula is described in further detail.
With the research and progress of artificial intelligence technology, research and application, example is unfolded in multiple fields in artificial intelligence technology Such as common smart home, intelligent wearable device, virtual assistant, intelligent sound box, intelligent marketing, unmanned, automatic Pilot, nothing Man-machine, robot, intelligent medical, intelligent customer service etc., it is believed that with the development of technology, artificial intelligence technology will be in more fields It is applied, and plays more and more important value.
Image procossing scheme provided by the embodiments of the present application is applied to intelligent medical field, can be related to the meter of artificial intelligence Calculation machine vision technique and machine learning techniques etc..
Wherein, computer vision technique is a research refers to the science of machine " seeing " further just It the machine vision such as replaces human eye to be identified, tracked to target with video camera and computer and is measured, and further do graphics process, Computer is set to be treated as the image for being more suitable for eye-observation or sending instrument detection to.As a branch of science, computer view Feel and study relevant theory and technology, it is intended to establish the artificial intelligence system that can obtain information from image or multidimensional data System.Computer vision technique generally includes image procossing, image recognition, image, semantic understanding, image retrieval, OCR (Optical Character Recognition, optical character identification), video processing, video semanteme understanding, video content/Activity recognition, Three-dimension object reconstruction, 3D (3 Dimensions, three-dimensional) technology, virtual reality, augmented reality, synchronous superposition etc. Technology further includes the biometrics identification technologies such as common recognition of face, fingerprint recognition.
And machine learning is a multi-field cross discipline, and it is multiple to be related to probability theory, statistics, Approximation Theory, convextiry analysis, algorithm The multiple subjects such as redundancy theory.The learning behavior that the mankind were simulated or realized to computer how is specialized in, to obtain new knowledge Or technical ability, it reorganizes the existing structure of knowledge and is allowed to constantly improve the performance of itself.Machine learning is the core of artificial intelligence, It is the fundamental way for making computer that there is intelligence, application spreads the every field of artificial intelligence.Machine learning and deep learning Generally include the technologies such as artificial neural network, confidence network, intensified learning, transfer learning, inductive learning, formula teaching habit.
A kind of digestive endoscope image processing method provided by the present application is illustrated especially by following examples:
Before to the embodiment of the present application carrying out that explanation is explained in detail, first to the invention relates to some names arrived Word is explained.
Digestive endoscope image: digestive endoscope is also referred to as digestive endoscopy.
In one possible implementation, by checking scope attributive classification used, digestive endoscope can be divided into oesophagoscope, Pharyngoscope, gastroscope, duodenoscope, Colon and rectum mirror, ductus pancreaticus mirror etc.；Check point and function classification are pressed, digestive endoscope can divide For superior gastrointestinal endoscope, lower digestive tract scope etc.；Classify by clinical application, digestive endoscope can be divided into diagnostic digestive endoscopy and Therapeutic digestive endoscopy etc..
Wherein, digestive endoscope image refers to medical image relevant to the alimentary canal of human body.As an example, it digests Road endoscopic image, which can be to be insinuated into, carries out the medical imaging that image taking obtains inside alimentary canal, the embodiment of the present application to this not Specifically limited.Correspondingly, digestive endoscope image include but is not limited to oesophagoscope image, pharyngoscope image, gastroscope image, Duodenoscope image, Colon and rectum mirror image and ductus pancreaticus mirror etc..
Mark is consistent: referring to for same digestive endoscope image, the complete phase of label of different professional mark personnel Together.
Wherein, professional mark personnel herein generally refer to the doctor in alimentary canal field.
It marks inconsistent: referring to for same digestive endoscope image, the label of different mark personnel's marks is different.
Wherein, for same digestive endoscope image, marking inconsistent both may include that each mark personnel mark The inconsistent situation of label, may also be included in which part mark personnel mark label it is identical, but with other some marks The different situation of the label of note personnel mark, the embodiment of the present application is to this without specifically limiting.
As an example, it is assumed that one, which shares 3 mark personnel, is labeled same digestive endoscope image, then marks Infusing inconsistent may include the following two kinds situation: 2 mark personnel marks are consistent, but different with third mark personnel mark It causes；3 mark personnel mark inconsistent.
In addition, it is necessary to explanation, mark is inconsistent, and there may be multiple grades.
As an example, by taking one about human colorectal digestive endoscope image as an example, doctor A may mark the image There are adenomatous polyp, doctor B may mark the image there are gland cancer, and doctor C may mark the image, and there are non-adenomatous breaths The mark conclusion that meat, i.e. 3 doctors provide is inconsistent；It is deposited it is of course also possible to will appear doctor A and doctor B and mark the image In adenomatous polyp, and doctor C marks image the case where there are gland cancer, i.e. the mark conclusion of 2 people is consistent.
Clean data set: whole mark personnel are referred to and mark the data acquisition system that consistent sample image is constituted.
It should be noted that herein presented sample image all refers to digestive endoscope image.
Wherein, in statistical significance, which has higher accuracy rate.In the embodiment of the present application, clean data Collection is also referred to as clean data collection, with symbol Dc reference.
Assuming that 3 doctors carry out the mark of same digestive endoscope image, then clean data set refers to this 3 doctor's marks Consistent data acquisition system is marked in the digestive endoscope image infused.
Noise data collection: mark personnel are referred to and mark the data acquisition system that inconsistent sample image is constituted.
In one possible implementation, it is assumed that there are n professional mark personnel to be labeled sample image, then at this For the total number that noise data integrates in application embodiment as n-1, n is the positive integer not less than 2.
It wherein, include that n-1 mark personnel mark consistent sample image in noise data collection Dn-1；Noise data collection It include that n-2 mark personnel mark consistent sample image in Dn-2；And so on, it include n mark in noise data collection D1 Personnel mark inconsistent sample image.
Assuming that 3 doctors carry out the mark of same digestive endoscope image, then noise data collection D2 indicates 2 doctor's marks The data acquisition system of consistent digestive endoscope image construction is infused, D1 indicates that 3 doctors mark inconsistent digestive endoscope figure As the data acquisition system constituted.
Deep neural network: being based on deep learning, is derived from neural network, wherein and deep learning is the subclass of machine learning, Deep neural network from literal upper understanding be profound neural network.As an example, by simple monolayer neuronal net Hidden layer in network expands when carrying out multilayer, has just obtained deep neural network.
In one possible implementation, the deep neural network that the embodiment of the present application uses includes but is not limited to DenseNet (Densely Connected Convolutional Networks, intensively connect convolutional network), VGG (Visual Geometry Group Network, visual geometric group) network, the embodiment of the present application is to this without specifically limiting It is fixed.
At present in high-incidence malignant tumour, the disease incidence of the alimentary tract cancers such as cancer of the esophagus, colon cancer and gastric cancer It is in continue ascendant trend with the death rate.If can find and be treated in time in lesion early stage, most morning cancer patient It can be cured completely.Therefore, it is particularly significant to carry out cancer early screening.
In recent years, with the development of machine learning techniques and computer vision technique, medical domain also achieves great prominent Broken, i.e., the artificial intelligence based on machine learning techniques and computer vision technique, which can assist a physician, carries out disease detection.Wherein, Digestive endoscope image is one of the optional condition that artificial intelligence auxiliary doctor carries out disease detection.That is, utilizing digestive endoscope Image can assist doctor to carry out disease detection in conjunction with artificial intelligence.It as an example, can using digestive endoscope image To combine artificial intelligence to carry out disease of digestive tract detection.
Wherein, disease detection is carried out based on machine learning techniques and computer vision technique auxiliary, also needed using training number Machine learning model is trained according to collecting.And before carrying out model training, first also existing disappear need to be marked by professional mark personnel Change road endoscopic image, as training dataset.
It should be noted that be different from the object in daily life on common picture, digestive endoscope in some cases The lesion characteristics that image is showed, even some doctors by professional training, it is also possible to more difficult its lesion nature of resolution, because This annotation process has certain error probability, i.e., no due to the respective professional knowledge of different doctors, working experience, working condition etc. Together, it is possible that inconsistent mark conclusion can be provided.However, the mark accuracy rate to training dataset is most important, because It determine based on the training dataset train come the detection effect that can reach of model.
Based on this, in annotation process, for same digestive endoscope image, it will usually there is multiple professional mark personnel It is labeled.Wherein, multiple professional mark personnel provide the data of consistency mark conclusion, have in statistical significance higher Accuracy rate.But consistent data are marked if be used only, and abandon and mark inconsistent data, then it will be greatly reduced training The data volume for including in data set, results in waste of resources, and the modelling effect finally trained may also be poor.And for marking not Consistent data are marked if being only simply to randomly select the annotation results of some professional mark personnel as physical tags A possibility that checking out mistake is higher, very likely can be right if with the consistent data of mark together as training dataset Model training causes to mislead.
In view of the above problems, digestive endoscope image processing method provided by the embodiments of the present application can be effectively sharp Inconsistent data are marked with this part, can be realized while increasing sample data volume, are reduced due to label error to mould It is influenced caused by type, to improve the predictablity rate for training the machine learning model come.
Below to a kind of implementation environment that digestive endoscope image processing method is related to progress provided by the embodiments of the present application It introduces.
It include model training equipment 101 and image processing equipment 102 in the implementation environment referring to Fig. 1.
Wherein, model training equipment 101 is used to be based on model training for carrying out model training, image processing equipment 102 The trained machine learning model of equipment 101 carries out image procossing, i.e., is completed based on the machine learning model to digestion to be detected Road endoscopic image is classified, for example is identified in digestive endoscope image to be detected with the presence or absence of disease and kinds of Diseases.
In one possible implementation, image processing equipment 102 generally includes display screen, for showing engineering The prediction classification results of model output are practised, and clinician is prompted.
In the embodiment of the present application, above-mentioned model training equipment 101 and image processing equipment 102 constitute image procossing system System.
As an example, image processing equipment 102 includes display screen；Wherein, model training equipment 101 is used for Under the constraint of second model, based on the first training dataset the first model of training, the first training dataset includes clean data collection With noise data collection, the second model is to be obtained before the first model of training based on the training of the second training dataset；Wherein, pure Net data set includes marking consistent sample image, and noise data collection includes the inconsistent sample image of mark, the second training number According to the subset that collection is the first training dataset and including clean data collection.
In one possible implementation, for currently training the first model, then the second model is in training first It is obtained, is specifically referred to, the second model is in current training process based on the training of the second training dataset before model It is obtained in previous step training process based on the training of the second training dataset.
Image processing equipment 102 includes processor and memory, is stored at least one instruction in memory, and at least one Instruction is loaded by processor and is executed to realize: being obtained digestive endoscope image to be detected, is disappeared based on the first model to be detected Change road endoscopic image to classify；Display screen is for showing the prediction classification results of output.
In addition, can also include that display is set in the implementation environment if image processing equipment 102 does not include display screen Standby, image processing equipment 102 is responsible for control display equipment output prediction classification results and is prompted clinician.Its In, display equipment is usually display.
As an example, the mode prompted includes but is not limited to: voice prompting, display equipment or display screen The special prompt of warning of upper indicator light is highlighted the focal area detected etc. in the picture of display, the embodiment of the present application To this without specifically limiting.
Wherein, 101 image processing equipment 102 of model training equipment is the computer equipment with computing capability, wherein The type of model training equipment 101 includes but is not limited to the fixed apparatus such as desktop computer or server shown in FIG. 1, intelligent hand The mobile units such as machine or tablet computer, the type of image processing equipment 102 include but is not limited to that mobile medical terminal etc. is moved Dynamic formula equipment, the embodiment of the present application is to this without specifically limiting.
Wherein, the embodiment of the present application can mark inconsistent data using professional mark personnel and professional mark personnel mark Infuse consistent data, the prediction precision of Lai Tigao model.For example, being directed to medical field, use is provided by the embodiments of the present application Model training mode enables to the machine learning model trained that doctor is preferably assisted to carry out disease detection.
Change a kind of expression way, the embodiment of the present application during model training, in addition to using mark consistent data it Outside, additionally it is possible to which effective use marks inconsistent data, realizes while increasing the data volume of training dataset, and can The influence due to error of the label of mark to model prediction precision is reduced, so as to mark inconsistent number in effective use On the basis of, the predictablity rate of model is effectively improved.
As an example, the consistent data of above-mentioned mark are also referred to as clean data set in the embodiment of the present application, on It states the inconsistent data of mark and is also referred to as noise data collection in the embodiment of the present application.
In one possible implementation, the disease of digestive tract detection based on digestive endoscope image includes but unlimited In: the cancer of the esophagus, throat, gastric cancer, dudenal disease, colorectal diseases etc..Wherein, Fig. 2 shows gone out in partial digestive tract The example of mirror image.
Below a kind of digestive endoscope image processing method provided by the embodiments of the present application is carried out that explanation is explained in detail.
In addition, the first, second, third, fourth equal description hereinafter occurred, is only for distinguishing different pairs As without constituting any other restriction.
Fig. 3 is a kind of flow chart of digestive endoscope image processing method provided by the embodiments of the present application.This method is held Row main body is model training equipment and image processing equipment shown in Fig. 1, referring to Fig. 3, method provided by the embodiments of the present application Process includes:
Model training stage
In one possible implementation, for model training process, the embodiment of the present application devises such as Fig. 4 institute The knowledge distillation frame shown completes machine learning model training according to knowledge distillation frame.Wherein, the instruction of machine learning model Practicing sample image used in process is digestive endoscope image.Below with reference to knowledge shown in Fig. 4 distillation frame to model Explanation is introduced in training process.
301, model training equipment is based on clean data collection and carries out model training, obtains submodel.
As it was noted above, it includes that different labeled personnel mark consistent sample image that clean data, which is concentrated,.
As an example, it is assumed that there are n mark personnel to carry out the mark of sample image, then clean data integrates marks as n Note personnel provide the set of the sample image of consistency annotation results.
Referring to fig. 4, due in statistical significance clean data collection there is higher accuracy rate, the embodiment of the present application is first Model training is carried out first with the high clean data collection of mark label accuracy rate.That is, the embodiment of the present application is first with pure number Deep neural network is trained according to collection Dc, obtains submodel fDc。
In one possible implementation, deep neural network includes but is not limited to DenseNet, VGG etc., the application Embodiment is to this without specifically limiting.
302, model training equipment is under the constraint of submodel, based on clean data collection and the (n-1)th noise data collection into Row model training obtains the (n-1)th model.
As it was noted above, it includes marking inconsistent sample image that noise data, which is concentrated, mark in the embodiment of the present application The number of personnel is n, wherein n is the positive integer not less than 2, then the total number that noise data integrates is n-1.
In the embodiment of the present application, noise data collection Dn-1In include the consistent sample image of n-1 mark personnel mark, Wherein, noise data collection Dn-1Also referred to as the (n-1)th noise data collection herein.
Noise data collection Dn-2In include the consistent sample image of n-2 mark personnel mark, wherein noise data collection Dn-2 Also referred to as the n-th -2 noise data collection herein.
And so on, noise data collection D1In include n mark personnel mark inconsistent sample image, wherein make an uproar Sound data set D1Also referred to as the first noise data collection herein.
As shown in figure 4, in short, this step is used for according to submodel fDcThe spy learnt on clean data collection Dc Sign instructs deep neural network to learn clean data collection Dc and noise data collection Dn-1On knowledge, with this training pattern fDn-1。
A kind of expression way is changed, this step is in submodel fDcConstraint under, be based on clean data collection Dc and noise data Collect Dn-1Model training is carried out to deep neural network, obtains model fDn-1。
In addition, assuming that more mark personnel mark consistent data herein, have in statistical significance higher A possibility that accuracy rate, the label accordingly marked is correct, is bigger.Therefore, the embodiment of the present application is first based on the higher data of accuracy rate Model training is carried out, and then subsequent training process is instructed based on obtained model.
As an example, there is in statistical significance higher accuracy rate due to clean data collection, be primarily based on Clean data collection carries out model training；In next training process, due to noise data collection Dn-1In whole noise datas Concentrating has highest accuracy rate, therefore is directed to second step training process, in submodel fDcGuidance under, using clean data Collection+noise data collection Dn-1Model training is carried out, model f is obtainedDn-1。
303, model training equipment is under the constraint of the (n-1)th model, based on clean data collection, the (n-1)th noise data collection and N-th -2 noise data collection carries out model training, obtains the n-th -2 model.
As it was noted above, above-mentioned n-th -2 noise data collection refers to noise data collection Dn-2, above-mentioned n-th -2 model i.e. reference Model fDn-2。
Referring to fig. 4, in short, this step is used for according to model fDn-1In clean data collection Dc+ noise data collection Dn-1It goes to school The feature practised instructs deep neural network to learn clean data collection Dc+ noise data collection Dn-1+ noise data collection Dn-2On know Know, with this training pattern fDn-2。
That is, third step training process is directed to, due to noise data collection Dn-2Concentrating in remaining noise data has highest Accuracy rate, therefore in third step training process, in the model f that second step training process obtainsDn-1Guidance under, use is pure Net data set+noise data collection Dn-1+ noise data collection Dn-2Model training is carried out, model f is obtainedDn-2。
304, and so on, model training equipment repeats the constraint of the model obtained in previous step training process Under, based on the process of model training is carried out with the matched training dataset of current training process, until obtaining the first model.
In the embodiment of the present application, under the constraint for constantly repeating the model obtained in previous step training process, base In the process for carrying out model training with the matched training dataset of current training process, until by noise data collection D1It is added to instruction During white silk, the first model is obtained.Wherein, the first model is also referred to as model f hereinD1。
It in one possible implementation, referring to fig. 4, include pure with the matched training dataset of first step training process Net data set Dc；It include clean data collection Dc+ noise data collection D with the matched training dataset of second step training processn-1；With The matched training dataset of second step training process includes clean data collection Dc+ noise data collection Dn-1+ noise data collection Dn-2；With This analogizes, and includes clean data collection Dc+ noise data collection D with the matched training dataset of final step training processn-1+ noise Data set Dn-2+ ...+noise data collection D1。
That is, the sample data volume that uses of next step training process is than the sample data volume that previous step training process uses Greatly, specifically, be each step training process compared to previous step training process for, more noise data collection change Noise data collection D is additionally added in a kind of expression way, second step training processn-1, noise number is additionally added in third step training process According to collection Dn-1, and so on, noise data collection D is additionally added in the n-th step training process, that is, final step training process1。
In conclusion the embodiment of the present application is realized according to submodel fDcThe feature acquired on clean data collection Dc, Instruct the knowledge on deep neural network study clean data collection Dc+ noise data collection Dn-1, training pattern fDn-1.With such It pushes away, final step, uses model f obtained in the previous stepD2, deep neural network is instructed to learn clean data collection Dc+ noise data Collect Dn-1+ ...+noise data collection D1On knowledge, training pattern fD1, this training process provided by the embodiments of the present application is referred to as For knowledge distillation process.
In addition, model f hereinD1Also referred to as the first model, model fD2Also referred to as the second model.
With model fD1And fD2For, then model fD1It is the model f obtained in previous step training processD2Constraint under, base It is obtained in the training of the first training dataset；Wherein, the first training dataset includes clean data collection and noise data collection, in detail For, including n-1 noise data collection, respectively noise data collection Dn-1+ noise data collection Dn-2+ ...+noise data collection D1；And For training pattern fD2The second training dataset be the subset of the first training dataset and including clean data collection, i.e., the second instruction Practicing data set includes clean data collection and n-2 noise data collection, respectively noise data collection Dn-1+ noise data collection Dn-2+…+ Noise data collection D2。
That is, model fD1Training process, comprising: in model fD2Constraint under, be based on clean data collection, noise data collection Dn-1+ noise data collection Dn-2+ ...+noise data collection D1, model training is carried out, model f is obtainedD1。
Complete model fD1Training after, so far model training process terminates, trained model fD1It is being integrated into figure After in processing equipment, it can be realized and classified by image processing equipment auxiliary to digestive endoscope image to be detected.By Model f is carried out in use digestive endoscope imageD1Training, if by trained model fD1It is integrated into mobile medical terminal On, then as the mobile medical terminal of image processing equipment, doctor can be assisted to carry out digestive endoscope image to be detected Disease of digestive tract detection.
The image detection stage
305, image processing equipment obtains digestive endoscope image to be detected.
In the embodiment of the present application, body area can be referred to alimentary canal position, i.e., based on digestive endoscope image to digestion Tract disease is detected, and the embodiment of the present application is to this without specifically limiting.
As an example, digestive endoscope image to be detected is usually that the camera of medical instrument is deep into body area Inside carry out Image Acquisition and obtain.And camera can be transferred directly to image procossing after collecting digestive endoscope image Equipment.
In the embodiment of the present application, it carries out detecting it by digestive endoscope image unit device learning model to be detected Before, also first digestive endoscope image to be detected can be pre-processed, wherein pretreatment includes but is not limited to size cutting processing And registration process, the embodiment of the present application is to this without specifically limiting.
306, image processing equipment classifies to digestive endoscope image to be detected based on the first model, and exports and obtain Prediction classification results.
In the embodiment of the present application, digestive endoscope image to be detected is input in the first model, the first model will Output prediction classification results, i.e. output diagnostic result.For example, providing whether relative patient suffers from certain disease of digestive tract.For example, It is input to a Colon and rectum endoscopic image in image processing equipment, the diagnostic result of image processing equipment output is gland cancer.
In one possible implementation, it can be classified by the display screen of image processing equipment to the prediction of output and be tied Fruit is shown.
Method provided by the embodiments of the present application at least has the advantages that
During model training, training dataset has been divided into clean data collection and noise data by the embodiment of the present application Collection, wherein it includes marking consistent sample image that clean data, which is concentrated, and noise data collection includes the inconsistent sample graph of mark Picture, further noise data collection is subdivided into n-1, wherein includes n-1 mark personnel mark in noise data collection Dn-1 Consistent sample image includes the n-2 consistent sample image of mark personnel mark in noise data collection Dn-2, and so on, It include that n mark personnel mark inconsistent sample image in noise data collection D1.
Consistent data are marked due to more marking personnel, there is higher accuracy rate in statistical significance, it is corresponding to mark A possibility that label of note is correct is bigger, therefore the embodiment of the present application knowledge based distills frame, first higher based on accuracy rate Data carry out model training, and then instruct subsequent training process based on obtained model, namely during model training, weight Under the multiple constraint for executing the model obtained in previous step training process, it is based on and the matched training dataset of current training process The process of model training is carried out, until noise data collection D1 is added in training process, obtains model fD1。
In addition, it includes clean data collection that the training data that each step training process uses, which is concentrated, and is trained in next step The noise data collection more compared to previous step training process of training dataset used in journey, i.e., each step training process It is that a noise data collection is additionally added again on the basis of previous step training process, wherein this noise being additionally added Data set is that remaining noise data concentrates accuracy rate highest.
In conclusion the embodiment of the present application is other than using consistent data are marked during model training, moreover it is possible to Enough effective uses mark inconsistent data, realize while increasing the data volume of training dataset, but can reduce because The label of mark malfunctions and the influence to model prediction precision, so as to mark the base of inconsistent data in effective use On plinth, the predictablity rate of model is effectively improved.
A kind of expression way is changed, digestive endoscope image processing method provided by the embodiments of the present application can efficiently use this Part marks inconsistent data, can be realized while increasing sample data volume, reduces and makes because of label error to model At influence, thus improve train come machine learning model predictablity rate.
The precision for training the machine learning model come namely based on above-mentioned training method is preferable, it can be ensured that image detection Detection precision in the process.
Pseudo label calculates and loss function design
In another embodiment, the core of the aforementioned knowledge distillation thought referred to also resides in the design of loss function, In, loss function is used to measure trained machine learning model to the true value of the predicted value of sample image and sample image not Consistent degree.Assuming that y is the label marked by professional mark personnel, y* is unknown true tag, due to mark personnel's mark Y there is error probability to a certain extent, therefore the embodiment of the present application can reset the value of y according to certain rule, with Make it closer to true tag y*, wherein the y of assignment again is defined as pseudo label by the embodiment of the present applicationWherein, above-mentioned Pseudo label is also referred to as physical tags herein.
In one possible implementation, correspond to knowledge above-mentioned and distill frame, the embodiment of the present application is using distillation Method calculates pseudo labelWherein, pseudo labelCalculation formula it is as follows.
In above-mentioned formula,Refer to the physical tags of sample image；λ refers to adjustable coefficient, and value is one normal Number；The mark label of y reference sample image；S refers to model obtained in previous step training process to the pre- of respective sample image It surveys as a result, such as model fDc,fDn-1The prediction result of equal outputs, wherein prediction result herein is also referred to as prediction label.
First point for needing to illustrate is that pseudo label is obtained by mark label and the previous step training of professional mark personnel The prediction label of model is obtained according to certain weight proportion, this is as closely as possible to pseudo label in statistical significance really Label y* may finally make the modelling effect trained more preferable.
The second point for needing to illustrate is, by the above formula, the pseudo label of sample imageAssignment again be related to The prediction result for the model that previous step training obtains, and knowledge distillation thought is exactly to embody here, the mould that previous step training obtains The prediction result of type, can be to the pseudo label of sample image in next step training processAssignment has an impact, i.e., previous step was trained The feature that model learns in respective sample data in journey is used in the training process for instructing next step.
In addition, the embodiment of the present application calculates loss, the i.e. meter of loss function in the calculating of loss function, using the way of distillation It is as follows to calculate formula.
L(yi,f(xi))=l (λ yi+(1-λ)si,f(xi))
Wherein, λ refers to adjustable coefficient, and value is a constant；The value of i is positive integer, xiRefer to i-th of sample Image, yiIt is mark personnel to sample image xiMark label, f (xi) currently trained model is referred to sample image xi's Prediction label, siModel obtained in previous step training process is referred to sample image xiPrediction label.
In one possible implementation, si=δ [fD(xi)/T], wherein δ [] refers to sigmod activation primitive, and T is One constant, fDFor model obtained in previous step training process.L () function can be common cross entropy loss function, this Shen Please embodiment to this without specifically limiting.
In above-mentioned formula, pseudo label isIt is the mark label y by professional mark personneliWith Model fDPrediction result siIt is obtained according to certain weight proportion, so that pseudo label is as closely as possible to very in statistical significance Real label may finally make the modelling effect trained more preferable.
Below with reference to the calculation of above-mentioned pseudo label and the design method of loss function, to having in above-mentioned steps 304 About model fD1Training process be described in detail.
In one possible implementation, in model fD2Constraint under, be based on clean data collection and n-1 noise data Collection carries out model training, obtains model fD1, comprising:
The sample image that clean data collection and n-1 noise data are concentrated is inputted into deep neural network；Obtain depth mind Through network to the prediction label of sample image；Based on loss function, continuous iteration updates the network parameter of deep neural network, directly It is restrained to deep neural network.
Wherein, which is the aforementioned loss function based on distillation thought referred to, is used to measure sample graph The physical tags of picture and the inconsistent degree of prediction label；In addition, as it was noted above, physical tags, that is, pseudo label acquisition Journey, comprising: obtain mark personnel to the mark label of sample image；Obtain model fD2To the prediction label of sample image；It is based on The mark label and model f of sample imageD2The prediction label of output generates the physical tags of sample image.
It should be noted that by model fD2Guidance model fD1During being trained, model fD2It can be to all marks Consistent data are predicted with inconsistent data are marked, that is, export prediction label, and then combine mark label, and generation is pseudo- to mark Label, the embodiment of the present application is to this without specifically limiting.
In another embodiment, in addition to calculating pseudo label using the way of distillationExcept, the following two kinds puppet mark can also be taken LabelCalculation.
The first, smooth stamp methods
This method calculates pseudo label using following formulaRegularization constraint is carried out to model, introduces one independently of sample This distribution is uniformly distributed u, by modification true value distribution, avoids model over-confident to prediction result.
Wherein, u is a constant vector, and λ is adjustable coefficient.
Second, Bootload
This method calculates pseudo label using following formulaBut due in the training process without introducing any additional letter Breath, therefore usual s ' and mark label y have high correlation, using mark label y, there is no too in this way with directly for institute Big difference.
Wherein, s ' is the prediction result that last round of iteration obtains during model training.
First point for needing to illustrate be, in neural network the number of iterations refer to entire training dataset be input to network into The number of row training, that is, the model that aforementioned each step training obtains, is all based on entire training number during hands-on It is obtained after repeatedly being trained according to collection progress.
The second point for needing to illustrate is, the embodiment of the present application when calculating pseudo label using the calculation of the way of distillation, Its principle for not only having used for reference smooth stamp methods and Bootload, but it is better than both methods again simultaneously, because of previous step mould The prediction result of type output is better than a constant constant vector u, and Bootload is actually in processing noise data collection It does not have a clear superiority in problem.
In the embodiment of the present application, since pseudo label is obtained by mark label and the previous step training of professional mark personnel The prediction label of model obtained according to certain weight proportion, this is as closely as possible to pseudo label in statistical significance very Real label y* may finally make the modelling effect trained more preferable.
In conclusion the embodiment of the present application distills the above-mentioned calculation of frame, pseudo label and loss function using knowledge, The effect for marking inconsistent data can be maximized, the effective use to inconsistent data are marked is realized, can be realized While increasing sample data volume, reducing influences caused by model due to label error, to improve the machine for training and The predictablity rate of learning model.
In one possible implementation, to carry out disease of digestive tract detection, Fig. 5 is participated in, the embodiment of the present application provides The overall flow of digestive endoscope image processing method include:
501, model training equipment is trained deep neural network based on clean data collection, obtains submodel fDc, Wherein, it includes that n doctor marks consistent digestive endoscope image that clean data, which is concentrated,.
Wherein, n is the doctor's sum for being trained data set mark.
502, model training equipment is in submodel fDcConstraint under, be based on clean data collection and noise data collection Dn-1It is right Deep neural network is trained, and obtains model fDn-1。
503, model training equipment is in model fDn-1Constraint under, be based on clean data collection, noise data collection Dn-1And noise Data set Dn-2, deep neural network is trained, model f is obtainedDn-2。
504, and so on, model training equipment repeats the constraint of the model obtained in previous step training process Under, based on the process of model training is carried out with the matched training dataset of current training process, until obtaining model fD1。
Wherein, image processing equipment is integrated with trained model fD1。
505, image processing equipment obtains digestive endoscope image to be detected.
506, image processing equipment is based on model fD1Classify to digestive endoscope image to be detected, and exports and obtain Predict classification results.
The embodiment of the present application can maximize the effect that doctor marks inconsistent data, can be effectively reduced doctor's mark The influence of the label of mistake can be realized while increasing sample data volume, reduce due to label error caused by model It influences, to the classification accuracy of alimentary canal endoscopic image, preferably auxiliary doctor carries out based in alimentary canal the final model that improves The disease of digestive tract of mirror image diagnoses.
Fig. 6 is a kind of structural schematic diagram of digestive endoscope image processing apparatus provided by the embodiments of the present application.Referring to figure 6, which includes:
Module 601 is obtained, for obtaining digestive endoscope image to be detected；
Processing module 602, for being classified to the digestive endoscope image to be detected based on the first model, described the One model is to be obtained under the constraint of the second model based on the training of the first training dataset, and first training dataset includes Clean data collection and noise data collection, second model are before training first model based on the second training dataset What training obtained；
Wherein, the clean data collection includes marking consistent sample image, and the noise data collection includes that mark is different The sample image of cause, second training dataset be first training dataset subset and including the clean data Collection, the sample image are digestive endoscope image.
Training dataset has been divided into clean data collection and noise data collection by device provided by the embodiments of the present application, wherein It includes marking consistent sample image that clean data, which is concentrated, and it includes marking inconsistent sample image that noise data, which is concentrated, in order to Evade and mark inconsistent data adverse effect caused by model training, realizes that effective use marks inconsistent data, In It include multistep model training during model training, such as the first mould for classifying to digestive endoscope image to be detected Type be under the constraint of the second model training obtain, wherein the training process of the second model training the first model before, Namely model training is a knowledge distillation process, the model obtained before can instruct current model training, wherein Mei Yibu It includes clean data collection that the training data that training uses, which is concentrated, and currently training uses the data set that training uses before The subset of data set, it ensures that knowledge distillation process is the process of a prediction precision for stepping up model.
A kind of expression way is changed, the embodiment of the present application is other than it can utilize the consistent data of mark, additionally it is possible to effectively It using inconsistent data are marked, realizes while increasing the data volume of training dataset, and can reduce because mark is marked The wrong and influence to model prediction precision is checked out, so as to have on the basis of effective use marks inconsistent data Effect improves the predictablity rate of model.The precision for training the machine learning model come namely based on above-mentioned training method is preferable, In After getting digestive endoscope image to be detected, the first model can be directly based upon, digestive endoscope image to be detected is divided Class, it is ensured that the detection precision in image-detection process.
In one possible implementation, for the total number that the noise data integrates as n-1, n is just whole not less than 2 Number；
Wherein, it includes that n-1 mark personnel mark consistent sample image that the (n-1)th noise data, which is concentrated,；N-th -2 noise number It include that n-2 mark personnel mark consistent sample image according to concentrating；And so on, it includes n mark that the first noise data, which is concentrated, Note personnel mark inconsistent sample image.
In one possible implementation, second training dataset includes that the clean data collection and n-2 make an uproar Sound data set, referring to Fig. 7, the device further include:
Training module 603, for being based on the clean data collection and n-1 noise under the constraint of second model Data set carries out model training, obtains first model.
In one possible implementation, training module 603 are also used to carry out model instruction based on the clean data collection Practice, obtains submodel；Under the constraint of the submodel, it is based on the clean data collection and (n-1)th noise data Collection carries out model training, obtains the (n-1)th model；Under the constraint of (n-1)th model, based on the clean data collection, described (n-1)th noise data collection and n-th -2 noise data collection carry out model training, obtain the n-th -2 model；And so on, it repeats Execute the model obtained in previous step training process constraint under, based on the matched training dataset of current training process into The process of row model training, until obtaining first model.
In one possible implementation, training module 603 are also used to the clean data collection and n-1 noise Sample image in data set inputs deep neural network；The deep neural network is obtained to the pre- mark of the sample image Label；Based on loss function, continuous iteration updates the network parameter of the deep neural network, until the deep neural network is received It holds back, the loss function is used to measure the physical tags of the sample image and the inconsistent degree of prediction label；
Wherein, the corresponding prediction label that the physical tags are exported based on second model obtains.
In one possible implementation, training module 603 are also used to obtain the mark personnel to the sample graph The mark label of picture；
Second model is obtained to the prediction label of the sample image；
The prediction label of mark label and second model output based on the sample image, generates the sample graph The physical tags of picture.
In one possible implementation, using following formula, the physical tags of the sample image are generated:
Wherein,Refer to the physical tags of the sample image；λ refers to adjustable coefficient, and value is a constant；y Refer to the mark label of the sample image；S refers to model obtained in previous step training process to the pre- of the sample image Mark label.
In one possible implementation, the calculation formula of the loss function are as follows:
L(yi,f(xi))=l (λ yi+(1-λ)si,f(xi))
Wherein, λ refers to adjustable coefficient, and value is a constant；The value of i is positive integer, xiRefer to i-th of sample Image, yiIt is the mark personnel to sample image xiMark label, f (xi) currently trained model is referred to sample image xiPrediction label, siModel obtained in previous step training process is referred to sample image xiPrediction label.
All the above alternatives can form the alternative embodiment of the disclosure, herein no longer using any combination It repeats one by one.
It should be understood that digestive endoscope image processing apparatus provided by the above embodiment is when performing image processing, Only the example of the division of the above functional modules, it in practical application, can according to need and by above-mentioned function distribution It is completed by different functional modules, i.e., the internal structure of device is divided into different functional modules, it is described above to complete All or part of function.In addition, digestive endoscope image processing apparatus provided by the above embodiment and digestive endoscope image Processing method embodiment belongs to same design, and specific implementation process is detailed in embodiment of the method, and which is not described herein again.
Fig. 8 is a kind of structural schematic diagram of model training equipment provided by the embodiments of the present application, the model training equipment 800 Bigger difference can be generated because configuration or performance are different, may include one or more processors (central Processing units, CPU) 801 and one or more memory 802, wherein it is stored in the memory 802 There is at least one instruction, at least one instruction is loaded by the processor 801 and executed to realize that above-mentioned each method is real The digestive endoscope image processing method of example offer is provided.Certainly, which can also have wired or wireless network The components such as interface, keyboard and input/output interface, to carry out input and output, which can also include other For realizing the component of functions of the equipments, this will not be repeated here.
In the exemplary embodiment, a kind of computer readable storage medium is additionally provided, the memory for example including instruction, Above-metioned instruction can be executed by the processor in model training equipment to complete the digestive endoscope image procossing in above-described embodiment Method.For example, the computer readable storage medium can be ROM, random access memory (RAM), CD-ROM, tape, soft Disk and optical data storage devices etc..
Fig. 9 is a kind of structural schematic diagram of image processing equipment provided by the embodiments of the present application, the image processing equipment 900 Bigger difference can be generated because configuration or performance are different, may include one or more processors (central Processing units, CPU) 901 and one or more memory 902, wherein it is stored in the memory 902 There is at least one instruction, at least one instruction is loaded by the processor 901 and executed to realize that above-mentioned each method is real The digestive endoscope image processing method of example offer is provided.Certainly, which can also have wired or wireless network The components such as interface, keyboard and input/output interface, to carry out input and output, which can also include other For realizing the component of functions of the equipments, this will not be repeated here.
In the exemplary embodiment, a kind of computer readable storage medium is additionally provided, the memory for example including instruction, Above-metioned instruction can be executed by the processor in image processing equipment to complete the digestive endoscope image procossing in above-described embodiment Method.For example, the computer readable storage medium can be ROM, random access memory (RAM), CD-ROM, tape, soft Disk and optical data storage devices etc..
Those of ordinary skill in the art will appreciate that realizing that all or part of the steps of above-described embodiment can pass through hardware It completes, relevant hardware can also be instructed to complete by program, the program can store in a kind of computer-readable In storage medium, storage medium mentioned above can be read-only memory, disk or CD etc..
The foregoing is merely the preferred embodiments of the application, not to limit the application, it is all in spirit herein and Within principle, any modification, equivalent replacement, improvement and so on be should be included within the scope of protection of this application.

Claims (15)

1. a kind of digestive endoscope image processing method, which is characterized in that the described method includes:
Obtain digestive endoscope image to be detected；
Classified based on the first model to the digestive endoscope image to be detected, first model is in the second model It is obtained under constraint based on the training of the first training dataset, first training dataset includes clean data collection and noise data Collection, second model are to be obtained before training first model based on the training of the second training dataset；
Wherein, the clean data collection includes marking consistent sample image, and the noise data collection includes that mark is inconsistent Sample image, second training dataset be first training dataset subset and including the clean data collection, institute Stating sample image is digestive endoscope image.
2. n is not the method according to claim 1, wherein the total number that the noise data integrates is n-1 Positive integer less than 2；
Wherein, it includes that n-1 mark personnel mark consistent sample image that the (n-1)th noise data, which is concentrated,；N-th -2 noise data collection In include the consistent sample image of n-2 mark personnel mark；And so on, it includes n mark people that the first noise data, which is concentrated, Member marks inconsistent sample image.
3. according to the method described in claim 2, it is characterized in that, second training dataset includes the clean data collection With n-2 noise data collection, the training process of first model, comprising:
Under the constraint of second model, model training is carried out based on the clean data collection and n-1 noise data collection, is obtained To first model.
4. according to the method in claim 2 or 3, which is characterized in that the method also includes:
Model training is carried out based on the clean data collection, obtains submodel；
Under the constraint of the submodel, model instruction is carried out based on the clean data collection and the (n-1)th noise data collection Practice, obtains the (n-1)th model；
Under the constraint of (n-1)th model, it is based on the clean data collection, the (n-1)th noise data collection and described n-th -2 Noise data collection carries out model training, obtains the n-th -2 model；
And so on, under the constraint for repeating the model obtained in previous step training process, it is based on and current training process Matched training dataset carries out the process of model training, until obtaining first model.
5. according to the method described in claim 3, it is characterized in that, described under the constraint of second model, based on described Clean data collection and n-1 noise data collection carry out model training, comprising:
The sample image that the clean data collection and n-1 noise data are concentrated is inputted into deep neural network；
The deep neural network is obtained to the prediction label of the sample image；
Based on loss function, continuous iteration updates the network parameter of the deep neural network, until the deep neural network Convergence, the loss function are used to measure the physical tags of the sample image and the inconsistent degree of prediction label；
Wherein, the corresponding prediction label that the physical tags are exported based on second model obtains.
6. according to the method described in claim 5, it is characterized in that, the acquisition process of the physical tags of the sample image, packet It includes:
The mark personnel are obtained to the mark label of the sample image；
Second model is obtained to the prediction label of the sample image；
The prediction label of mark label and second model output based on the sample image, generates the sample image Physical tags.
7. according to the method described in claim 6, it is characterized in that, generating the reality of the sample image using following formula Label:
Wherein,Refer to the physical tags of the sample image；λ refers to adjustable coefficient, and value is a constant；Y is referred to The mark label of the sample image；S refers to model obtained in previous step training process to the pre- mark of the sample image Label.
8. according to the method described in claim 5, it is characterized in that, the calculation formula of the loss function are as follows:
L(yi,f(xi))=l (λ yi+(1-λ)si,f(xi))
Wherein, λ refers to adjustable coefficient, and value is a constant；The value of i is positive integer, xiI-th of sample image is referred to, yiIt is the mark personnel to sample image xiMark label, f (xi) currently trained model is referred to sample image xiIt is pre- Mark label, siModel obtained in previous step training process is referred to sample image xiPrediction label.
9. a kind of digestive endoscope image processing apparatus, which is characterized in that described device includes:
Module is obtained, for obtaining digestive endoscope image to be detected；
Processing module, for being classified based on the first model to the digestive endoscope image to be detected, first model It is to be obtained under the constraint of the second model based on the training of the first training dataset, first training dataset includes pure number According to collection and noise data collection, second model is trained based on the second training dataset before training first model It arrives；
Wherein, the clean data collection includes marking consistent sample image, and the noise data collection includes that mark is inconsistent Sample image, second training dataset be first training dataset subset and including the clean data collection, institute Stating sample image is digestive endoscope image.
10. device according to claim 9, which is characterized in that second training dataset includes the clean data Collection and n-2 noise data collection, described device further include:
Training module, under the constraint of second model, based on the clean data collection and n-1 noise data collection into Row model training obtains first model.
11. device according to claim 10, which is characterized in that the training module is also used to based on the pure number Model training is carried out according to collection, obtains submodel；Under the constraint of the submodel, based on the clean data collection and described (n-1)th noise data collection carries out model training, obtains the (n-1)th model；Under the constraint of (n-1)th model, based on described pure Net data set, the (n-1)th noise data collection and n-th -2 noise data collection carry out model training, obtain the n-th -2 model； And so on, under the constraint for repeating the model obtained in previous step training process, it is based on matching with current training process Training dataset carry out the process of model training, until obtain first model.
12. device according to claim 10, which is characterized in that the training module is also used to the clean data The sample image that collection and n-1 noise data are concentrated inputs deep neural network；The deep neural network is obtained to the sample The prediction label of this image；Based on loss function, continuous iteration updates the network parameter of the deep neural network, until described Deep neural network convergence, the loss function be used for measure the sample image physical tags and prediction label it is inconsistent Degree；
Wherein, the corresponding prediction label that the physical tags are exported based on second model obtains.
13. a kind of storage medium, which is characterized in that it is stored at least one instruction in the storage medium, described at least one It instructs as processor loads and executes to realize the digestive endoscope figure as described in any of claim 1 to 8 claim As processing method.
14. a kind of image processing equipment, which is characterized in that the equipment includes processor and memory, is deposited in the memory At least one instruction is contained, at least one instruction is loaded by the processor and executed to realize as in claim 1 to 8 Digestive endoscope image processing method described in any one claim.
15. a kind of image processing system, which is characterized in that the system comprises: model training equipment and image processing equipment, institute Stating image processing equipment includes display screen；
The model training equipment, for being based on the first training dataset the first model of training, institute under the constraint of the second model Stating the first training dataset includes clean data collection and noise data collection, second model be training first model it It is preceding to be obtained based on the training of second training dataset；Wherein, the clean data collection includes marking consistent sample image, described Noise data collection includes the inconsistent sample image of mark, and second training dataset is the son of first training dataset Collect and including the clean data collection；
Described image processing equipment includes processor and memory, is stored at least one instruction in the memory, it is described extremely A few instruction is loaded by the processor and is executed to realize: being obtained digestive endoscope image to be detected, is based on described first Model classifies to the digestive endoscope image to be detected；
The display screen is for showing the prediction classification results of output.
CN201910670467.XA 2019-07-24 2019-07-24 Digestive endoscope image processing method, device, storage medium, equipment and system Pending CN110427994A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910670467.XA CN110427994A (en) 2019-07-24 2019-07-24 Digestive endoscope image processing method, device, storage medium, equipment and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910670467.XA CN110427994A (en) 2019-07-24 2019-07-24 Digestive endoscope image processing method, device, storage medium, equipment and system

Publications (1)

Publication Number Publication Date
CN110427994A true CN110427994A (en) 2019-11-08

Family

ID=68410593

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910670467.XA Pending CN110427994A (en) 2019-07-24 2019-07-24 Digestive endoscope image processing method, device, storage medium, equipment and system

Country Status (1)

Country Link
CN (1) CN110427994A (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111126574A (en) * 2019-12-30 2020-05-08 腾讯科技（深圳）有限公司 Method and device for training machine learning model based on endoscopic image and storage medium
CN111414946A (en) * 2020-03-12 2020-07-14 腾讯科技（深圳）有限公司 Artificial intelligence-based medical image noise data identification method and related device
CN112842245A (en) * 2020-12-30 2021-05-28 中原工学院 Method and system for detecting friction injury between endoscopic clamp and intestinal tissue
CN113470031A (en) * 2021-09-03 2021-10-01 北京字节跳动网络技术有限公司 Polyp classification method, model training method and related device
CN113496489A (en) * 2021-09-06 2021-10-12 北京字节跳动网络技术有限公司 Training method of endoscope image classification model, image classification method and device

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107392255A (en) * 2017-07-31 2017-11-24 深圳先进技术研究院 Generation method, device, computing device and the storage medium of minority class picture sample

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107392255A (en) * 2017-07-31 2017-11-24 深圳先进技术研究院 Generation method, device, computing device and the storage medium of minority class picture sample

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
SHENG GUO等: "CurriculumNet: Weakly Supervised Learning from Large-Scale Web Images", 《ECCV》 *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111126574A (en) * 2019-12-30 2020-05-08 腾讯科技（深圳）有限公司 Method and device for training machine learning model based on endoscopic image and storage medium
CN111126574B (en) * 2019-12-30 2023-07-28 腾讯科技（深圳）有限公司 Method, device and storage medium for training machine learning model based on endoscopic image
CN111414946A (en) * 2020-03-12 2020-07-14 腾讯科技（深圳）有限公司 Artificial intelligence-based medical image noise data identification method and related device
CN111414946B (en) * 2020-03-12 2022-09-23 腾讯科技（深圳）有限公司 Artificial intelligence-based medical image noise data identification method and related device
CN112842245A (en) * 2020-12-30 2021-05-28 中原工学院 Method and system for detecting friction injury between endoscopic clamp and intestinal tissue
CN113470031A (en) * 2021-09-03 2021-10-01 北京字节跳动网络技术有限公司 Polyp classification method, model training method and related device
WO2023030298A1 (en) * 2021-09-03 2023-03-09 北京字节跳动网络技术有限公司 Polyp typing method, model training method and related apparatus
CN113496489A (en) * 2021-09-06 2021-10-12 北京字节跳动网络技术有限公司 Training method of endoscope image classification model, image classification method and device
CN113496489B (en) * 2021-09-06 2021-12-24 北京字节跳动网络技术有限公司 Training method of endoscope image classification model, image classification method and device

Similar Documents

Publication Publication Date Title
Min et al. Overview of deep learning in gastrointestinal endoscopy
CN110390674A (en) Image processing method, device, storage medium, equipment and system
US10902588B2 (en) Anatomical segmentation identifying modes and viewpoints with deep learning across modalities
CN110427994A (en) Digestive endoscope image processing method, device, storage medium, equipment and system
Zulkifley et al. Pterygium-Net: a deep learning approach to pterygium detection and localization
Liu et al. Medical-vlbert: Medical visual language bert for covid-19 ct report generation with alternate learning
Ahishali et al. Advance warning methodologies for covid-19 using chest x-ray images
Huan et al. Deep convolutional neural networks for classifying body constitution based on face image
CN112365980A (en) Brain tumor multi-target point auxiliary diagnosis and prospective treatment evolution visualization method and system
CN106897572A (en) Lung neoplasm case matching assisted detection system and its method of work based on manifold learning
CN116848588A (en) Automatic labeling of health features in medical images
Raut et al. Transfer learning based video summarization in wireless capsule endoscopy
Sasi Kumar et al. DeepQ Residue Analysis of Brain Computer Classification and Prediction Using Deep CNN
CN117237351B (en) Ultrasonic image analysis method and related device
Ly et al. New compact deep learning model for skin cancer recognition
Li et al. Artificial intelligence accelerates multi-modal biomedical process: A Survey
Lamia et al. Detection of pneumonia infection by using deep learning on a mobile platform
Liao et al. Convolutional herbal prescription building method from multi-scale facial features
Hasan et al. Dental impression tray selection from maxillary arch images using multi-feature fusion and ensemble classifier
CN109859813A (en) A kind of entity modification word recognition method and device
Alissa Parkinson's Disease Diagnosis Using Deep Learning
Elif et al. Accurate myocardial pathology segmentation with residual u-net
Fu et al. PKA 2-net: Prior knowledge-based active attention network for accurate pneumonia diagnosis on chest X-ray images
Charulatha et al. Computer-Aided Design for skin disease Identification and Categorization Using Image Processing
Zhao et al. Two-stream vision transformer based multi-label recognition for TCM prescriptions construction

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20191108