CN110189305A - A kind of multitask tongue picture automatic analysis method - Google Patents
A kind of multitask tongue picture automatic analysis method Download PDFInfo
- Publication number
- CN110189305A CN110189305A CN201910397988.2A CN201910397988A CN110189305A CN 110189305 A CN110189305 A CN 110189305A CN 201910397988 A CN201910397988 A CN 201910397988A CN 110189305 A CN110189305 A CN 110189305A
- Authority
- CN
- China
- Prior art keywords
- multitask
- tongue
- deep neural
- neural network
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0082—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence adapted for particular medical purposes
- A61B5/0088—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence adapted for particular medical purposes for oral or dental tissue
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/48—Other medical applications
- A61B5/4854—Diagnosis based on concepts of traditional oriental medicine
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Medical Informatics (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Surgery (AREA)
- Computational Linguistics (AREA)
- Veterinary Medicine (AREA)
- Computing Systems (AREA)
- Public Health (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Animal Behavior & Ethology (AREA)
- Pathology (AREA)
- Heart & Thoracic Surgery (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Alternative & Traditional Medicine (AREA)
- Quality & Reliability (AREA)
- Radiology & Medical Imaging (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Dentistry (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a kind of multitask tongue picture automatic analysis methods, comprising the following steps: all images are divided into strong tally set and non label set according to the presence or absence of label by acquisition tongue picture picture；Use the multitask deep neural network of strong two kinds of the tally set training different frameworks；Use the multitask deep neural network of two kinds of the no label data alternative optimization different frameworks；Tongue picture picture is input to two trained multitask deep neural networks, the two output results analyzed averagely are last tongue picture multitask analysis result.A kind of multitask tongue picture automatic analysis method disclosed by the invention, it is for limitation and defect present in the lingual diagnosis analytical technology currently based on artificial intelligence, we attempt using semi-supervised learning method, in the case where possessing has label tongue picture data on a small quantity, using others without label data, the feature for learning tongue picture, improves the nicety of grading and generalization ability of model.
Description
Technical field
The present invention relates to intelligent diagnostics field more particularly to a kind of multitask tongue picture automatic analysis methods.
Background technique
Traditional Chinese Medicine possesses thousands of years history in China, mainly passes through the four methods of diagnosis and carries out to the illness of physical therapy person
Judgement is being cured the disease and there is significant effect in the field of taking care of health.As a part of observation, TCM tongue diagnosis also becomes tcm diagnosis
One important evidence.The presence or absence of the shape of tongue, color, crackle and the presence or absence of the depth, indentation can embody disease to a certain extent
The property of disease, the depth of patient's condition, the prosperity and decline of qi and blood.Supply falls short of demand by traditional Chinese physician, and with based on machine learning, deep learning
Image recognition technology computer field flourish, cause the intelligent analysis of TCM tongue diagnosis to have become trend.
The determination method and system of the TCM tongue diagnosis model based on convolutional Neural metanetwork have been invented (specially by University of Science & Technology, Beijing
Benefit ZL 201610300494.4).This method by adjusting every layer of neuron of convolutional Neural metanetwork number, using on tongue
Image is trained multiple and different convolutional Neural metanetworks adjusted as training set respectively, determines multiple and different structures
The parameter of convolutional Neural metanetwork finally chooses the wherein highest network mould of accuracy rate of diagnosis using test set as input
Type.
Tongue object detection method (patent No. ZL under a kind of open environment has invented in Xiamen University
201610300494.4).This method acquires image first, then carries out linear transformation to the RGB component of image and realizes color school
Just.Then cut zone is sought into union after being split using distinct methods to image.Then using the texture of tongue body after segmentation
Information classifies to tongue picture as feature.
A kind of tongue fur constitution discriminating conduct (publication number CN based on deep neural network has invented in South China Science & Engineering University
106683087 A).The tongue picture of acquisition is first carried out the normalization of color space and the modulation of size by this method, then sharp
It is further accurately fixed with a kind of fast convolution neural network Preliminary detection tongue fur position based on region, then with neural network algorithm
Position tongue fur position extracts tongue fur feature using convolutional neural networks algorithm and recurrent neural network algorithm, finally utilizes softmax
Classifier or support vector machine classifier carry out tongue fur classification of TCM constitution.
It is (open that a kind of Chinese medicine tongue color coating colour automatic analysis method based on convolutional neural networks has been invented by Beijing University of Technology
Number 107330889 A of CN).This method acquires tongue image data by Tongue instrument, realizes tongue fur using K mean cluster
With the separation of tongue nature, and by Traditional Chinese Medicine experts demarcate tongue color coating colour.Convolutional neural networks are constructed respectively to tongue color data set and coating colour
Data set is trained, and obtains tongue color disaggregated model and coating colour disaggregated model.
A kind of tongue picture constitution neural network based discrimination side has been invented by scientific and technological (Shanghai) Co., Ltd of little Wu health
Method and equipment (109199334 A of publication number CN).This method trains 1 tongue picture picture by deep neural network algorithm and knows
Other model, 5 tongue picture Feature Selection Models and 1 tongue picture constitution model.When predicting test set, pass through tongue picture figure first
Piece identification model judges whether it is tongue picture, if tongue picture then utilizes 5 kinds of tongue picture Feature Selection Models to extract tongue nature face respectively
Color, tongue as form, whether there is or not five kinds of tongue fur, thickness of the tongue coating, tongue fur color features.5 kinds of features of extraction are inputted into tongue picture constitution mould
Type obtains user and corresponds to constitution.
Above based on artificial intelligence in the analysis method of tongue picture, just for some feature of tongue picture, such as tongue
Shape, tongue color, coating colour etc. construct single analysis model, although in 109199334 A of publication number CN, while considering tongue
Five kinds of features of elephant, but the patent is still to construct five kinds of different identification models for five kinds of features.And in practical application
In, traditional Chinese physician would generally multiple features to tongue picture carry out discriminatory analysis simultaneously, so intelligently to multiple features of tongue picture
It is necessary for carrying out analysis simultaneously.
Since supply falls short of demand by traditional Chinese physician, although tongue picture data is caused to be easy to obtain, there is the figure of complete diagnosis information
As data, it to be more difficult acquisition that we term it strong label datas, and diagnostic message missing or no data, we are referred to as
For no label data, it is often easier to acquisition.Semi-supervised learning can allow disaggregated model not depend on as a kind of learning method
External world's interaction automatically promotes learning performance using unmarked sample, can make up exemplar not foot straps to a certain extent
The defect come, avoids disaggregated model since the very little bring accuracy rate of training data is low, the problem of Generalization Capability difference.And it is based on
Semi-supervised image classification algorithms have also been widely applied in the classification problem of natural image, but in the classification problem of tongue picture
In be still to be studied.
A kind of multiclass image classification method (publication number CN based on semi-supervised extreme learning machine has invented in Southeast China University
104992184 B).The sample that unlabeled exemplars are concentrated put back to the training subset that resampling constitutes difference, then with
Semi-supervised extreme learning machine model is respectively trained in marked training sample, is taken by the output summation of extreme learning machine corresponding node
It is average, it takes most uncertain sample manually to be marked from unlabeled exemplars concentration and is transferred in marked training set, again
Sorter model is updated, until iteration terminates, solving image classification in the related technology, there are classification accuracy rate is low and study speed
Low problem is spent, has established certain basis for accurate, quick, stable image classification.
Zhongshan University is it is also proposed that a kind of image classification method (publication number CN based on active semi-supervised learning
109376796A), invention random selection part marker samples and all unmarked samples, for semi-supervised in training pattern
Then dictionary learning component is introduced into a user to mark the full and accurate sample in unmarked sample, is added to the data set of label
In, for the Active Learning component in training pattern, the iteration that repeats the above steps more new model is until algorithm is finally restrained or reached
To a certain the number of iterations.The invention combination semi-supervised learning and Active Learning, effectively utilize all training datas, improve calculation
The performance of method model.
Guangdong University of Technology discloses a kind of scene image mark side based on Active Learning and multi-tag multi-instance learning
Method (105117429 A of publication number CN), the invention utilize active learning strategies, by the confidence level of computation model, select not true
Qualitative maximum is without label image, and then expert manually marks the image, while guaranteeing disaggregated model accuracy, greatly
Reduce the scene image quantity for needing manually to mark greatly, to reduce mark cost.Meanwhile image is converted into multi-tag
More sample datas make image complexity semanteme obtain reasonable representation, improve the accuracy of image labeling.
Summary of the invention
To sum up, it has been found that existing lingual diagnosis technology has following defect: (1) being directed in the existing analysis method to tongue picture,
It is to analyze some individual feature of tongue picture, that is, constructs single model of modal analysis and tongue picture feature is analyzed,
But tongue picture may include tongue color, ligulate, tongue mind and coating nature etc..So individually signature analysis has limitation, and do not account for
Interdependence between each feature；(2) due to there is label data amount few, lead to Analysis of Lingual Picture model generalization performance difference and standard
The low problem of exactness, (utilization rate of data is low, can not largely apply in existing Analysis of Lingual Picture model without label data).
It is to be solved by this invention for limitation and defect present in the above-mentioned lingual diagnosis analytical technology currently based on artificial intelligence
Technical problem is: we attempt using semi-supervised learning method, in the case where possessing has label tongue picture data on a small quantity, utilize
Others learn the feature of tongue picture without label data, improve the nicety of grading and generalization ability of model.Meanwhile we intend adopting
The analysis of the different characteristic of tongue picture is obtained simultaneously with single disaggregated model as a result, simplified model, saves the training time of model.
To achieve the above object, the present invention provides a kind of multitask tongue picture automatic analysis methods, comprising the following steps:
Tongue picture picture is acquired, all images are divided into strong tally set and non label set according to the presence or absence of label；
Use the multitask deep neural network of strong two kinds of the tally set training different frameworks；
Use the multitask deep neural network of two kinds of the no label data alternative optimization different frameworks；
Tongue picture picture is input to two trained multitask deep neural networks, two outputs analyzed
As a result averagely as last tongue picture multitask analysis result.
Using the multitask deep neural network of strong two kinds of the tally set training different frameworks, handed over using no label data
For the multitask deep neural network of two kinds of optimization different frameworks, following steps are specifically included by the tongue picture in the strong tally set
Sample set is randomly divided into two parts, is denoted as tally set S1 and tally set S2；Unlabeled exemplars collection is set, U is denoted as；Define two skies
Collection, is denoted as A1 and A2, and A1 comes from for storing the unlabeled exemplars set from depth residual error network, A2 for storing
The high unlabeled exemplars set of the confidence level of GoogleNet network；Two empty sets are defined, are denoted as B1 and B2, B1 is for storing nothing
Exemplar and its confidence level generated by depth residual error network, B2 is for storing unlabeled exemplars and its passing through
The confidence level that GoogleNet network generates；Define two sorter models, the respectively disaggregated model based on depth residual error network
The disaggregated model for being denoted as F1 and the deep neural network based on GoogleNet is denoted as F2；Define counter i.
Further, the multitask deep neural network includes the multitask disaggregated model note based on depth residual error network
For F1 and the multitask disaggregated model of the deep neural network based on GoogleNet is denoted as F2.
Further, tongue picture is input to two trained multitask deep neural networks, two analyzed
A output result is averagely last tongue picture multitask analysis as a result, specifically including: more based on depth residual error network
Business disaggregated model F1 is analyzed and predicted tongue color, ligulate, the tongue mind of the tongue picture picture, coating nature, and is based on GoogleNet
Deep neural network multitask disaggregated model F2 to the tongue color, ligulate, tongue of the tongue picture picture mind, coating nature carry out analysis and
Prediction.Further, using the multitask deep neural network of two kinds of the no label data alternative optimization different frameworks
Include: select multitask disaggregated model F1 described in the confidence sample optimization that no label data is concentrated based on depth residual error network and
Select the multitask disaggregated model based on GoogleNet deep neural network described in the confidence sample optimization that no label data is concentrated
F2。
Further, the multitask based on depth residual error network described in the confidence sample optimization that no label data is concentrated is selected
Disaggregated model F1, specifically includes:
1. obtaining disaggregated model F1 ' using tally set S1 multitask deep neural network F1；
2. integrating the sample for selecting size in U as n from unlabeled exemplars at random, it is denoted as u={ u1,u2,u3…un}；
3. predicting using disaggregated model F1 ' all samples in sample u, prediction label F ' is obtained1(u)；
4. enabling counter i=1；
5. using S1 ∪ (ui,F′1(ui)) data set multitask deep neural network F1 obtains F "1；
6. using F '1With F "1Tally set S1 is predicted, sample u is calculated according to confidence calculations formula (1)iIt is corresponding
Level of confidence △1i:
Wherein, xjTo there is label data collection S1In sample, yjTo there is label data xjCorresponding true tag.
7. recording uiConfidence level, be denoted as (ui,△1i), it stores in set B1；
8.i=i+1, judgement execute step 5 and otherwise perform the next step if i≤n；
9. selecting the highest sample of level of confidence in set B1, it is denoted as a, takes out sample a and F '1(a), it is put into empty set A2
In.
Further, the confidence sample optimization of no label data concentration is selected based on GoogleNet deep neural network
Multitask disaggregated model F2, specifically includes:
1. obtaining F2 ' using tally set S2 multitask deep neural network F1；
2. integrating the sample for selecting size in U as n from unlabeled exemplars at random, it is denoted as u={ u1,u2,u3…un}；
3. predicting using disaggregated model F2 ' all samples in u, prediction label F ' is obtained2(u)；
4. enabling counter i=1；
5. using S2 ∪ (ui,F′2(ui)) data set multitask deep neural network F2 obtains F "2；
6. using F '2With F "2Tally set S2 is predicted, calculates sample u according to confidence calculations formula (2)iIt is corresponding
Level of confidence △2i:
Wherein, xjTo there is label data collection S2In sample, yjTo there is label data xjCorresponding true tag；
7. recording uiConfidence level, be denoted as (ui,△2i), it stores in set B2；
8.i=i+1, judgement, if i≤n, otherwise return step 5 performs the next step；
9. selecting the highest sample of level of confidence in set B2, it is denoted as b, takes out sample b and F '2(b), it is put into A1.
Further, using the multitask depth nerve net of two kinds of the no label data alternative optimization different frameworks
Network is further comprising the steps of:
1. more new data S1=S1 ∪ A2, S2=S2 ∪ A1；
2. judging whether S1 and S2 changes, do not change, exit, otherwise, is chosen from unlabeled exemplars collection U at random again
Selecting size is the sample of n, multitask disaggregated model F1 of the re -training based on depth residual error network and selects no label data collection
In confidence sample optimization described in the multitask disaggregated model F2 based on GoogleNet deep neural network, until set S1 and
S2 does not change.
Technical effect
1, the invention discloses a kind of analysis method for tongue picture, a kind of multitask tongue picture automatic analysis method is used
Single deep neural network disaggregated model substitutes tradition sofmax function by using sigmoid activation primitive, and uses two
It is worth cross entropy and realizes that ligulate, tongue color, tongue mind and coating nature are analyzed in use simultaneously as loss function, and provides analysis knot
Fruit.Analysis system exports the ligulate of different tongue pictures, tongue color, tongue mind and coating nature information as a whole, it is contemplated that different special
Correlation between sign.
2, the present invention uses the semi-supervised learning method based on coorinated training, constructs the neural network of two different structures,
By way of screening confidence sample mutually, no label data is made full use of, is solved due to there is label data amount to lead to model less
Generalization Capability difference and the low problem of accuracy.
It is described further below with reference to technical effect of the attached drawing to design of the invention, specific structure and generation, with
It is fully understood from the purpose of the present invention, feature and effect.
Detailed description of the invention
Fig. 1 is a kind of flow diagram of multitask tongue picture automatic analysis method of a preferred embodiment of the invention；
Fig. 2 is that the use residual error network of a preferred embodiment of the invention carries out feature extraction and multitask point to tongue picture
The network architecture diagram of class；
Fig. 3 is that a preferred embodiment of the invention using GoogleNet network carries out feature extraction and more to tongue picture
The network architecture diagram of classification of task.
Specific embodiment
As shown in Figure 1, the embodiment of the invention provides a kind of multitask tongue picture automatic analysis methods, comprising the following steps:
Step 100, tongue picture picture is acquired, all images according to the presence or absence of label are divided into strong tally set and without label
Collection；
Step 200, using the multitask deep neural network of strong two kinds of the tally set training different frameworks；
Step 300, using the multitask depth nerve net of two kinds of the no label data alternative optimization different frameworks
Network；
Step 400, tongue picture picture is input to two trained multitask deep neural networks, analyzed
The averagely as last tongue picture multitask analysis result of two output results.It is lacked for present in current Analysis of Lingual Picture technology
Point, the present invention provide a kind of multitask tongue picture automatic analysis method, efficiently using no label data, for solving due to there is label
Data volume leads to Analysis of Lingual Picture model generalization performance difference and the low problem of accuracy less.
The present invention realizes the multiple characteristic differentiation classification of tongue picture using improved deep neural network, by changing network end
Activation primitive and loss function complete tongue picture feature multitask output, in addition, using based on the semi-supervised of coorinated training
Method will be added in training pattern without label data, increase model data amount, solution conventional model generalization ability is poor, accurately
The low problem of rate.That is the scheme of the embodiment of the present invention is the semi-supervised method based on coorinated training, is believed using containing multiple features
The tongue picture data of breath train multitask deep neural network model end to end, obtain dividing for ligulate, tongue color, tongue mind and coating nature
Class result.Wherein, using the semi-supervised learning method based on coorinated training, residual error neural network and GoogleNet are used respectively
Two kinds of disaggregated models from different angles classify to tongue picture data, select think that believable unlabeled exemplars are added to mutually
In training set, a kind of complementation is formed, improves the nicety of grading of entire disaggregated model.Meanwhile by introduce new activation primitive and
Traditional single task neural network is changed to that ligulate, tongue color, tongue mind, the multitask of coating nature feature can be exported simultaneously by loss function
Neural network.
It will be explained in detail a kind of the detailed based on semi-supervised multitasked algorithm of multitask tongue picture automatic analysis method below
Step:
Initialization: the tongue picture data containing multiple characteristic informations are randomly divided into two parts, are denoted as tally set S1 and tally set
S2；By unlabeled exemplars collection, it is denoted as U；Two empty sets are defined, are denoted as A1 and A2, A1 is for storing from depth residual error network
Unlabeled exemplars set, A2 is for storing the high unlabeled exemplars set of the confidence level from GoogleNet network；Define two
Empty set is denoted as B1 and B2, and B1 is used for the confidence level storing unlabeled exemplars and its generating by depth residual error network, and B2 is for depositing
The confidence level putting unlabeled exemplars and its being generated by GoogleNet network；Two sorter models are defined, depth residual error is based on
The disaggregated model of network is denoted as F1, and the deep neural network based on GoogleNet is denoted as F2；Define counter i；
Step 1 obtains F1 ' using tally set S1 multitask deep neural network F1；
Step 2 obtains F2 ' using tally set S2 multitask deep neural network F1；
Step 3 integrates the sample that size is selected in U as n from unlabeled exemplars at random, is denoted as u={ u1,u2,u3…un}；
Step 4 predicts all samples in u using disaggregated model F1 ', obtains prediction label F '1(u)；
Step 5, another counter i=1；
Step 6 uses S1 ∪ (ui,F′1(ui)) data set multitask deep neural network F1 obtains F "1；
Step 7 uses F '1With F "1Tally set S1 is predicted, sample u is calculated according to confidence calculations formula (1)i
Corresponding level of confidence △1i:
Wherein, xjTo there is label data collection S1In sample, yjTo there is label data xjCorresponding true tag.
Record uiConfidence level, be denoted as (ui,△1i), it stores in set B1；
Step 8, i=i+1, judgement, if i≤n, otherwise return step six performs the next step；
Step 9 selects the highest sample of level of confidence in set B1, is denoted as a, takes out sample a and F '1(a), it is put into
In A2；
Step 10 predicts all samples in u using disaggregated model F2 ', obtains prediction label F '2(u)；
Step 11, another counter i=1；
Step 12 uses S2 ∪ (ui,F′2(ui)) data set multitask deep neural network F2 obtains F "2；
Step 13 uses F '2With F "2Tally set S2 is predicted, calculates sample according to confidence calculations formula (2)
uiCorresponding level of confidence △2i:
Wherein, xjTo there is label data collection S2In sample, yjTo there is label data xjCorresponding true tag；
Record uiConfidence level, be denoted as (ui,△2i), it stores in set B2；
Step 14: i=i+1, judgement, if i≤n, otherwise return step 12 performs the next step；
Step 15 selects the highest sample of level of confidence in set B2, is denoted as b, takes out sample b and F '2(b), it puts
Enter in A1；
Step 10 six, more new data S1=S1 ∪ A2, S2=S2 ∪ A1；
Step 10 seven, judges whether S1 and S2 changes, and does not change, and exits, otherwise, return step three.
For two sorter models F1 and F2 employed in the present invention, the present invention is respectively adopted based on depth residual error net
Network and GoogleNet based on Inception module, go to train using data from two different angles, reach complementary effect
Fruit improves the precision of entire model.Classifier F1 is based on depth residual error network and carries out multi-task learning and classification to tongue picture.Relatively
For traditional Analysis of Lingual Picture technology, the neural network of the deep layer based on deep learning is meant to extract different stage
Image information can directly input information be detoured by introducing residual block in deep layer network and pass to output, in protection letter
While the integrality of breath, the disappearance of cause gradient or gradient explosion that deep layer network is easy to cause are avoided the problem that.Classifier F1
Using including 14 layer networks, the network architecture of 6 residual blocks.Whole network is all made of the convolution kernel of lesser 3x3.It specifically includes
1 input layer, 12 convolutional layers, 1 is fully connected layer and 1 output layer.Wherein first, in two residual blocks, convolution kernel
Number be 32, third, in four residual blocks, the number of convolution kernel is 64, the 5th, in six residual blocks, convolution kernel
Number is 128.
Comparison-of-pair sorting's device F2 carries out multitask classification to tongue picture using the GoogleNet based on Inception module.The net
Network obtains the image low-level features of different dimensions type using different filter operators (convolution, convergence of various sizes etc.), and
These low-level features are combined later, next layer network is allowed independently to select useful input, so that different rulers are arrived in study
The characteristics of image of degree.Classifier F2 uses the GoogleNet network architecture containing 4 Inception modules.Each
There is 1x1 in Inception module, tri- kinds of different filter operators of 3x3,5x5 are used to extract the characteristics of image of different scale.For
The multitask output for realizing classifier F1 and F2 activates replacement to pass in the last layer of deep neural network using sigmoid
The SoftMax activation of system network end-point:
Meanwhile two-value cross entropy is as classification cross entropy loss function:
Wherein C indicates cost, and x indicates that sample, y indicate physical tags value, and a indicates network output valve, and n indicates the total of sample
Number.
The preferred embodiment of the present invention has been described in detail above.It should be appreciated that those skilled in the art without
It needs creative work according to the present invention can conceive and makes many modifications and variations.Therefore, all technologies in the art
Personnel are available by logical analysis, reasoning, or a limited experiment on the basis of existing technology under this invention's idea
Technical solution, all should be within the scope of protection determined by the claims.
Claims (8)
1. a kind of multitask tongue picture automatic analysis method, which comprises the following steps:
Tongue picture picture is acquired, all images are divided into strong tally set and non label set according to the presence or absence of label；
Use the multitask deep neural network of strong two kinds of the tally set training different frameworks；
Use the multitask deep neural network of two kinds of the no label data alternative optimization different frameworks；
Tongue picture picture is input to two trained multitask deep neural networks, the two output results analyzed
Be averagely last tongue picture multitask analysis result.
2. a kind of multitask tongue picture automatic analysis method as described in claim 1, which is characterized in that the multitask depth mind
It include that the multitask disaggregated model based on depth residual error network is denoted as F1 and based on the deep neural network of GoogleNet through network
Multitask disaggregated model be denoted as F2.
3. a kind of multitask tongue picture automatic analysis method as claimed in claim 2, which is characterized in that use the strong tally set
The multitask deep neural network of two kinds of training different frameworks uses more of two kinds of no label data alternative optimization different frameworks
Business deep neural network, specifically includes the following steps:
By the tongue in the strong tally set as sample set is randomly divided into two parts, it is denoted as tally set S1 and tally set S2；It is arranged without mark
Sample set is signed, U is denoted as；Two empty sets are defined, are denoted as A1 and A2, A1 is for storing the unlabeled exemplars from depth residual error network
Set, A2 is for storing the high unlabeled exemplars set of the confidence level from GoogleNet network；Two empty sets are defined, are denoted as
B1 and B2, B1 are used for the confidence level storing unlabeled exemplars and its generating by depth residual error network, and B2 is for storing without label
Sample and its confidence level generated by GoogleNet network；Two sorter models are defined, depth residual error net is respectively based on
The disaggregated model of network is denoted as F1 and the disaggregated model of the deep neural network based on GoogleNet is denoted as F2；Define counter i.
4. a kind of multitask tongue picture automatic analysis method as claimed in claim 3, which is characterized in that tongue picture is input to two
The trained multitask deep neural network, averagely as last tongue picture more for the two output results analyzed
Business analysis is as a result, specifically include: tongue color of the multitask disaggregated model F1 based on depth residual error network to the tongue picture picture, tongue
Shape, tongue mind, coating nature are analyzed and predicted, and F2 pairs of the multitask disaggregated model of the deep neural network based on GoogleNet
Tongue color, ligulate, tongue mind, the coating nature of the tongue picture picture are analyzed and predicted.
5. a kind of multitask tongue picture automatic analysis method as claimed in claim 4, which is characterized in that use the no number of tags
Multitask deep neural network according to two kinds of the alternative optimization different frameworks includes: the confidence sample selecting no label data and concentrating
This optimization multitask disaggregated model F1 based on depth residual error network and the confidence sample for selecting no label data concentration are excellent
Change the multitask disaggregated model F2 based on GoogleNet deep neural network.
6. a kind of multitask tongue picture automatic analysis method as claimed in claim 5, which is characterized in that select no label data collection
In confidence sample optimization described in the multitask disaggregated model F1 based on depth residual error network, specifically include:
Disaggregated model F1 ' is obtained using tally set S1 training multitask deep neural network F1；
Integrate the sample for selecting size in U as n from unlabeled exemplars at random, is denoted as u={ u1, u2, u3...un}；
All samples in sample u are predicted using disaggregated model F1 ', obtain prediction label F '1(u)；
Enable counter i=1；
Use S1 ∪ (ui, F '1(ui)) data set multitask deep neural network F1 obtains F "1；
Use F '1With F "1Tally set S1 is predicted, sample u is calculated according to confidence calculations formula (1)iCorresponding confidence level
Horizontal Δ1i:
Wherein, xjTo there is label data collection S1In sample, yjTo there is label data xjCorresponding true tag.
Record uiConfidence level, be denoted as (ui, Δ1i), it stores in set B1；
I=i+1, judgement, if i≤n, otherwise return step six performs the next step；
The highest sample of level of confidence in set B1 is selected, a is denoted as, takes out sample a and F '1(a), it is put into empty set A2.
7. a kind of multitask tongue picture automatic analysis method as claimed in claim 5, which is characterized in that select no label data collection
In multitask disaggregated model F2 of the confidence sample optimization based on GoogleNet deep neural network, specifically include:
F2 ' is obtained using tally set S2 multitask deep neural network F1；
All samples in u are predicted using disaggregated model F2 ', obtain prediction label F '2(u)；
Enable counter i=1；
Use S2 ∪ (ui, F '2(ui)) data set multitask deep neural network F2 obtains F "2；
Use F '2With F "2Tally set S2 is predicted, calculates sample u according to confidence calculations formula (2)iCorresponding confidence level
Horizontal Δ2i:
Wherein, xjTo there is label data collection S2In sample, yjTo there is label data xjCorresponding true tag；
Record uiConfidence level, be denoted as (ui, Δ2i), it stores in set B2；
I=i+1, judgement, if i≤n, otherwise return step 12 performs the next step；
The highest sample of level of confidence in set B2 is selected, b is denoted as, takes out sample b and F '2(b), it is put into A1.
8. a kind of multitask tongue picture automatic analysis method as claimed in claims 6 or 7, which is characterized in that use the no mark
The multitask deep neural network of two kinds of data alternative optimization of the label different frameworks is further comprising the steps of:
More new data S1=S1 ∪ A2, S2=S2 ∪ A1；
Judge whether S1 and S2 changes, do not change, exit, otherwise, return selects size from unlabeled exemplars collection U at random
For the sample of n, it is denoted as u={ u1, u2, u3...un}；
Combine the multitask disaggregated model F1 and the base based on depth residual error network described in strong exemplar collection re -training
In the disaggregated model F2 of the deep neural network of GoogleNet.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910397988.2A CN110189305B (en) | 2019-05-14 | 2019-05-14 | Automatic analysis method for multitasking tongue picture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910397988.2A CN110189305B (en) | 2019-05-14 | 2019-05-14 | Automatic analysis method for multitasking tongue picture |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110189305A true CN110189305A (en) | 2019-08-30 |
| CN110189305B CN110189305B (en) | 2023-09-22 |
Family
ID=67716176
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910397988.2A Active CN110189305B (en) | 2019-05-14 | 2019-05-14 | Automatic analysis method for multitasking tongue picture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110189305B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110705425A (en) * | 2019-09-25 | 2020-01-17 | 广州西思数字科技有限公司 | Tongue picture multi-label classification learning method based on graph convolution network |
| CN111476259A (en) * | 2019-11-22 | 2020-07-31 | 上海大学 | Tooth mark tongue recognition algorithm based on convolutional neural network |
| CN111476260A (en) * | 2019-11-22 | 2020-07-31 | 上海大学 | Putrefaction classification algorithm based on convolutional neural network |
| CN113516634A (en) * | 2021-06-07 | 2021-10-19 | 北京博哥科技合伙企业(有限合伙) | Tongue picture teaching training device |
| CN113724228A (en) * | 2021-08-31 | 2021-11-30 | 平安科技（深圳）有限公司 | Tongue color and coating color identification method and device, computer equipment and storage medium |
| CN117392138A (en) * | 2023-12-13 | 2024-01-12 | 四川大学 | Tongue picture image processing method, storage medium and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107292330A (en) * | 2017-05-02 | 2017-10-24 | 南京航空航天大学 | A kind of iterative label Noise Identification algorithm based on supervised learning and semi-supervised learning double-point information |
| CN107437096A (en) * | 2017-07-28 | 2017-12-05 | 北京大学 | Image classification method based on the efficient depth residual error network model of parameter |
| CN108764281A (en) * | 2018-04-18 | 2018-11-06 | 华南理工大学 | A kind of image classification method learning across task depth network based on semi-supervised step certainly |
| CN109034205A (en) * | 2018-06-29 | 2018-12-18 | 西安交通大学 | Image classification method based on the semi-supervised deep learning of direct-push |
| CN109522961A (en) * | 2018-11-23 | 2019-03-26 | 中山大学 | A kind of semi-supervision image classification method based on dictionary deep learning |
-
2019
- 2019-05-14 CN CN201910397988.2A patent/CN110189305B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107292330A (en) * | 2017-05-02 | 2017-10-24 | 南京航空航天大学 | A kind of iterative label Noise Identification algorithm based on supervised learning and semi-supervised learning double-point information |
| CN107437096A (en) * | 2017-07-28 | 2017-12-05 | 北京大学 | Image classification method based on the efficient depth residual error network model of parameter |
| CN108764281A (en) * | 2018-04-18 | 2018-11-06 | 华南理工大学 | A kind of image classification method learning across task depth network based on semi-supervised step certainly |
| CN109034205A (en) * | 2018-06-29 | 2018-12-18 | 西安交通大学 | Image classification method based on the semi-supervised deep learning of direct-push |
| CN109522961A (en) * | 2018-11-23 | 2019-03-26 | 中山大学 | A kind of semi-supervision image classification method based on dictionary deep learning |
Non-Patent Citations (4)
| Title |
|---|
| WU HAO等: "Semi-Supervised Deep Learning Using Pseudo Labels for Hyperspectral image Classification", 《IEEE TRANSACTIONS ON IMAGE PROCESSING》 * |
| 李艳秋: "基于集成学习的人脸识别研究", 《中国知网博士电子期刊》 * |
| 汤一平等: "基于多任务卷积神经网络的舌象分类研究", 《计算机科学》 * |
| 王小鹏: "基于弱监督深度学习的图像检索技术研究与实现", 《中国知网硕士电子期刊》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110705425A (en) * | 2019-09-25 | 2020-01-17 | 广州西思数字科技有限公司 | Tongue picture multi-label classification learning method based on graph convolution network |
| CN110705425B (en) * | 2019-09-25 | 2022-06-28 | 广州西思数字科技有限公司 | Tongue picture multi-label classification method based on graph convolution network |
| CN111476259A (en) * | 2019-11-22 | 2020-07-31 | 上海大学 | Tooth mark tongue recognition algorithm based on convolutional neural network |
| CN111476260A (en) * | 2019-11-22 | 2020-07-31 | 上海大学 | Putrefaction classification algorithm based on convolutional neural network |
| CN111476260B (en) * | 2019-11-22 | 2023-07-21 | 上海大学 | Greasy fur classification algorithm based on convolutional neural network |
| CN113516634A (en) * | 2021-06-07 | 2021-10-19 | 北京博哥科技合伙企业(有限合伙) | Tongue picture teaching training device |
| CN113724228A (en) * | 2021-08-31 | 2021-11-30 | 平安科技（深圳）有限公司 | Tongue color and coating color identification method and device, computer equipment and storage medium |
| CN113724228B (en) * | 2021-08-31 | 2024-05-10 | 平安科技（深圳）有限公司 | Tongue color and tongue fur color identification method and device, computer equipment and storage medium |
| CN117392138A (en) * | 2023-12-13 | 2024-01-12 | 四川大学 | Tongue picture image processing method, storage medium and electronic equipment |
| CN117392138B (en) * | 2023-12-13 | 2024-02-13 | 四川大学 | Tongue picture image processing method, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110189305B (en) | 2023-09-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110189305A (en) | A kind of multitask tongue picture automatic analysis method | |
| Beevi et al. | Automatic mitosis detection in breast histopathology images using convolutional neural network based deep transfer learning | |
| Jiang et al. | White blood cells classification with deep convolutional neural networks | |
| CN106951825A (en) | A kind of quality of human face image assessment system and implementation method | |
| Ongun et al. | Feature extraction and classification of blood cells for an automated differential blood count system | |
| Colak et al. | Automated McIntosh-based classification of sunspot groups using MDI images | |
| Haehn et al. | Evaluating ‘graphical perception’with CNNs | |
| CN109815801A (en) | Face identification method and device based on deep learning | |
| CN109559300A (en) | Image processing method, electronic equipment and computer readable storage medium | |
| CN109543526A (en) | True and false facial paralysis identifying system based on depth difference opposite sex feature | |
| CN110139597A (en) | The system and method for being iterated classification using neuro-physiological signals | |
| CN109948647A (en) | A kind of electrocardiogram classification method and system based on depth residual error network | |
| WO2020224433A1 (en) | Target object attribute prediction method based on machine learning and related device | |
| Kerr et al. | Collaborative deep learning models to handle class imbalance in flowcam plankton imagery | |
| Li et al. | Dating ancient paintings of Mogao Grottoes using deeply learnt visual codes | |
| Ding et al. | Multiple lesions detection of fundus images based on convolution neural network algorithm with improved SFLA | |
| Feng et al. | Fine-tuning swin transformer and multiple weights optimality-seeking for facial expression recognition | |
| Selvia et al. | Skin lesion detection using feature extraction approach | |
| AU2021100367A4 (en) | A multi-task automatic analysis method for tongue manifestation | |
| Akbar et al. | Cluster-based learning from weakly labeled bags in digital pathology | |
| Townsend et al. | Discovering visual concepts and rules in convolutional neural networks | |
| CN113011436A (en) | Traditional Chinese medicine tongue color and fur color collaborative classification method based on convolutional neural network | |
| Sims et al. | A Neural Architecture for Detecting Confusion in Eye-tracking Data | |
| Harshini et al. | Machine Learning Approach for Various Eye Diseases using Modified Voting Classifier Model | |
| Lindsay et al. | Deep Learning Networks and Visual Perception |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |