CN113449776B - Deep learning-based Chinese herbal medicine identification method, device and storage medium - Google Patents

Deep learning-based Chinese herbal medicine identification method, device and storage medium Download PDF

Info

Publication number
CN113449776B
CN113449776B CN202110628514.1A CN202110628514A CN113449776B CN 113449776 B CN113449776 B CN 113449776B CN 202110628514 A CN202110628514 A CN 202110628514A CN 113449776 B CN113449776 B CN 113449776B
Authority
CN
China
Prior art keywords
herbal medicine
model
image set
preset
teacher
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110628514.1A
Other languages
Chinese (zh)
Other versions
CN113449776A (en
Inventor
郑禄
文晓国
龙文汉
帖军
徐胜舟
蓝佳宁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South Central Minzu University
Original Assignee
South Central University for Nationalities
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South Central University for Nationalities filed Critical South Central University for Nationalities
Priority to CN202110628514.1A priority Critical patent/CN113449776B/en
Publication of CN113449776A publication Critical patent/CN113449776A/en
Application granted granted Critical
Publication of CN113449776B publication Critical patent/CN113449776B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

The invention relates to the technical field of image recognition, and discloses a Chinese herbal medicine recognition method, a device and a storage medium based on deep learning, wherein the method comprises the following steps: extracting the characteristics of the herbal medicine picture to be identified to obtain the characteristic information of the herbal medicine to be processed; determining herbal category information according to the herbal characteristic information to be processed; inputting the characteristic information and the category information of the herbal medicine to be processed into a preset double-teacher distillation model to obtain the name and the confidence of the herbal medicine; and when the confidence coefficient of the herbal medicine is larger than a preset threshold value, extracting herbal medicine knowledge information corresponding to the herbal medicine names from a Chinese herbal medicine data database according to the herbal medicine names. Compared with the prior art, the identification of the Chinese herbal medicine is often realized manually, and researchers are required to have rich knowledge and experience of the Chinese herbal medicine, and the characteristic information and the category information of the Chinese herbal medicine to be processed are input into the preset double-teacher distillation model for Chinese herbal medicine identification and classification, so that the accuracy of Chinese herbal medicine identification is improved.

Description

Deep learning-based Chinese herbal medicine identification method, device and storage medium
Technical Field
The invention relates to the technical field of image recognition, in particular to a Chinese herbal medicine recognition method, device and storage medium based on deep learning.
Background
The national Chinese herbal medicine is taken as national treasure in China, and the inheritance development and the perfection of the knowledge base of the national Chinese herbal medicine require a great deal of manual support and knowledge accumulation of professionals. The identification and classification of traditional Chinese herbal medicines often need to be realized manually, and researchers are required to have quite abundant knowledge reserves and experiences of the Chinese herbal medicines. Meanwhile, the accuracy of the classification result is difficult to guarantee except for a longer time period in the whole process. Therefore, how to obtain the recognition result of the Chinese herbal medicine efficiently and accurately is a technical problem to be solved.
The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art.
Disclosure of Invention
The invention mainly aims to provide a Chinese herbal medicine recognition method, device and storage medium based on deep learning, and aims to solve the technical problem of how to efficiently and accurately acquire recognition results of Chinese herbal medicines.
In order to achieve the above object, the present invention provides a deep learning-based Chinese herbal medicine recognition method, which comprises the following steps:
acquiring a herbal medicine picture to be identified, and extracting the characteristics of the herbal medicine picture to be identified to obtain the characteristic information of the herbal medicine to be processed;
Determining herbal category information according to the herbal characteristic information to be processed;
inputting the characteristic information of the herbal medicine to be processed and the herbal medicine category information into a preset double-teacher distillation model to obtain the herbal medicine name and the herbal medicine confidence of the herbal medicine picture to be identified;
and when the herbal confidence coefficient is larger than a preset threshold value, extracting herbal knowledge information corresponding to the herbal names from a Chinese herbal medicine data database according to the herbal names.
Preferably, before the step of obtaining the herbal medicine characteristic information to be processed, the method further comprises the steps of:
acquiring a training image set corresponding to sample herbal medicines, traversing the training image set, and acquiring a traversed current training image;
transforming the current training image according to different image augmentation strategies to obtain a transformed image set;
clipping each transformed image in the transformed image set to obtain a clipped image set;
generating a clipping image set according to all obtained clipping image sets when the traversal is finished;
and constructing a preset double-teacher distillation model according to the training image set and the clipping image set.
Preferably, the step of constructing a preset dual teacher distillation model according to the training image set and the clipping image set includes:
inputting the training image set and the clipping image set into a first preset teacher model to obtain first prediction category probability distribution;
inputting the training image set and the clipping image set into a second preset teacher model to obtain second prediction category probability distribution;
determining a comprehensive category probability distribution from the first predicted category probability distribution and the second predicted category probability distribution;
inputting the training image set and the clipping image set into a preset student model to obtain third prediction category probability distribution;
and constructing a preset double-teacher distillation model according to the comprehensive category probability distribution and the third prediction category probability distribution.
Preferably, the step of constructing a preset double teacher distillation model according to the comprehensive category probability distribution and the third prediction category probability distribution includes:
determining a teacher soft label according to the comprehensive category probability distribution, and determining a teacher probability transformation value according to the teacher soft label and a preset temperature;
Determining a student probability transformation value according to the third prediction category probability distribution and the preset temperature;
and constructing a preset double-teacher distillation model according to the teacher probability transformation value and the student probability transformation value.
Preferably, the step of constructing a preset double-teacher distillation model according to the teacher probability transformation value and the student probability transformation value includes:
obtaining a soft tag loss function value through JS divergence according to the teacher probability transformation value and the student probability transformation value;
acquiring a hard tag of the preset student model, and determining a hard tag loss function value according to the hard tag and the third prediction category probability distribution;
and constructing a preset double-teacher distillation model according to the soft tag loss function value and the hard tag loss function value.
Preferably, the step of constructing a preset double-teacher distillation model according to the teacher probability transformation value and the student probability transformation value includes:
acquiring a loss weight value between the soft tag loss function value and the hard tag loss function value;
determining a comprehensive loss function value according to the soft tag loss function value, the hard tag loss function value and the loss weight value;
Training the preset student model according to the comprehensive loss function value and a preset learning descent strategy to obtain a preset double-teacher distillation model.
Preferably, the step of determining a composite loss function value from the soft tag loss function value, the hard tag loss function value, and the loss weight value includes:
calculating a comprehensive loss function value according to the soft tag loss function value, the hard tag loss function value and the loss weight value through a preset loss formula;
the preset loss formula is as follows:
wherein Loss is the value of the comprehensive Loss function, L soft For soft label loss function value, L hard For the value of the hard tag loss function,is a loss weight value.
In addition, in order to achieve the above object, the present invention also provides a deep learning-based Chinese herbal medicine recognition device, which comprises:
the acquisition module is used for acquiring herbal medicine pictures to be identified, and extracting the characteristics of the herbal medicine pictures to be identified to obtain the characteristic information of the herbal medicine to be processed;
the determining module is used for determining herbal category information according to the characteristic information of the herbal medicine to be processed;
the processing module is used for inputting the characteristic information of the herbal medicine to be processed and the information of the herbal medicine category into a preset double-teacher distillation model to obtain the herbal medicine name and the herbal medicine confidence of the herbal medicine picture to be identified;
And the identification module is used for extracting the herbal knowledge information corresponding to the herbal names from the Chinese herbal medicine data database according to the herbal names when the herbal confidence coefficient is larger than a preset threshold.
In addition, in order to achieve the above object, the present invention also provides a deep learning-based Chinese herbal medicine recognition apparatus, comprising: the device comprises a memory, a processor and a deep learning-based Chinese herbal medicine recognition program stored on the memory and capable of running on the processor, wherein the deep learning-based Chinese herbal medicine recognition program realizes the steps of the deep learning-based Chinese herbal medicine recognition method when being executed by the processor.
In addition, in order to achieve the above object, the present invention also proposes a storage medium having stored thereon a deep learning-based Chinese herbal medicine recognition program which, when executed by a processor, implements the steps of the deep learning-based Chinese herbal medicine recognition method as described above.
According to the method, firstly, a herbal medicine picture to be identified is obtained, characteristic extraction is carried out on the herbal medicine picture to be identified, characteristic information of herbal medicine to be processed is obtained, then herbal medicine category information is determined according to the characteristic information of the herbal medicine to be processed, the characteristic information of the herbal medicine to be processed and the herbal medicine category information are input into a preset double-teacher distillation model, herbal medicine names and herbal medicine confidence degrees are obtained, and finally when the herbal medicine confidence degrees are larger than a preset threshold value, herbal medicine knowledge information corresponding to the herbal medicine names is extracted from a Chinese herbal medicine data database according to the herbal medicine names. Compared with the prior art, the identification and classification of the Chinese herbal medicine are often realized manually, and researchers are required to have quite abundant knowledge and experience of the Chinese herbal medicine, and the characteristic information and the category information of the Chinese herbal medicine to be processed are input into the preset double-teacher distillation model for Chinese herbal medicine identification and classification, so that the accuracy of Chinese herbal medicine identification is improved.
Drawings
FIG. 1 is a schematic diagram of a deep learning based Chinese herbal medicine recognition device of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a first embodiment of a deep learning based Chinese herbal medicine recognition method of the present invention;
FIG. 3 is a schematic flow chart of a second embodiment of a deep learning based Chinese herbal medicine recognition method of the present invention;
fig. 4 is a block diagram showing the construction of a first embodiment of the deep learning-based herbal medicine recognition apparatus of the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a deep learning-based Chinese herbal medicine recognition device in a hardware operation environment according to an embodiment of the present invention.
As shown in fig. 1, the deep learning-based Chinese herbal medicine recognition apparatus may include: a processor 1001, such as a central processing unit (Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, a memory 1005. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display (Display), and the optional user interface 1003 may also include a standard wired interface, a wireless interface, and the wired interface for the user interface 1003 may be a USB interface in the present invention. The network interface 1004 may optionally include a standard wired interface, a WIreless interface (e.g., a WIreless-FIdelity (WI-FI) interface). The Memory 1005 may be a high-speed random access Memory (Random Access Memory, RAM) Memory or a stable Memory (NVM), such as a disk Memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
It will be appreciated by those skilled in the art that the structure shown in fig. 1 does not constitute a limitation of the deep learning based chinese herbal medicine recognition device, and may include more or less components than illustrated, or may combine certain components, or may be arranged in different components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and a deep learning-based herbal medicine recognition program may be included in a memory 1005, which is considered to be a type of computer storage medium.
In the deep learning-based Chinese herbal medicine recognition device shown in fig. 1, the network interface 1004 is mainly used for connecting a background server and performing data communication with the background server; the user interface 1003 is mainly used for connecting user equipment; the deep learning-based Chinese herbal medicine recognition apparatus calls the deep learning-based Chinese herbal medicine recognition program stored in the memory 1005 through the processor 1001 and performs the deep learning-based Chinese herbal medicine recognition method provided by the embodiment of the present invention.
Based on the hardware structure, the embodiment of the Chinese herbal medicine recognition method based on deep learning is provided.
Referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of a deep learning-based Chinese herbal medicine recognition method according to the present invention.
In a first embodiment, the deep learning-based Chinese herbal medicine recognition method includes the steps of:
step S10: and obtaining a herbal medicine picture to be identified, and carrying out feature extraction on the herbal medicine picture to be identified to obtain the feature information of the herbal medicine to be processed.
It should be noted that, the execution subject of the present embodiment is a deep learning based Chinese herbal medicine recognition device, where the device is a deep learning based Chinese herbal medicine recognition device having functions of image processing, data communication, program running, and the like, and may be other devices, which is not limited in this embodiment.
The herbal picture to be identified is selected by a user, the herbal picture to be identified can be a single herbal picture or a plurality of herbal pictures, the herbal picture to be identified can be processed by a preset double-teacher distillation model, and herbal characteristic information corresponding to the herbal picture to be identified is obtained, wherein the herbal characteristic information can be herbal contour information, shape information, color information and the like.
Before the step of obtaining the characteristic information of the herbal medicine to be processed, the training image set corresponding to the sample herbal medicine is required to be obtained, the training image set is traversed to obtain a traversed current training image, the current training image is transformed according to different image augmentation strategies to obtain a transformed image set, each transformed image in the transformed image set is cut to obtain a cut image set, a cut image set is generated according to all the obtained cut image sets when the traversing is finished, and a preset double-teacher distillation model is constructed according to the training image set and the cut image set.
In a specific implementation, to identify herbal materials, a web crawler technique is used to crawl 10 herbal data sets from a hundred degree gallery. And finally obtaining 1000 data sets through impurity removal and screening. The herbal medicine data set comprises 10 Chinese herbal medicines of white paeony root, radix stemonae, fructus aurantii, rhizoma polygonati, turmeric, pollen typhae, fructus cnidii, motherwort, medlar and radix curcumae. Each category contains 200 pictures (parameters 320px×320px,96dpi×96 dpi). Meanwhile, when training a teacher model (teacher network), the data set is as follows: 2:1 data set partitioning was performed, with 700 training sets (70%), 200 validation sets (20%), 100 test sets (10%). When training a preset student model (student network), the training set and the test set are combined and expanded, so that the effect of the knowledge distillation task is improved by using more data when knowledge distillation is performed later.
When a network model is trained, the data volume and the universality of data sets greatly influence the recognition capability and the actual performance of the model, meanwhile, the sample data of each data set type is less, and the problem of over fitting of a deep learning model is easily caused in limited training data. In order to reduce overfitting and improve generalization capability of a model, the model is trained by using a data enhancement technology, 4 image enhancement strategies are used before training the data, more complex data description is realized, normalization is carried out on the data under simulated real conditions under the condition that original data characteristics are maintained, and a data set is expanded to 10000 through the image enhancement strategies. The image augmentation strategy includes: 1) Image transformation class, using an unsupervised augmentation (randa) strategy, forgoes conventional manual selection augmentation strategies (e.g.: multiple sub-strategies such as rotation, overturn, clipping, contrast and the like) to amplify the image, and the same probability is set for all the sub-strategies to amplify the image, so that the effect that the multiple sub-strategies for amplifying act on a single image simultaneously through probability combination is achieved. Through the strategy, the factors such as brightness, contrast, saturation, hue and the like of the image can be adjusted simultaneously by using the random factors, the shooting angle and actual illumination difference in the actual shooting situation are simulated, the parameters are ensured to accord with the actual situation, the influence of the image angle and light is reduced, and the robustness is enhanced.
And clipping each conversion image in the conversion image set to obtain a clipping image set, wherein the image clipping mode can be that the data set is clipped by using three clipping strategies of clipping drawing tool (CutOut), random erasure data augmentation method (random erasure) and image clipping type (HideandSeek) at the same time, and the pixel value of a clipping region is set to 0 in different modes. The method is mainly used for simulating the classification situation when the main body is partially blocked in the real scene, and simultaneously preventing the model from being sensitive to the obvious region of the image, so that the overfitting phenomenon occurs.
Step S20: and determining herbal category information according to the characteristic information of the herbal medicine to be processed.
The herbal characteristic information may be herbal contour information, shape information, color information, etc. And then the shape information, the color information and the outline information can be matched to determine the herbal category information, wherein the herbal category information can be white paeony root, stemona root, bitter orange slice, rhizoma polygonati, turmeric, cattail pollen, cnidium, motherwort, medlar, turmeric and the like.
Step S30: inputting the characteristic information of the herbal medicine to be processed and the information of the herbal medicine category into a preset double-teacher distillation model to obtain the herbal medicine name and the herbal medicine confidence of the herbal medicine picture to be identified.
It should be understood that the step of constructing the preset dual-teacher distillation model includes inputting a training image set and a clipping image set to a first preset teacher model to obtain a first prediction class probability distribution, inputting the training image set and the clipping image set to a second preset teacher model to obtain a second prediction class probability distribution, determining an integrated class probability distribution according to the first prediction class probability distribution and the second prediction class probability distribution, determining a teacher soft tag according to the integrated class probability distribution, determining a teacher probability conversion value according to the teacher soft tag and a preset temperature, inputting the training image set and the clipping image set to a preset student model to obtain a third prediction class probability distribution, determining a student probability conversion value according to the third prediction class probability distribution and the preset temperature, obtaining a soft tag loss function value according to the teacher probability conversion value and the student probability conversion value through JS dispersion, obtaining a hard tag of the preset student model, obtaining a loss function value between the soft tag loss function value and the hard tag loss function value according to the hard tag loss function value, determining an integrated loss function value and a weight value according to the soft tag loss function value and the integrated loss and the weight value, and performing a preset learning and lowering strategy to the preset student model to obtain a dual-teacher distillation model.
It should be noted that the first preset teacher model may be a residual learning network model (res net), and the residual learning network model is mainly used for reducing the training burden of the network model, and solving the problem of degradation of the model performance caused by gradient disappearance/explosion with increasing network depth. The network constructs an identity mapping by fitting the forward propagating neural network of the stacked nonlinear layers to the mapping formula of the shortcut connection (shortcut connections), thereby ensuring that the deep-level network achieves the same performance as the shallow-level network. The ResNet_vd network is a fine-tuning of the ResNet network structure, and various residual module variants are proposed. The accuracy is significantly higher than other structural variants.
The embodiment applies the service set identification (Service Set Identifier, SSLD) technology to the network, and the model performance of the ResNet50_vd_ssld is superior to that of the original model network. Therefore, the ResNet50_vd_ssld obtained by distilling the ResNet50_vd model through SSLD is used as a teacher model in the SSLD scheme to distill the DenseNet model, so that the performance of the DenseNet model is improved.
It should be further noted that the second preset teacher model may be a dense convolutional neural network (DenseConvolutionalNetwork, denseNet), which has very good fitting resistance and speaking resistance compared to the res net model network, while achieving the same accuracy on the ImageNet classification dataset, the parameters of the classical image recognition model densene and the required computation amount are less than half of those of the res net.
In the mainstream deep convolutional network model, the performance of each model has reached a higher accuracy by using different technologies and continuously improving, and the current general trend is still to sacrifice the indexes of model volume, prediction speed and the like for achieving higher accuracy. The MobileNet series is a model which aims at the phenomenon and is small in size, quick to detect and capable of meeting the requirements of embedded equipment, the mobilenet_v3 is a lightweight network combining the characteristics of v1 and v2, and the most recent generation of improvement is made, and the lightweight network mainly has the following 4 characteristics: 1) A depth separable convolution (depthwise separable convolutions); 2) An inverse residual structure (the inverted residual with linear bottleneck) with linear bottlenecks; 3) A lightweight attention model; 4) The activation function h-swish replaces the swish function, so that a MobileNet_v3 model can be selected by presetting the student model.
It should be understood that the first predicted class probability distribution is that the training image set and the clipping image set are input into the first preset teacher model, the probability distribution about the image class is output, and the like; the second prediction category probability distribution is that a training image set and a clipping image set are input into a second preset teacher model, and probability distribution and the like about image categories are output; and the third prediction category probability distribution is that a training image set and a clipping image set are input into a preset student model, and probability distribution about image categories is output.
It should be further noted that, inputting the herbal medicine picture to be identified into the preset double teacher distillation model, the herbal medicine name and the herbal medicine confidence coefficient corresponding to the herbal medicine picture to be identified may be output, where the herbal medicine name and the herbal medicine confidence coefficient have a one-to-one correspondence.
Assuming that the herbal picture to be identified is input into a preset double-teacher distillation model, the herbal name and the herbal confidence corresponding to the herbal picture to be identified can be output, wherein the herbal name can be medlar and radix stemonae, the medlar confidence is 0.93, the radix stemonae confidence is 0.89 and the like.
Step S40: and when the herbal confidence coefficient is larger than a preset threshold value, extracting herbal knowledge information corresponding to the herbal names from a Chinese herbal medicine data database according to the herbal names.
The preset threshold may be user-defined, may be 0.8, may be 0.95, etc., and the embodiment is not limited.
In a specific implementation, after the model outputs the prediction result, the confidence of the output result is compared with a threshold value. If the threshold is lower, the user is required to re-capture the image. Otherwise, searching the Chinese herbal medicine data database for the Chinese herbal medicine details and returning the result, and simultaneously performing history preservation and the like on the mobile device.
In this embodiment, firstly, a herbal picture to be identified is obtained, feature extraction is performed on the herbal picture to be identified to obtain herbal feature information to be processed, then herbal category information is determined according to the herbal feature information to be processed, then the herbal feature information to be processed and the herbal category information are input into a preset double teacher distillation model to obtain herbal names and herbal confidence degrees, and finally herbal knowledge information corresponding to the herbal names is extracted from a Chinese herbal medicine data database according to the herbal names when the herbal confidence degrees are larger than a preset threshold value. Compared with the prior art, the identification and classification of the Chinese herbal medicine are often realized manually, so that researchers are required to have quite abundant knowledge and experience of the Chinese herbal medicine, and the characteristic information and the category information of the Chinese herbal medicine to be processed are input into a preset double-teacher distillation model for Chinese herbal medicine identification and classification, so that the accuracy of Chinese herbal medicine identification is improved.
In addition, referring to fig. 3, fig. 3 is a schematic diagram showing a first embodiment of a method for identifying Chinese herbal medicine based on deep learning according to the present invention.
In the second embodiment, before the step S10 in the deep learning-based Chinese herbal medicine recognition method, the method further includes:
Step S001: and acquiring a training image set corresponding to the sample herbal medicine, and traversing the training image set to obtain a traversed current training image.
To identify herbal materials, a web crawler technique was used to crawl 10 herbal data sets from a hundred degree gallery. And finally obtaining 1000 data sets through impurity removal and screening. The herbal medicine data set comprises 10 Chinese herbal medicines of white paeony root, radix stemonae, fructus aurantii, rhizoma polygonati, turmeric, pollen typhae, fructus cnidii, motherwort, medlar and radix curcumae. Each category contains 200 pictures (parameters 320px×320px,96dpi×96 dpi). Meanwhile, when training a teacher model (teacher network), the data set is as follows: 2:1, data set division is carried out, wherein 700 training sets (70%) are used, and when a preset student model (student network) is trained, the training sets and the test sets are combined and expanded.
Step S002: and transforming the current training image according to different image augmentation strategies to obtain a transformed image set.
When a network model is trained, the data volume and the universality of data sets greatly influence the recognition capability and the actual performance of the model, meanwhile, the sample data of each data set type is less, and the problem of over fitting of a deep learning model is easily caused in limited training data. In order to reduce overfitting and improve generalization capability of a model, the model is trained by using a data enhancement technology, 4 image enhancement strategies are used before training the data, more complex data description is realized, normalization is carried out on the data under simulated real conditions under the condition that original data characteristics are maintained, and a data set is expanded to 10000 through the image enhancement strategies. The image augmentation strategy includes: 1) Image transformation class, using an unsupervised augmentation (randa) strategy, forgoes conventional manual selection augmentation strategies (e.g.: multiple sub-strategies such as rotation, overturn, clipping, contrast and the like) to amplify the image, and the same probability is set for all the sub-strategies to amplify the image, so that the effect that the multiple sub-strategies for amplifying act on a single image simultaneously through probability combination is achieved. Through the strategy, the factors such as brightness, contrast, saturation, hue and the like of the image can be adjusted simultaneously by using the random factors, the shooting angle and actual illumination difference in the actual shooting situation are simulated, the parameters are ensured to accord with the actual situation, the influence of the image angle and light is reduced, and the robustness is enhanced.
Step S003: and clipping each transformation image in the transformation image set to obtain a clipping image set.
It should be noted that, the image clipping method may be to clip the data set by using three clipping policies of clipping tool (cut out), random erasing data augmentation method (random erasing) and image clipping class (HideAndSeek), and set the pixel value of the clipping region to 0 in different manners. The method is mainly used for simulating the classification situation when the main body is partially blocked in the real scene, and simultaneously preventing the model from being sensitive to the obvious region of the image, so that the overfitting phenomenon occurs.
Step S004: at the end of the traversal, a set of cropped image sets is generated from all the acquired cropped image sets.
The clipping image set sets are provided with a plurality of training image sets corresponding to the transformed and clipped sample herbal medicines.
Step S005: and constructing a preset double-teacher distillation model according to the training image set and the clipping image set.
A step of constructing a preset double-teacher distillation model according to a training image set and a cutting image set, wherein the training image set and the cutting image set are input into a first preset teacher model to obtain first prediction category probability distribution, the training image set and the cutting image set are input into a second preset teacher model to obtain second prediction category probability distribution, and comprehensive category probability distribution is determined according to the first prediction category probability distribution and the second prediction category probability distribution; and inputting the training image set and the clipping image set into a preset student model to obtain third prediction category probability distribution, and constructing a preset double-teacher distillation model according to the comprehensive category probability distribution and the third prediction category probability distribution.
It should be noted that the first preset teacher model may be a residual learning network model (res net), and the residual learning network model is mainly used for reducing the training burden of the network model, and solving the problem of degradation of the model performance caused by gradient disappearance/explosion with increasing network depth. The network constructs an identity mapping by fitting the forward propagating neural network of the stacked nonlinear layers to the mapping formula of the shortcut connection (shortcut connections), thereby ensuring that the deep-level network achieves the same performance as the shallow-level network. The ResNet_vd network is a fine-tuning of the ResNet network structure, and various residual module variants are proposed. The accuracy is significantly higher than other structural variants.
The embodiment applies the service set identification (Service Set Identifier, SSLD) technology to the network, and the model performance of the ResNet50_vd_ssld is superior to that of the original model network. Therefore, the ResNet50_vd_ssld obtained by distilling the ResNet50_vd model through SSLD is used as a teacher model in the SSLD scheme to distill the DenseNet model, so that the performance of the DenseNet model is improved.
It should be further noted that the second preset teacher model may be a dense convolutional neural network (DenseConvolutionalNetwork, denseNet), which has very good fitting resistance and speaking resistance compared to the res net model network, while achieving the same accuracy on the ImageNet classification dataset, the parameters of the classical image recognition model densene and the required computation amount are less than half of those of the res net.
In the mainstream deep convolutional network model, the performance of each model has reached a higher accuracy by using different technologies and continuously improving, and the current general trend is still to sacrifice the indexes of model volume, prediction speed and the like for achieving higher accuracy. The MobileNet series is a model which aims at the phenomenon and is small in size, quick to detect and capable of meeting the requirements of embedded equipment, the mobilenet_v3 is a lightweight network combining the characteristics of v1 and v2, and the most recent generation of improvement is made, and the lightweight network mainly has the following 4 characteristics: 1) A depth separable convolution (depthwise separable convolutions); 2) An inverse residual structure (the inverted residual with linear bottleneck) with linear bottlenecks; 3) A lightweight attention model; 4) The activation function h-swish replaces the swish function, so that a MobileNet_v3 model can be selected by presetting the student model.
Calculating comprehensive category probability distribution according to the first prediction category probability distribution and the second prediction category probability distribution through a preset comprehensive formula, wherein the preset comprehensive formula is as follows:
in Out teacher Out is a comprehensive class probability distribution teacher1 Out for the first predicted class probability distribution teacher2 For a second predictive category probability distribution.
The method comprises the steps of establishing a preset double-teacher distillation model according to the comprehensive category probability distribution and the third prediction category probability distribution, determining a teacher soft label according to the comprehensive category probability distribution, determining a teacher probability conversion value according to the teacher soft label and a preset temperature, determining a student probability conversion value according to the third prediction category probability distribution and the preset temperature, and establishing the preset double-teacher distillation model according to the teacher probability conversion value and the student probability conversion value.
The preset temperature is set by observing the recombination distribution condition of the model output probability under different T conditions, and when the value of T is smaller than 1, the difference between the real predicted value and the dark knowledge is enlarged, namely the specific gravity of the real prediction is emphasized. When the value of T is larger than 1, the whole prediction distribution is more gentle, namely the proportion of the dark knowledge is enlarged. Therefore, it is proposed that the experiment guess, in the early stage of training, is set to a value smaller than 1, so that the basic correct parameters can be found quickly by the preset student model in the early stage. And as the training degree is deepened, the proportion of the dark knowledge is continuously enlarged, so that the preset student model with high accuracy further learns the dark knowledge part in the correct prediction distribution given by the teacher model, thereby improving the accuracy rate. Based on this, the value of T is set to a function value that grows with the training process. Along with the experimental process, the preset temperature is calculated according to the current training times and the total training times of the preset student model through a preset temperature formula, the ratio of the current training times to the total training times of the preset student model is deepened continuously, the current training times and the total training times are increased in an S-shaped curve, and the main value range is [0,3], so that the preset student model can learn dark knowledge of different degrees in different processes.
The preset temperature formula is:
wherein T is a preset temperature, step is the current training times, and epochs is the total training times.
A step of constructing a preset double-teacher distillation model according to the teacher probability transformation value and the student probability transformation value, obtaining a soft tag loss function value through JS divergence according to the teacher probability transformation value and the student probability transformation value, obtaining a hard tag of the preset student model, determining a hard tag loss function value according to the hard tag and a third prediction category probability distribution, and constructing the preset double-teacher distillation model according to the soft tag loss function value and the hard tag loss function value, wherein the hard tag is a real data tag.
Calculating a preset hard tag loss function value according to the prediction probability distribution of the real tag and a preset student model through a preset function formula, wherein the preset function formula is as follows:
wherein CE (Label, prediction) is a cross entropy function of two probability distributions, label is a real Label, and prediction is a prediction probability distribution of a preset student model, L hard Out, which is a hard tag loss function value student And presetting a prediction probability distribution of the student model.
According to the teacher probability transformation value and the student probability transformation value, calculating a soft tag loss function value through a preset JS divergence formula, wherein the preset JS divergence formula is as follows:
Wherein L is soft Out, which is the soft tag loss function value teacher T is teacher probability transformation value, out student and/T is the student probability transformation value.
Constructing a preset double-teacher distillation model according to the teacher probability transformation value and the student probability transformation value, acquiring a loss weight value between a soft tag loss function value and a hard tag loss function value, determining a comprehensive loss function value according to the soft tag loss function value, the hard tag loss function value and the loss weight value, and training the preset student model according to the comprehensive loss function value and a preset learning descent strategy to acquire the preset double-teacher distillation model.
The loss weight value is combined with the fact that students in different learning stages, and teachers show different importance. In knowledge distillation, when the preset student model is in different training stages, the combination weights of the teacher model and the real labels should also be different. In the early stage of training, the learning fitting is mainly carried out by combining the transfer learning and the real label hard label, so that the high accuracy performance based on the pre-training model can be obtained in the training of the whole model. With the deepening of the training process, the accuracy cannot be further improved because the preset student model reaches a better convergence condition through self-learning (hard label), and then the specific gravity of the teacher model is continuously improved, so that the preset student model is turned to acquire the dark knowledge distribution in the prediction distribution of the teacher model. Thereby achieving the effect of improving the performance of the model.
Along with the experimental process, calculating a loss weight value according to the current training times and the total training times of a preset student model through a preset weight formula, wherein the preset weight formula is as follows:
in the method, in the process of the invention,for loss of weight, step is the current number of exercises and epochs is the total number of exercises.
Determining a comprehensive loss function value according to the soft tag loss function value, the hard tag loss function value and the loss weight value, and calculating the comprehensive loss function value according to a preset loss formula according to the soft tag loss function value, the hard tag loss function value and the loss weight value, wherein the preset loss formula is as follows:
wherein Loss is the value of the comprehensive Loss function, L soft For soft label loss function value, L hard For the value of the hard tag loss function,is a loss weight value.
The preset learning descent strategy comprises an exponential decay preheating strategy Exponential Warmup, a sectional decay strategy Pieceweise and a Cosine decay strategy Cosine, and in the exponential decay preheating strategy, the learning rate (a variable of the update degree of a control parameter is utilized when back propagation is carried out) mainly decays exponentially along with the training degree. At the same time, a learning rate preheating technology is used (when training is started, a smaller learning rate is used for training at first, then the learning rate is increased continuously until the preset learning rate is reached, and iteration is performed according to a set learning strategy); the sectional attenuation strategy is to set different learning rate constant values in corresponding intervals by presetting different training time intervals; in the cosine attenuation strategy, attenuation is carried out through a simulated cosine function, the learning rate firstly slowly drops along with the deepening of the training degree, then drops in an accelerating way, and then drops slowly again so as to form a cycle drop strategy. Referring to table 1, table 1 is a comparison table of the results of the mobilenet_v3_small_dtsd model parameter combinations. When other experimental parameters are the same, the DTSD distillation technology using the Exponential Warmup learning rate reduction strategy has the highest accuracy of 98.60%, and the worse the Pieccweise reduction strategy is, the more improved by 1.80%. Therefore, the learning rate reduction strategy of Exponential Warmup is considered to be more beneficial to the training model of the DTSD technology.
TABLE 1
In a specific implementation, in the whole dual-teacher supervised adaptive decay (DTSD), two teacher models are respectively selected as a complex and high-precision model res net50_vd and a Densenet121, and the final accuracy of the two teacher models is 98.90% and 98.70% respectively through model parameter adjustment and transfer learning. In order to meet the requirements of the Chinese herbal medicine recognition system in the preset student model, a lightweight network MobileNet_v3_smal is selected.
To reduce the experimental time and training costs, it is common to use a classical network that a learner has designed by spending a lot of time computing costs on a large-scale dataset such as ImageNet2012 (1600 tens of thousands of pictures), and can use its basic parameters to perform parameter initialization in the model to be explored instead of random initialization.
To explore the impact of the pre-training model on the performance of the training process and the final model in the training process, two teacher models ResNet50_vd, denseNet121 and a preset student model MoblieNet_v3_small are respectively trained based on whether the pre-training model is adopted or not, and the pre-training models are all similar models trained in advance on the ImageNet 2012. Taking mobilenet_v3_small as an example, the loss value in the training process by using the pre-training model has higher convergence speed than the loss value in the training process without using the training, and a more ideal convergence state is achieved in the early training period, so that the trained model added with the pre-training model has higher performance under the condition of training the same epochs. Meanwhile, the model Acc after the pre-training model is used always keeps high accuracy for the model without pre-training, and high performance is achieved in the early stage of training.
As shown in Table 2, table 2 shows the effect of the pre-training model, and the final accuracy of the three models pre-trained in Table 2 using the transfer learning was improved by 12.40%,7.35% and 9.5%, respectively. Therefore, the pre-training model can be considered to have an effect of optimizing the performance of the model, and in order to embody the superiority of the DTSD technology, the pre-training is performed by adopting the transfer learning in the later experiments.
TABLE 2
Through the experiments, the pre-training model is proved to play a role in enhancing the model performance. Therefore, in order to verify the effect of the DTSD technology on mobilenet_v3_small, a high-precision teacher model resnet50_vd (98.9%) and a Densenet121 (98.7%) were trained on the current experimental book in advance.
After training the optimal mobilenet_v3_small_dtsd model obtained by adjusting the learning rate-dropping strategy, it was compared longitudinally with mobilenet_v3_small using different techniques in order to prove its superiority. The comparison model is mainly as follows: 1) Mobilenet_v3_small without using a transfer learning pre-training model; 2) MobileNet_v3_small_Pre using a Pre-trained model 3) MobileNet_v3_small_SSLD trained using SSLD (semi-supervised tag knowledge distillation) technique, refer to Table 3, table 3 MobileNet_v3_small, each series of model results vs.
TABLE 3 Table 3
The experimental results are shown in table 3, and the results prove that under the condition that the training parameters are identical, the accuracy rate reaches 98.60% by the improved DTSD technology, and compared with the most original training model, the improvement is 11.15%, and compared with the SSLD technology which improves knowledge distillation as well, the improvement is 1.50%.
In this embodiment, firstly, a training image set corresponding to sample herbal medicine is obtained, the training image set is traversed to obtain a traversed current training image, then, the current training image is transformed according to different image augmentation strategies to obtain a transformed image set, each transformed image in the transformed image set is cut to obtain a cut image set, when the traversing is finished, a cut image set is generated according to all obtained cut image sets, finally, a preset double-teacher distillation model is built according to the training image set and the cut image set, and compared with the prior art, the identification and classification of the herbal medicine often need to be manually realized, and researchers are required to have quite abundant knowledge and experience of the herbal medicine. Meanwhile, the accuracy of the classification result is difficult to guarantee except for a longer time period in the whole process, and the preset double-teacher distillation model is built through the training image set and the cutting image set in the embodiment, so that the recognition speed and accuracy of the Chinese herbal medicine are improved.
In addition, the embodiment of the invention also provides a storage medium, wherein the storage medium is stored with a Chinese herbal medicine recognition program based on deep learning, and the Chinese herbal medicine recognition program based on deep learning realizes the steps of the Chinese herbal medicine recognition method based on deep learning.
In addition, referring to fig. 4, the embodiment of the invention further provides a deep learning-based Chinese herbal medicine recognition device, which comprises:
the acquisition module 4001 is used for acquiring a herbal medicine picture to be identified, and extracting characteristics of the herbal medicine picture to be identified to obtain characteristic information of the herbal medicine to be processed.
The herbal picture to be identified is selected by a user, the herbal picture to be identified can be a single herbal picture or a plurality of herbal pictures, the herbal picture to be identified can be processed by a preset double-teacher distillation model, and herbal characteristic information corresponding to the herbal picture to be identified is obtained, wherein the herbal characteristic information can be herbal contour information, shape information, color information and the like.
Before the step of obtaining the characteristic information of the herbal medicine to be processed, the training image set corresponding to the sample herbal medicine is required to be obtained, the training image set is traversed to obtain a traversed current training image, the current training image is transformed according to different image augmentation strategies to obtain a transformed image set, each transformed image in the transformed image set is cut to obtain a cut image set, a cut image set is generated according to all the obtained cut image sets when the traversing is finished, and a preset double-teacher distillation model is constructed according to the training image set and the cut image set.
In a specific implementation, to identify herbal materials, a web crawler technique is used to crawl 10 herbal data sets from a hundred degree gallery. And finally obtaining 1000 data sets through impurity removal and screening. The herbal medicine data set comprises 10 Chinese herbal medicines of white paeony root, radix stemonae, fructus aurantii, rhizoma polygonati, turmeric, pollen typhae, fructus cnidii, motherwort, medlar and radix curcumae. Each category contains 200 pictures (parameters 320px×320px,96dpi×96 dpi). Meanwhile, when training a teacher model (teacher network), the data set is as follows: 2:1 data set partitioning was performed, with 700 training sets (70%), 200 validation sets (20%), 100 test sets (10%). When training a preset student model (student network), the training set and the test set are combined and expanded, so that the effect of the knowledge distillation task is improved by using more data when knowledge distillation is performed later.
When a network model is trained, the data volume and the universality of data sets greatly influence the recognition capability and the actual performance of the model, meanwhile, the sample data of each data set type is less, and the problem of over fitting of a deep learning model is easily caused in limited training data. In order to reduce overfitting and improve generalization capability of a model, the model is trained by using a data enhancement technology, 4 image enhancement strategies are used before training the data, more complex data description is realized, normalization is carried out on the data under simulated real conditions under the condition that original data characteristics are maintained, and a data set is expanded to 10000 through the image enhancement strategies. The image augmentation strategy includes: 1) Image transformation class, using an unsupervised augmentation (randa) strategy, forgoes conventional manual selection augmentation strategies (e.g.: multiple sub-strategies such as rotation, overturn, clipping, contrast and the like) to amplify the image, and the same probability is set for all the sub-strategies to amplify the image, so that the effect that the multiple sub-strategies for amplifying act on a single image simultaneously through probability combination is achieved. Through the strategy, the factors such as brightness, contrast, saturation, hue and the like of the image can be adjusted simultaneously by using the random factors, the shooting angle and actual illumination difference in the actual shooting situation are simulated, the parameters are ensured to accord with the actual situation, the influence of the image angle and light is reduced, and the robustness is enhanced.
And clipping each conversion image in the conversion image set to obtain a clipping image set, wherein the image clipping mode can be that the data set is clipped by using three clipping strategies of clipping drawing tool (CutOut), random erasure data augmentation method (random erasure) and image clipping type (HideandSeek) at the same time, and the pixel value of a clipping region is set to 0 in different modes. The method is mainly used for simulating the classification situation when the main body is partially blocked in the real scene, and simultaneously preventing the model from being sensitive to the obvious region of the image, so that the overfitting phenomenon occurs.
A determining module 4002 for determining herbal category information based on the herbal characteristic information to be processed.
The herbal characteristic information may be herbal contour information, shape information, color information, etc. And then the shape information, the color information and the outline information can be matched to determine the herbal category information, wherein the herbal category information can be white paeony root, stemona root, bitter orange slice, rhizoma polygonati, turmeric, cattail pollen, cnidium, motherwort, medlar, turmeric and the like.
The processing module 4003 is configured to input the characteristic information of the herbal medicine to be processed and the information of the category of the herbal medicine into a preset double teacher distillation model, and obtain a herbal name and a herbal confidence of the herbal medicine picture to be identified.
It should be understood that the step of constructing the preset dual-teacher distillation model includes inputting a training image set and a clipping image set to a first preset teacher model to obtain a first prediction class probability distribution, inputting the training image set and the clipping image set to a second preset teacher model to obtain a second prediction class probability distribution, determining an integrated class probability distribution according to the first prediction class probability distribution and the second prediction class probability distribution, determining a teacher soft tag according to the integrated class probability distribution, determining a teacher probability conversion value according to the teacher soft tag and a preset temperature, inputting the training image set and the clipping image set to a preset student model to obtain a third prediction class probability distribution, determining a student probability conversion value according to the third prediction class probability distribution and the preset temperature, obtaining a soft tag loss function value according to the teacher probability conversion value and the student probability conversion value through JS dispersion, obtaining a hard tag of the preset student model, obtaining a loss function value between the soft tag loss function value and the hard tag loss function value according to the hard tag loss function value, determining an integrated loss function value and a weight value according to the soft tag loss function value and the integrated loss and the weight value, and performing a preset learning and lowering strategy to the preset student model to obtain a dual-teacher distillation model.
It should be noted that the first preset teacher model may be a residual learning network model (res net), and the residual learning network model is mainly used for reducing the training burden of the network model, and solving the problem of degradation of the model performance caused by gradient disappearance/explosion with increasing network depth. The network constructs an identity mapping by fitting the forward propagating neural network of the stacked nonlinear layers to the mapping formula of the shortcut connection (shortcut connections), thereby ensuring that the deep-level network achieves the same performance as the shallow-level network. The ResNet_vd network is a fine-tuning of the ResNet network structure, and various residual module variants are proposed. The accuracy is significantly higher than other structural variants.
The embodiment applies the service set identification (Service Set Identifier, SSLD) technology to the network, and the model performance of the ResNet50_vd_ssld is superior to that of the original model network. Therefore, the ResNet50_vd_ssld obtained by distilling the ResNet50_vd model through SSLD is used as a teacher model in the SSLD scheme to distill the DenseNet model, so that the performance of the DenseNet model is improved.
It should be further noted that the second preset teacher model may be a dense convolutional neural network (DenseConvolutionalNetwork, denseNet), which has very good fitting resistance and speaking resistance compared to the res net model network, while achieving the same accuracy on the ImageNet classification dataset, the parameters of the classical image recognition model densene and the required computation amount are less than half of those of the res net.
In the mainstream deep convolutional network model, the performance of each model has reached a higher accuracy by using different technologies and continuously improving, and the current general trend is still to sacrifice the indexes of model volume, prediction speed and the like for achieving higher accuracy. The MobileNet series is a model which aims at the phenomenon and is small in size, quick to detect and capable of meeting the requirements of embedded equipment, the mobilenet_v3 is a lightweight network combining the characteristics of v1 and v2, and the most recent generation of improvement is made, and the lightweight network mainly has the following 4 characteristics: 1) A depth separable convolution (depthwise separable convolutions); 2) An inverse residual structure (the inverted residual with linear bottleneck) with linear bottlenecks; 3) A lightweight attention model; 4) The activation function h-swish replaces the swish function, so that a MobileNet_v3 model can be selected by presetting the student model.
It should be understood that the first predicted class probability distribution is that the training image set and the clipping image set are input into the first preset teacher model, the probability distribution about the image class is output, and the like; the second prediction category probability distribution is that a training image set and a clipping image set are input into a second preset teacher model, and probability distribution and the like about image categories are output; and the third prediction category probability distribution is that a training image set and a clipping image set are input into a preset student model, and probability distribution about image categories is output.
It should be further noted that, inputting the herbal medicine picture to be identified into the preset double teacher distillation model, the herbal medicine name and the herbal medicine confidence coefficient corresponding to the herbal medicine picture to be identified may be output, where the herbal medicine name and the herbal medicine confidence coefficient have a one-to-one correspondence.
Assuming that the herbal picture to be identified is input into a preset double-teacher distillation model, the herbal name and the herbal confidence corresponding to the herbal picture to be identified can be output, wherein the herbal name can be medlar and radix stemonae, the medlar confidence is 0.93, the radix stemonae confidence is 0.89 and the like.
The identification module 4004 is configured to extract, from a database of Chinese herbal materials, information of knowledge of herbs corresponding to the herbal names according to the herbal names when the confidence level of the herbs is greater than a preset threshold.
The preset threshold may be user-defined, may be 0.8, may be 0.95, etc., and the embodiment is not limited.
In a specific implementation, after the model outputs the prediction result, the confidence of the output result is compared with a threshold value. If the threshold is lower, the user is required to re-capture the image. Otherwise, searching the Chinese herbal medicine data database for the Chinese herbal medicine details and returning the result, and simultaneously performing history preservation and the like on the mobile device.
In this embodiment, firstly, a herbal picture to be identified is obtained, feature extraction is performed on the herbal picture to be identified to obtain herbal feature information to be processed, then herbal category information is determined according to the herbal feature information to be processed, then the herbal feature information to be processed and the herbal category information are input into a preset double teacher distillation model to obtain herbal names and herbal confidence degrees, and finally herbal knowledge information corresponding to the herbal names is extracted from a Chinese herbal medicine data database according to the herbal names when the herbal confidence degrees are larger than a preset threshold value. Compared with the prior art, the identification and classification of the Chinese herbal medicine are often realized manually, so that researchers are required to have quite abundant knowledge and experience of the Chinese herbal medicine, and the characteristic information and the category information of the Chinese herbal medicine to be processed are input into a preset double-teacher distillation model for Chinese herbal medicine identification and classification, so that the accuracy of Chinese herbal medicine identification is improved.
Other embodiments or specific implementation manners of the deep learning-based Chinese herbal medicine recognition device of the present invention may refer to the above method embodiments, and will not be described herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the terms first, second, third, etc. do not denote any order, but rather the terms first, second, third, etc. are used to interpret the terms as names.
From the above description of embodiments, it will be clear to a person skilled in the art that the above embodiment method may be implemented by means of software plus a necessary general hardware platform, but may of course also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. read only memory mirror (Read Only Memory image, ROM)/random access memory (Random Access Memory, RAM), magnetic disk, optical disk), comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.

Claims (4)

1. The Chinese herbal medicine recognition method based on the deep learning is characterized by comprising the following steps of:
acquiring a herbal medicine picture to be identified, and extracting the characteristics of the herbal medicine picture to be identified to obtain the characteristic information of the herbal medicine to be processed;
determining herbal category information according to shape information, color information and contour information in the herbal characteristic information to be processed;
inputting the characteristic information of the herbal medicine to be processed and the herbal medicine category information into a preset double-teacher distillation model to obtain the herbal medicine name and the herbal medicine confidence of the herbal medicine picture to be identified;
when the herbal confidence coefficient is larger than a preset threshold value, extracting herbal knowledge information corresponding to the herbal names from a Chinese herbal medicine data database according to the herbal names;
the step of obtaining the herbal medicine picture to be identified, extracting the characteristics of the herbal medicine picture to be identified, and before the step of obtaining the characteristic information of the herbal medicine to be processed, further comprises the steps of:
Acquiring a training image set corresponding to sample herbal medicines, traversing the training image set, and acquiring a traversed current training image;
transforming the current training image according to different image augmentation strategies to obtain a transformed image set;
clipping each transformed image in the transformed image set to obtain a clipped image set;
generating a clipping image set according to all obtained clipping image sets when the traversal is finished;
inputting the training image set and the clipping image set into a ResNet50_vd_ssld model to obtain a first prediction category probability distribution;
inputting the training image set and the clipping image set into a DenseNet model to obtain a second prediction category probability distribution;
determining a comprehensive category probability distribution from the first predicted category probability distribution and the second predicted category probability distribution;
determining a teacher soft label according to the comprehensive category probability distribution, and determining a teacher probability transformation value according to the teacher soft label and a preset temperature;
inputting the training image set and the clipping image set into a MobileNet_v3 model to obtain third prediction category probability distribution;
determining a student probability transformation value according to the third prediction category probability distribution and the preset temperature;
Obtaining a soft tag loss function value through JS divergence according to the teacher probability transformation value and the student probability transformation value;
acquiring a hard tag of the MobileNet_v3 model, and determining a hard tag loss function value according to the hard tag and the third prediction category probability distribution;
acquiring a loss weight value between the soft tag loss function value and the hard tag loss function value, and calculating a comprehensive loss function value according to the soft tag loss function value, the hard tag loss function value and the loss weight value through a preset loss formula;
the preset loss formula is as follows:
；
wherein Loss is the value of the comprehensive Loss function, L soft For soft label loss function value, L hard For the value of the hard tag loss function,is a loss weight value;
and training the MobileNet_v3 model according to the comprehensive loss function value and a preset learning descent strategy to obtain a preset double-teacher distillation model.
2. The Chinese herbal medicine recognition device based on the deep learning is characterized by comprising:
the acquisition module is used for acquiring herbal medicine pictures to be identified, and extracting the characteristics of the herbal medicine pictures to be identified to obtain the characteristic information of the herbal medicine to be processed;
The determining module is used for determining herbal category information according to shape information, color information and contour information in the herbal characteristic information to be processed;
the processing module is used for inputting the characteristic information of the herbal medicine to be processed and the information of the herbal medicine category into a preset double-teacher distillation model to obtain the herbal medicine name and the herbal medicine confidence of the herbal medicine picture to be identified;
the recognition module is used for extracting herbal knowledge information corresponding to the herbal names from a Chinese herbal medicine data database according to the herbal names when the herbal confidence coefficient is larger than a preset threshold;
the processing module is also used for acquiring a training image set corresponding to the sample herbal medicine, traversing the training image set and acquiring a traversed current training image; transforming the current training image according to different image augmentation strategies to obtain a transformed image set; clipping each transformed image in the transformed image set to obtain a clipped image set; generating a clipping image set according to all obtained clipping image sets when the traversal is finished; inputting the training image set and the clipping image set into a ResNet50_vd_ssld model to obtain a first prediction category probability distribution; inputting the training image set and the clipping image set into DenseNet to obtain second prediction category probability distribution; determining a comprehensive category probability distribution from the first predicted category probability distribution and the second predicted category probability distribution; determining a teacher soft label according to the comprehensive category probability distribution, and determining a teacher probability transformation value according to the teacher soft label and a preset temperature; inputting the training image set and the clipping image set into a MobileNet_v3 model to obtain third prediction category probability distribution; determining a student probability transformation value according to the third prediction category probability distribution and the preset temperature; obtaining a soft tag loss function value through JS divergence according to the teacher probability transformation value and the student probability transformation value; acquiring a hard tag of the MobileNet_v3 model, and determining a hard tag loss function value according to the hard tag and the third prediction category probability distribution; acquiring a loss weight value between the soft tag loss function value and the hard tag loss function value, and calculating a comprehensive loss function value according to the soft tag loss function value, the hard tag loss function value and the loss weight value through a preset loss formula;
The preset loss formula is as follows:
；
wherein Loss is the value of the comprehensive Loss function, L soft For soft label loss function value, L hard For the value of the hard tag loss function,is a loss weight value;
and training the MobileNet_v3 model according to the comprehensive loss function value and a preset learning descent strategy to obtain a preset double-teacher distillation model.
3. Deep learning-based Chinese herbal medicine recognition equipment, characterized by, the chinese herbal medicine recognition equipment based on deep learning includes: the device comprises a memory, a processor and a deep learning based Chinese herbal medicine recognition program stored on the memory and capable of running on the processor, wherein the deep learning based Chinese herbal medicine recognition program realizes the steps of the deep learning based Chinese herbal medicine recognition method as claimed in claim 1 when being executed by the processor.
4. A storage medium having stored thereon a deep learning based chinese herbal medicine recognition program which when executed by a processor performs the steps of the deep learning based chinese herbal medicine recognition method of claim 1.
CN202110628514.1A 2021-06-04 2021-06-04 Deep learning-based Chinese herbal medicine identification method, device and storage medium Active CN113449776B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110628514.1A CN113449776B (en) 2021-06-04 2021-06-04 Deep learning-based Chinese herbal medicine identification method, device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110628514.1A CN113449776B (en) 2021-06-04 2021-06-04 Deep learning-based Chinese herbal medicine identification method, device and storage medium

Publications (2)

Publication Number Publication Date
CN113449776A CN113449776A (en) 2021-09-28
CN113449776B true CN113449776B (en) 2023-07-25

Family

ID=77810896

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110628514.1A Active CN113449776B (en) 2021-06-04 2021-06-04 Deep learning-based Chinese herbal medicine identification method, device and storage medium

Country Status (1)

Country Link
CN (1) CN113449776B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114821572B (en) * 2022-03-11 2023-04-21 德阳市人民医院 Deep learning oral pill identification method based on multi-view and data expansion
CN115909221A (en) * 2023-02-16 2023-04-04 江西博微新技术有限公司 Image recognition method, system, computer device and readable storage medium

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111639710A (en) * 2020-05-29 2020-09-08 北京百度网讯科技有限公司 Image recognition model training method, device, equipment and storage medium

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018126213A1 (en) * 2016-12-30 2018-07-05 Google Llc Multi-task learning using knowledge distillation
CN106803074A (en) * 2017-01-12 2017-06-06 司马大大（北京）智能系统有限公司 Chinese herbal medicine recognition methods and device
US10713540B2 (en) * 2017-03-07 2020-07-14 Board Of Trustees Of Michigan State University Deep learning system for recognizing pills in images
US20180268292A1 (en) * 2017-03-17 2018-09-20 Nec Laboratories America, Inc. Learning efficient object detection models with knowledge distillation
CN110399929B (en) * 2017-11-01 2023-04-28 腾讯科技（深圳）有限公司 Fundus image classification method, fundus image classification apparatus, and computer-readable storage medium
CN110008842A (en) * 2019-03-09 2019-07-12 同济大学 A kind of pedestrian's recognition methods again for more losing Fusion Model based on depth
AU2019100352A4 (en) * 2019-04-04 2019-05-09 Duan, Zicheng Mr A Flower-Image Search System Based on Deep Learning
CN110223281B (en) * 2019-06-06 2022-06-28 东北大学 Pulmonary nodule image classification method when data set contains uncertain data
CN110852360A (en) * 2019-10-30 2020-02-28 腾讯科技（深圳）有限公司 Image emotion recognition method, device, equipment and storage medium
CN111814862A (en) * 2020-06-30 2020-10-23 平安国际智慧城市科技股份有限公司 Fruit and vegetable identification method and device
CN111860537B (en) * 2020-07-17 2021-07-02 中南民族大学 Deep learning-based green citrus identification method, equipment and device
CN112036499A (en) * 2020-09-04 2020-12-04 西南民族大学 Traditional Chinese medicine identification method based on convolutional neural network
CN112232397A (en) * 2020-09-30 2021-01-15 上海眼控科技股份有限公司 Knowledge distillation method and device of image classification model and computer equipment
CN112418343B (en) * 2020-12-08 2024-01-05 中山大学 Multi-teacher self-adaptive combined student model training method
CN112784964A (en) * 2021-01-27 2021-05-11 西安电子科技大学 Image classification method based on bridging knowledge distillation convolution neural network

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111639710A (en) * 2020-05-29 2020-09-08 北京百度网讯科技有限公司 Image recognition model training method, device, equipment and storage medium

Also Published As

Publication number Publication date
CN113449776A (en) 2021-09-28

Similar Documents

Publication Publication Date Title
CN111427995B (en) Semantic matching method, device and storage medium based on internal countermeasure mechanism
CN113449776B (en) Deep learning-based Chinese herbal medicine identification method, device and storage medium
CN109902665A (en) Similar face retrieval method, apparatus and storage medium
CN106203625B (en) A kind of deep-neural-network training method based on multiple pre-training
CN109783817A (en) A kind of text semantic similarity calculation model based on deeply study
CN108961245A (en) Picture quality classification method based on binary channels depth parallel-convolution network
CN108804677A (en) In conjunction with the deep learning question classification method and system of multi-layer attention mechanism
CN107239733A (en) Continuous hand-written character recognizing method and system
CN112733602B (en) Relation-guided pedestrian attribute identification method
CN112464865A (en) Facial expression recognition method based on pixel and geometric mixed features
CN112613581A (en) Image recognition method, system, computer equipment and storage medium
CN108121975A (en) A kind of face identification method combined initial data and generate data
CN109582782A (en) A kind of Text Clustering Method based on Weakly supervised deep learning
CN113011253B (en) Facial expression recognition method, device, equipment and storage medium based on ResNeXt network
CN108492301A (en) A kind of Scene Segmentation, terminal and storage medium
CN113592007A (en) Knowledge distillation-based bad picture identification system and method, computer and storage medium
CN111694977A (en) Vehicle image retrieval method based on data enhancement
Yang A CNN-based broad learning system
CN113205103A (en) Lightweight tattoo detection method
CN114283345A (en) Small sample city remote sensing image information extraction method based on meta-learning and attention
CN112101364A (en) Semantic segmentation method based on parameter importance incremental learning
CN109934239B (en) Image feature extraction method
CN115457332A (en) Image multi-label classification method based on graph convolution neural network and class activation mapping
CN115049814A (en) Intelligent eye protection lamp adjusting method adopting neural network model
CN110399547A (en) For updating the method, apparatus, equipment and storage medium of model parameter

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant