CN109637660B - Tongue diagnosis analysis method and system based on deep convolutional neural network - Google Patents
Tongue diagnosis analysis method and system based on deep convolutional neural network Download PDFInfo
- Publication number
- CN109637660B CN109637660B CN201811558698.3A CN201811558698A CN109637660B CN 109637660 B CN109637660 B CN 109637660B CN 201811558698 A CN201811558698 A CN 201811558698A CN 109637660 B CN109637660 B CN 109637660B
- Authority
- CN
- China
- Prior art keywords
- tongue
- positioning
- model
- analysis
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 81
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 38
- 238000003745 diagnosis Methods 0.000 title claims abstract description 28
- 238000001514 detection method Methods 0.000 claims abstract description 25
- 238000000034 method Methods 0.000 claims abstract description 22
- 238000007781 pre-processing Methods 0.000 claims abstract description 14
- 206010043946 Tongue conditions Diseases 0.000 claims abstract description 13
- 238000004364 calculation method Methods 0.000 claims abstract description 12
- 238000005520 cutting process Methods 0.000 claims abstract description 10
- 238000012549 training Methods 0.000 claims description 44
- 238000013135 deep learning Methods 0.000 claims description 15
- 239000011248 coating agent Substances 0.000 claims description 11
- 238000000576 coating method Methods 0.000 claims description 11
- 238000013526 transfer learning Methods 0.000 claims description 8
- 238000004422 calculation algorithm Methods 0.000 abstract description 11
- 230000000694 effects Effects 0.000 abstract description 3
- 239000003814 drug Substances 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 210000001835 viscera Anatomy 0.000 description 5
- 201000010099 disease Diseases 0.000 description 4
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 4
- 238000007689 inspection Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000002405 diagnostic procedure Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000035943 smell Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 239000013078 crystal Substances 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 239000002932 luster Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 210000000952 spleen Anatomy 0.000 description 1
- 210000002784 stomach Anatomy 0.000 description 1
- 208000011580 syndromic disease Diseases 0.000 description 1
- 229940126680 traditional chinese medicines Drugs 0.000 description 1
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The invention discloses a tongue diagnosis analysis method and a tongue diagnosis analysis system based on a deep convolutional neural network, wherein the method comprises the following steps: acquiring an original image, and cutting the original image according to a preset region of interest to obtain an ROI image; judging whether the ROI image contains tongue according to a tongue detection model; according to the tongue positioning model, positioning the tongue to obtain a positioned tongue region image; according to the tongue analysis model, analyzing tongue conditions in the tongue region image, and outputting tongue analysis results; according to the method and the system, the time consumption of the algorithm is reduced by using a preprocessing link, the performance of the algorithm is improved by using a lightweight deep convolutional neural network, and the calculation efficiency is further improved by reducing the size of an SSD positioning target in tongue positioning; according to the method and the system, the effect and the performance of tongue diagnosis analysis are improved through multiple angles, so that the method and the system can still be effectively used in tongue diagnosis equipment with poor performance, and the timeliness of tongue diagnosis analysis is improved.
Description
Technical Field
The invention relates to the field of medical image analysis, in particular to a tongue diagnosis analysis method and system based on a deep convolutional neural network.
Background
The traditional Chinese medicine is the treasure of our Chinese nationality, and is the intelligent crystal which is continuously perfected by many generations of people for thousands of years. With the development of the times and the progress of society and the deep penetration of the concept of treating the disease in traditional Chinese medicine, the combination of traditional Chinese medicine and modern technology produces a series of modern achievements. Besides modern extraction and preparation of traditional Chinese medicines, diagnostic methods of traditional Chinese medicine are also developing towards automation and digitization. As described in ancient and modern medical systems: the looking at smells to ask and ask four words is the compendium of medical science. "the hope and smell inquiry constitutes four diagnostic methods of traditional Chinese medicine diagnosis. "Lingqiu. Benzang" in: "to see the external appearance, to know the viscera, it is sufficient to know the disease. "inspection of the skin is known to have a very important role. Inspection can be classified into facial inspection and lingual inspection. Tongue-distinguishing guide (b): the differentiation of the tongue nature can differentiate the deficiency and excess of viscera, and the observation of the tongue coating can examine the shallow and deep of six excesses. The tongue is the seedling of the heart, the spleen is the exterior syndrome, and the coating is produced by stomach qi. Viscera are connected with the tongue through meridians, and viscera lesions can be reflected on the tongue body and tongue coating. The tongue diagnosis mainly diagnoses the morphology, color and luster of the tongue and tongue coating, and so on, and thus determines the nature of the disease, the depth of the disease, the abundance or insufficiency of qi and blood, and the deficiency or excess of viscera. With the gradual development of image processing technology in the recent years, the artificial intelligence technology such as machine learning, deep learning and the like is mature, and the deep convolutional neural network is applied to tongue diagnosis of traditional Chinese medicine and generates various methods.
The existing technology based on the deep convolutional neural network is very time-consuming in overall application due to the use of algorithms such as an image semantic segmentation algorithm, and poor in timeliness of tongue diagnosis analysis, so that the time length for reaction and analysis is poor when the technology is applied to tongue diagnosis equipment with poor performance.
Disclosure of Invention
In order to solve the problems of time consumption and analysis time length and poor user experience of the existing tongue diagnosis analysis algorithm based on the deep convolutional neural network in the background technology, the invention provides a tongue diagnosis analysis method and system based on the deep convolutional neural network; the method and the system reduce the algorithm time consumption by using a preprocessing link, and improve the algorithm performance by using a lightweight deep convolutional neural network, and the tongue diagnosis analysis method based on the deep convolutional neural network comprises the following steps:
acquiring an original image, and cutting the original image according to a preset region of interest to obtain an ROI image;
judging whether the ROI image contains tongue according to a tongue detection model;
according to the tongue positioning model, positioning the tongue to obtain a positioned tongue region image;
according to the tongue analysis model, analyzing tongue conditions in the tongue region image, and outputting tongue analysis results;
the tongue detection model, the tongue positioning model and the tongue analysis model are obtained through training in advance by a lightweight deep convolutional neural network;
further, before clipping the original image according to the preset region of interest to obtain the ROI image, the method further includes:
preprocessing the original image; the pretreatment mode comprises white balance adjustment;
the white balance adjustment mode comprises fixed adjustment according to preset white balance parameters and automatic white balance adjustment according to preset rules.
Further, the preset region of interest is set as a fixed size region in the middle of the original image frame.
Furthermore, the tongue is positioned according to an SSD network frame based on a lightweight deep convolutional neural network;
performing positioning target calculation by limiting the positioning target size of the SSD network frame;
the positioning target calculation only calculates target positioning layers with smaller number of four candidate areas of the SSD network frame; the target positioning layer with the smaller number of candidate areas comprises Conv8_2, conv9_2, conv10_2 and Conv11_2.
Further, the tongue analysis model is multiple and corresponds to multiple tongue characteristics; the tongue features include tooth trace, prick, putrefaction, thickness, crack, fatness, thin, red tongue edge tip, tongue color, and tongue coating color;
the tongue analysis models are obtained through SquezeNet deep convolutional neural network training under a Caffe deep learning framework;
according to tongue condition analysis requirements, the multiple tongue analysis models are invoked through a dnn module of OpenCV.
Further, the tongue detection model, the tongue positioning model and the tongue analysis model are obtained by training in a set SqueEzeNet pre-training model in a mode of transfer learning under a Caffe deep learning frame;
the training samples of the tongue detection model and the tongue positioning model are a plurality of pre-acquired ROI images containing tongue; the training samples of the tongue analysis model are a plurality of pre-labeled tongue region images.
The tongue diagnosis analysis system based on the deep convolutional neural network comprises:
the ROI image acquisition unit is used for acquiring an original image, cutting the original image according to a preset region of interest and acquiring an ROI image;
the tongue detection unit is used for receiving the ROI image of the ROI image acquisition unit and judging whether the ROI image contains a tongue or not according to a tongue detection model;
the tongue positioning unit is used for positioning the tongue according to the tongue positioning model to obtain a positioned tongue region image;
the tongue analysis unit is used for analyzing tongue conditions in the tongue region image according to the tongue analysis model and outputting tongue analysis results;
the model training unit is used for obtaining the tongue detection model, the tongue positioning model and the tongue analysis model through training in advance by a lightweight deep convolutional neural network.
Further, the system also comprises a preprocessing unit, wherein the preprocessing unit is used for preprocessing the original image; the pretreatment mode comprises white balance adjustment;
the white balance adjustment mode comprises fixed adjustment according to preset white balance parameters and automatic white balance adjustment according to preset rules.
Further, the preset region of interest clipped by the ROI image acquisition unit is set as a fixed-size region in the middle of the original image frame.
Furthermore, the tongue positioning unit is used for positioning the tongue according to an SSD network frame on the basis of a lightweight deep convolutional neural network;
the tongue positioning unit is used for calculating a positioning target by limiting the positioning target size of the SSD network frame;
the positioning target calculation only calculates target positioning layers with smaller number of four candidate areas of the SSD network frame; the target positioning layer with the smaller number of candidate areas comprises Conv8_2, conv9_2, conv10_2 and Conv11_2.
Further, the tongue analysis model is multiple and corresponds to multiple tongue characteristics; the tongue features include tooth trace, prick, putrefaction, thickness, crack, fatness, thin, red tongue edge tip, tongue color, and tongue coating color;
the tongue analysis models are obtained through SquezeNet deep convolutional neural network training under a Caffe deep learning framework;
and the tongue analysis unit is used for calling the tongue analysis models through a dnn module of OpenCV according to tongue condition analysis requirements.
Further, the model training unit is used for training in the set SqueezeNet pre-training model in a mode of transfer learning under the Caffe deep learning frame;
the training samples of the tongue detection model and the tongue positioning model are a plurality of pre-acquired ROI images containing tongue; the training samples of the tongue analysis model are a plurality of pre-labeled tongue region images.
The beneficial effects of the invention are as follows: the technical scheme of the invention provides a tongue diagnosis analysis method and a tongue diagnosis analysis system based on a deep convolutional neural network; the method and the system reduce the algorithm time consumption by using a preprocessing link, ensure the tongue positioning and classifying effects by using transfer learning, and improve the algorithm performance by using a lightweight deep convolutional neural network; according to the method and the system, the effect and the performance of tongue diagnosis analysis are improved through multiple angles, so that the method and the system can still be effectively used in tongue diagnosis equipment with poor performance, and the timeliness of tongue diagnosis analysis is improved.
Drawings
Exemplary embodiments of the present invention may be more completely understood in consideration of the following drawings:
FIG. 1 is a flow chart of a tongue diagnosis analysis method based on a deep convolutional neural network according to an embodiment of the present invention;
fig. 2 is a block diagram of a tongue diagnosis analysis system based on a deep convolutional neural network according to an embodiment of the present invention.
Detailed Description
The exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, however, the present invention may be embodied in many different forms and is not limited to the examples described herein, which are provided to fully and completely disclose the present invention and fully convey the scope of the invention to those skilled in the art. The terminology used in the exemplary embodiments illustrated in the accompanying drawings is not intended to be limiting of the invention. In the drawings, like elements/components are referred to by like reference numerals.
Unless otherwise indicated, terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. In addition, it will be understood that terms defined in commonly used dictionaries should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense.
FIG. 1 is a flow chart of a tongue diagnosis analysis method based on a deep convolutional neural network according to an embodiment of the present invention; as shown in fig. 1, the method includes:
step 110, acquiring an original image, and cutting the original image according to a preset region of interest to obtain an ROI image;
the original image is a picture taken by a patient facing the camera and extending the tongue part, and the tongue part of the patient is displayed in the middle area of the image picture;
the preset region of interest is set as a fixed size region in the middle of the original image picture. The tongue is displayed in the preset area, namely the ROI image, by setting a fixed size area and requiring the patient to take a picture;
when a patient takes a tongue photo, the patient is required to take the photo which only displays the tongue as much as possible, so that the tongue photo is not easy to realize, inconvenient to take, and other influencing factors can be brought due to the conditions of shooting angle, head light blocking and the like; therefore, by setting the fixed-size area and cutting the fixed-size area of the original image, the image containing tongue can be effectively obtained, and the simple cutting process does not bring extra calculation burden; in this embodiment, the clipping may use a function in OpenCV to clip the fixed region.
Furthermore, because the use scenes are different, the environments of the images are different when the images are shot, and the brightness of the images is different, the original images are preprocessed after the original images are acquired; the pretreatment mode comprises white balance adjustment;
the white balance adjustment mode comprises fixed adjustment according to preset white balance parameters and automatic white balance adjustment according to preset rules.
By pretreatment, the treatment difference caused by different use environments is eliminated.
Step 120, judging whether the ROI image contains tongue according to a tongue detection model;
the tongue detection model is established and trained in advance through a deep product neural network SqueEzeNet, the ROI image is classified, and whether the ROI image contains a tongue body or not is judged; if not, prompting to re-provide the original image;
the tongue detection model is obtained by training in a set SqueezeNet pre-training model in a mode of transfer learning under a Caffe deep learning frame;
the tongue detection model training samples are a plurality of pre-acquired ROI images containing tongue;
step 130, positioning the tongue according to a tongue positioning model to obtain a positioned tongue region image;
the tongue is positioned according to an SSD network frame based on a lightweight deep convolutional neural network; finding out the specific position of the tongue through an SSD network architecture, and determining the upper left corner coordinate, the width and the height; and then cutting to obtain an accurate tongue region image.
In order to improve algorithm performance, positioning target calculation is performed by limiting the positioning target size of the SSD network frame;
the positioning target calculation only calculates target positioning layers with smaller number of four candidate areas of the SSD network frame; the target positioning layer with the smaller number of candidate areas comprises Conv8_2, conv9_2, conv10_2 and Conv11_2.
The number of target areas corresponding to feature maps in the four target positioning layers conv8_2, conv9_2, conv10_2, and conv11_2 is 6, 4, and 4 in order; the number of target areas to be calculated finally is 790 (10×10×6+5×5×6+3×3×4+1×1×4); the number of candidate target areas (8732) which are originally smaller than the number of the traditional candidate target areas using the SSD network architecture; therefore, by reducing the calculated target positioning layer, the calculation time consumption can be effectively reduced.
Further, the tongue positioning model is obtained by training in a set SqueEzeNet pre-training model in a mode of transfer learning under a Caffe deep learning frame;
the training samples of the tongue positioning model are a plurality of pre-acquired ROI images containing tongue.
Step 140, analyzing the tongue condition in the tongue region image according to the tongue analysis model, and outputting a tongue analysis result;
the tongue analysis models are multiple and correspond to the tongue characteristics; the tongue features include tooth trace, prick, putrefaction, thickness, crack, fatness, thin, red tongue edge tip, tongue color, and tongue coating color;
the tooth marks, the pricks, the thickness, the cracks and the red tongue edge tips are classified into three categories, namely, putrefaction comprises putrefaction, greasiness and non-putrefaction, fat and thin comprises fat, thin and moderate, and the tongue color comprises the subclasses of pale white, pale red, pale purple, red, dark red and the like, and the tongue coating color comprises the subclasses of white, yellow, huang Baixiang and grey black.
The tongue analysis models are obtained through SquezeNet deep convolutional neural network training under a Caffe deep learning framework; the training samples of the tongue analysis model are a plurality of pre-labeled tongue region images.
Further, according to tongue condition analysis requirements, the plurality of tongue analysis models are called through a dnn module of OpenCV; the call method is faster than the call of the deep learning convolutional neural network model by directly using Caffe; further improving the algorithm performance.
The tongue diagnosis analysis system based on the deep convolutional neural network comprises:
the ROI image acquisition unit is used for acquiring an original image, cutting the original image according to a preset region of interest and acquiring an ROI image;
the tongue detection unit is used for receiving the ROI image of the ROI image acquisition unit and judging whether the ROI image contains a tongue or not according to a tongue detection model;
further, the preset region of interest clipped by the ROI image acquisition unit is set as a fixed-size region in the middle of the original image frame.
The tongue positioning unit is used for positioning the tongue according to the tongue positioning model to obtain a positioned tongue region image;
furthermore, the tongue positioning unit is used for positioning the tongue according to an SSD network frame on the basis of a lightweight deep convolutional neural network;
the tongue positioning unit is used for calculating a positioning target by limiting the positioning target size of the SSD network frame;
the positioning target calculation only calculates target positioning layers with smaller number of four candidate areas of the SSD network frame; the target positioning layer with the smaller number of candidate areas comprises Conv8_2, conv9_2, conv10_2 and Conv11_2.
The tongue analysis unit is used for analyzing tongue conditions in the tongue region image according to the tongue analysis model and outputting tongue analysis results;
further, the tongue analysis model is multiple and corresponds to multiple tongue characteristics; the tongue features include tooth trace, prick, putrefaction, thickness, crack, fatness, thin, red tongue edge tip, tongue color, and tongue coating color;
the tongue analysis models are obtained through SquezeNet deep convolutional neural network training under a Caffe deep learning framework;
and the tongue analysis unit is used for calling the tongue analysis models through a dnn module of OpenCV according to tongue condition analysis requirements.
The model training unit is used for obtaining the tongue detection model, the tongue positioning model and the tongue analysis model through training in advance by a lightweight deep convolutional neural network.
Further, the model training unit is used for training in the set SqueezeNet pre-training model in a mode of transfer learning under the Caffe deep learning frame;
the training samples of the tongue detection model and the tongue positioning model are a plurality of pre-acquired ROI images containing tongue; the training samples of the tongue analysis model are a plurality of pre-labeled tongue region images.
Further, the system also comprises a preprocessing unit, wherein the preprocessing unit is used for preprocessing the original image; the pretreatment mode comprises white balance adjustment;
the white balance adjustment mode comprises fixed adjustment according to preset white balance parameters and automatic white balance adjustment according to preset rules.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the disclosure may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. Reference to step numbers in this specification is used solely to distinguish between steps and is not intended to limit the time or logical relationship between steps, including the various possible conditions unless the context clearly indicates otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the disclosure and form different embodiments. For example, any of the embodiments claimed in the claims may be used in any combination.
Various component embodiments of the present disclosure may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. The present disclosure may also be implemented as an apparatus or system program (e.g., a computer program and a computer program product) for performing a portion or all of the methods described herein. Such a program embodying the present disclosure may be stored on a computer readable medium, or may have the form of one or more signals. Such signals may be downloaded from an internet website, provided on a carrier signal, or provided in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the disclosure, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The disclosure may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware.
The foregoing is merely a specific embodiment of the disclosure, and it should be noted that it will be apparent to those skilled in the art that several improvements, modifications, and variations can be made without departing from the spirit of the disclosure, and these improvements, modifications, and variations are to be considered within the scope of the present application.
Claims (6)
1. A tongue diagnosis analysis method based on a deep convolutional neural network, the method comprising:
acquiring an original image, and cutting the original image according to a preset region of interest to obtain an ROI image;
judging whether the ROI image contains tongue according to a tongue detection model;
according to the tongue positioning model, positioning the tongue to obtain a positioned tongue region image;
according to the tongue analysis model, analyzing tongue conditions in the tongue region image, and outputting tongue analysis results;
the tongue detection model, the tongue positioning model and the tongue analysis model are obtained through training in advance by a lightweight deep convolutional neural network;
the tongue is positioned according to an SSD network frame based on a lightweight deep convolutional neural network;
performing positioning target calculation by limiting the positioning target size of the SSD network frame;
the positioning target calculation only calculates target positioning layers with small numbers of four candidate areas of the SSD network frame; the target positioning layer with the small number of the candidate areas comprises Conv8_2, conv9_2, conv10_2 and Conv11_2;
the tongue analysis models are multiple and correspond to the tongue characteristics; the tongue features include tooth trace, prick, putrefaction, thickness, crack, fatness, thin, red tongue edge tip, tongue color, and tongue coating color;
the tongue analysis models are obtained through SquezeNet deep convolutional neural network training under a Caffe deep learning framework;
according to tongue condition analysis requirements, invoking the tongue analysis models through a dnn module of OpenCV;
the tongue detection model, the tongue positioning model and the tongue analysis model are obtained by training in a set SqueEzeNet pre-training model in a mode of transfer learning under a Caffe deep learning frame;
the training samples of the tongue detection model and the tongue positioning model are a plurality of pre-acquired ROI images containing tongue; the training samples of the tongue analysis model are a plurality of pre-labeled tongue region images.
2. The method according to claim 1, characterized in that: before clipping the original image according to the preset region of interest to obtain an ROI image, the method further includes:
preprocessing the original image; the pretreatment mode comprises white balance adjustment;
the white balance adjustment mode comprises fixed adjustment according to preset white balance parameters and automatic white balance adjustment according to preset rules.
3. The method according to claim 1, characterized in that: the preset region of interest is set as a fixed size region in the middle of the original image picture.
4. A tongue diagnosis analysis system based on a deep convolutional neural network, the system comprising:
the ROI image acquisition unit is used for acquiring an original image, cutting the original image according to a preset region of interest and acquiring an ROI image;
the tongue detection unit is used for receiving the ROI image of the ROI image acquisition unit and judging whether the ROI image contains a tongue or not according to a tongue detection model;
the tongue positioning unit is used for positioning the tongue according to the tongue positioning model to obtain a positioned tongue region image;
the tongue analysis unit is used for analyzing tongue conditions in the tongue region image according to the tongue analysis model and outputting tongue analysis results;
the model training unit is used for obtaining the tongue detection model, the tongue positioning model and the tongue analysis model through training in advance by a lightweight deep convolutional neural network;
the tongue positioning unit is used for positioning the tongue according to an SSD network frame on the basis of a lightweight deep convolutional neural network;
the tongue positioning unit is used for calculating a positioning target by limiting the positioning target size of the SSD network frame;
the positioning target calculation only calculates target positioning layers with small numbers of four candidate areas of the SSD network frame; the target positioning layer with the small number of the candidate areas comprises Conv8_2, conv9_2, conv10_2 and Conv11_2;
the tongue analysis models are multiple and correspond to the tongue characteristics; the tongue features include tooth trace, prick, putrefaction, thickness, crack, fatness, thin, red tongue edge tip, tongue color, and tongue coating color;
the tongue analysis models are obtained through SquezeNet deep convolutional neural network training under a Caffe deep learning framework;
the tongue analysis unit performs the call of the tongue analysis models through a dnn module of OpenCV according to tongue condition analysis requirements;
the model training unit is used for training in the set SquezeNet pre-training model in a transfer learning mode under the Caffe deep learning framework;
the training samples of the tongue detection model and the tongue positioning model are a plurality of pre-acquired ROI images containing tongue; the training samples of the tongue analysis model are a plurality of pre-labeled tongue region images.
5. The system according to claim 4, wherein: the system also comprises a preprocessing unit, wherein the preprocessing unit is used for preprocessing the original image; the pretreatment mode comprises white balance adjustment;
the white balance adjustment mode comprises fixed adjustment according to preset white balance parameters and automatic white balance adjustment according to preset rules.
6. The system according to claim 5, wherein: the preset region of interest clipped by the ROI image acquisition unit is set as a fixed-size region in the middle of the original image frame.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811558698.3A CN109637660B (en) | 2018-12-19 | 2018-12-19 | Tongue diagnosis analysis method and system based on deep convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811558698.3A CN109637660B (en) | 2018-12-19 | 2018-12-19 | Tongue diagnosis analysis method and system based on deep convolutional neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109637660A CN109637660A (en) | 2019-04-16 |
| CN109637660B true CN109637660B (en) | 2024-01-23 |
Family
ID=66075512
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811558698.3A Active CN109637660B (en) | 2018-12-19 | 2018-12-19 | Tongue diagnosis analysis method and system based on deep convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109637660B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110119738A (en) * | 2019-05-21 | 2019-08-13 | 中电健康云科技有限公司 | Indentation detection method, device, electronic equipment and readable storage medium storing program for executing |
| CN110298829A (en) * | 2019-06-19 | 2019-10-01 | 上海国民集团健康科技有限公司 | A kind of lingual diagnosis method, apparatus, system, computer equipment and storage medium |
| CN110251084A (en) * | 2019-06-21 | 2019-09-20 | 福州数据技术研究院有限公司 | A kind of detection of tongue picture and recognition methods based on artificial intelligence |
| CN110599463A (en) * | 2019-08-26 | 2019-12-20 | 依脉人工智能医疗科技(天津)有限公司 | Tongue image detection and positioning algorithm based on lightweight cascade neural network |
| CN111209801A (en) * | 2019-12-24 | 2020-05-29 | 新绎健康科技有限公司 | Traditional Chinese medicine fat tongue identification method and device |
| CN113362334A (en) * | 2020-03-04 | 2021-09-07 | 北京悦熙兴中科技有限公司 | Tongue picture processing method and device |
| CN112750531A (en) * | 2021-01-21 | 2021-05-04 | 广东工业大学 | Automatic inspection system, method, equipment and medium for traditional Chinese medicine |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104217216A (en) * | 2014-09-01 | 2014-12-17 | 华为技术有限公司 | Method and device for generating detection model, method and device for detecting target |
| CN106295139A (en) * | 2016-07-29 | 2017-01-04 | 姹ゅ钩 | A kind of tongue body autodiagnosis health cloud service system based on degree of depth convolutional neural networks |
| CN107316307A (en) * | 2017-06-27 | 2017-11-03 | 北京工业大学 | A kind of Chinese medicine tongue image automatic segmentation method based on depth convolutional neural networks |
| CN107330889A (en) * | 2017-07-11 | 2017-11-07 | 北京工业大学 | A kind of traditional Chinese medical science tongue color coating colour automatic analysis method based on convolutional neural networks |
| CN108918536A (en) * | 2018-07-13 | 2018-11-30 | 广东工业大学 | Tire-mold face character defect inspection method, device, equipment and storage medium |
-
2018
- 2018-12-19 CN CN201811558698.3A patent/CN109637660B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104217216A (en) * | 2014-09-01 | 2014-12-17 | 华为技术有限公司 | Method and device for generating detection model, method and device for detecting target |
| CN106295139A (en) * | 2016-07-29 | 2017-01-04 | 姹ゅ钩 | A kind of tongue body autodiagnosis health cloud service system based on degree of depth convolutional neural networks |
| CN107316307A (en) * | 2017-06-27 | 2017-11-03 | 北京工业大学 | A kind of Chinese medicine tongue image automatic segmentation method based on depth convolutional neural networks |
| CN107330889A (en) * | 2017-07-11 | 2017-11-07 | 北京工业大学 | A kind of traditional Chinese medical science tongue color coating colour automatic analysis method based on convolutional neural networks |
| CN108918536A (en) * | 2018-07-13 | 2018-11-30 | 广东工业大学 | Tire-mold face character defect inspection method, device, equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 周爱明等.基于深度学习的蝴蝶科级标本图像自动识别.《昆虫学报》.2017,第第60卷卷(第第60卷期),摘要. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109637660A (en) | 2019-04-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109637660B (en) | Tongue diagnosis analysis method and system based on deep convolutional neural network | |
| CN110197493B (en) | Fundus image blood vessel segmentation method | |
| CN107292877B (en) | Left and right eye identification method based on fundus image characteristics | |
| CN108510502A (en) | Melanoma picture tissue segmentation methods based on deep neural network and system | |
| CN109858540B (en) | Medical image recognition system and method based on multi-mode fusion | |
| KR102250694B1 (en) | Method and device for predicting disease using segmentating vessel in eye image | |
| CN106557746A (en) | A kind of tongue based on image recognition technology is as detection method and system | |
| CN106157279A (en) | Eye fundus image lesion detection method based on morphological segment | |
| CN112597982B (en) | Image classification method, device, equipment and medium based on artificial intelligence | |
| CN111950595A (en) | Liver focus image processing method, system, storage medium, program, and terminal | |
| CN109754388B (en) | Carotid artery stenosis degree calculation method and device and storage medium | |
| CN109583331A (en) | Human wrist arteries and veins mouth position precise positioning method based on deep learning | |
| Dimauro et al. | Anaemia detection based on sclera and blood vessel colour estimation | |
| CN113205490A (en) | Mask R-CNN network-based auxiliary diagnosis system and auxiliary diagnosis information generation method | |
| Xue et al. | Optic disk detection and segmentation for retinal images using saliency model based on clustering | |
| Zhao et al. | Attention residual convolution neural network based on U-net (AttentionResU-Net) for retina vessel segmentation | |
| Hu et al. | Multi-scale interactive network with artery/vein discriminator for retinal vessel classification | |
| CN109636864A (en) | A kind of tongue dividing method and system based on color correction Yu depth convolutional neural networks | |
| Liu et al. | Retinal vessel segmentation using densely connected convolution neural network with colorful fundus images | |
| CN109711306B (en) | Method and equipment for obtaining facial features based on deep convolutional neural network | |
| CN110279402A (en) | A kind of imaging method of veins beneath the skin optical video image | |
| Ali et al. | Optic Disc Localization in Retinal Fundus Images Based on You Only Look Once Network (YOLO). | |
| CN109816665A (en) | A kind of fast partition method and device of optical coherence tomographic image | |
| CN114677713A (en) | Near-infrared light-based arm vein blood sampling point identification method and system | |
| Vijayalakshmi et al. | Development of prognosis tool for type-II diabetics using tongue image analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |