CN111524093A - Intelligent screening method and system for abnormal tongue picture - Google Patents
Intelligent screening method and system for abnormal tongue picture Download PDFInfo
- Publication number
- CN111524093A CN111524093A CN202010209218.3A CN202010209218A CN111524093A CN 111524093 A CN111524093 A CN 111524093A CN 202010209218 A CN202010209218 A CN 202010209218A CN 111524093 A CN111524093 A CN 111524093A
- Authority
- CN
- China
- Prior art keywords
- tongue
- picture
- abnormal
- tongue picture
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
Abstract
The invention provides an intelligent screening method and system for abnormal tongue picture, which provides objective and quantitative measurement means for tongue diagnosis information judgment by constructing a knowledge platform of traditional Chinese medicine tongue diagnosis and an expert auxiliary diagnosis system, and improves the correctness of diagnosis; adopting a fine-grained image classification algorithm based on an attention mechanism, wherein the attention mechanism reflects the perception difference of a human visual system to the surrounding environment, namely the attention is focused on a salient region in the environment; the convolutional neural network is used for simulating the visual attention characteristic of human beings, so that the information of a classification target is fully utilized, and the accuracy of tongue picture fine-grained image classification is improved; inspired by a visual attention mechanism, in the process of utilizing a fine-grained image classification algorithm, the spatial position characteristics are introduced into classification and modeling, namely, a plurality of key areas of the tongue picture are positioned, modeling is carried out according to the spatial relationship among all parts of the tongue picture, and the integrity and the discrimination of the part characteristics can be improved.
Description
Technical Field
The invention relates to the technical field of medical image processing and artificial intelligence auxiliary diagnosis, in particular to an intelligent abnormal tongue picture screening method and system.
Background
Tongue diagnosis is an important part of inspection and diagnosis in traditional Chinese medicine, and also an important means for clinical observation and diagnosis of diseases. The tongue diagnosis is based on the diagnosis of the disease by observing the abnormal tongue condition to understand the physiological function and pathological changes of the body. However, the traditional tongue diagnosis information is mainly obtained by clinical subjective judgment, and lacks of objective and quantitative measurement means, and the complexity of the self-state of the person to be diagnosed also affects the accuracy of the diagnosis result. Therefore, in order to solve the above problems, the invention provides an intelligent screening method and system for abnormal tongue picture, which provides solutions for knowledge and health management of different areas, ages, healthy or sub-healthy people and physical health status and tongue picture association by constructing a knowledge platform and an expert-assisted diagnosis system for tongue diagnosis in traditional Chinese medicine, and accurately extracts abnormal features of input tongue picture by using a fine-grained image classification algorithm and an artificial neural network algorithm.
Disclosure of Invention
In view of the above, the invention provides an intelligent screening method and system for abnormal tongue picture, which provides solutions for knowledge and health management of different areas, ages, healthy or sub-healthy people and body health status and tongue picture association by constructing a knowledge platform and an expert-assisted diagnosis system for tongue diagnosis in traditional Chinese medicine, and accurately extracts abnormal features of input tongue picture by using a fine-grained image classification algorithm and an artificial neural network algorithm.
The technical scheme of the invention is realized as follows: in one aspect, the invention provides an intelligent abnormal tongue picture screening method, which comprises the following steps:
s1, collecting abnormal tongue picture symptom data, carrying out professional tongue picture evaluation on the collected tongue picture data according to tongue pictures and physical health conditions of healthy or sub-healthy people in different regions and ages, and constructing a knowledge platform and an expert-aided diagnosis system of the tongue diagnosis of the traditional Chinese medicine by a consistent evaluation method;
s2, extracting abnormal symptom features of the input human tongue picture by using a fine-grained image classification algorithm and an artificial neural network algorithm, and mapping the abnormal symptom features of the human tongue picture to a neural network for image recognition and classification to obtain a classification result of the human tongue picture;
s3, inputting the classification result of the human tongue picture images into a knowledge platform of the tongue diagnosis of the traditional Chinese medicine and an expert auxiliary diagnosis system to obtain knowledge related to the tongue picture of different areas, ages, healthy or sub-healthy people and physical health conditions.
On the basis of the above technical solution, it is preferable that, before executing S2, the method further includes: the mirror image of the human tongue image is used for carrying out shape and color space conversion, and the image histogram is used for adjusting the contrast of the human tongue image, so that the contrast of the abnormal tongue symptom image in the human tongue image is enhanced.
On the basis of the above technical solution, preferably, the specific step of S2 is:
s101, completing training of an attention network model through a training set sample with a category label, a marking frame, a local area space position and description, inputting a human body tongue image with the category label, the marking frame, the local area space position and the description into the trained attention network model, and generating an attention distribution map corresponding to the attention network model;
s102, positioning of the saliency areas is completed according to the attention distribution map, and an object level classification network and a local level classification network are respectively trained according to the saliency areas of different levels;
s103, merging the output of the object level classification network and the output of the local level classification network to obtain a classification result of the input human tongue picture image.
Further preferably, the attention network model in S101 includes 12 convolutional layers, 5 pooling layers, 2 fully-connected layers, 1 global average pooling layer, 1 bilinear operation layer, and 1 softmax layer;
2 layers of convolution layer, 1 layer of pooling layer, 3 layers of convolution layer, 1 layer of pooling layer, 1 layer of bilinear operation layer, 1 layer of global average pooling layer, 2 layers of full-connection layer and 1 layer of softmax layer are connected in series in sequence.
Further preferably, the attention network model extracts the attention distribution map through a class activation mapping algorithm in S101.
More preferably, S102 specifically includes the following steps:
s201, converting the attention distribution map into an HSV color space;
s202, setting different H value threshold values to carry out binarization processing on the attention distribution map to obtain significance regions of different levels;
s203, corresponding the salient region coordinates with the coordinates of the input image, and cutting the input image according to the corresponding coordinates to obtain salient region slices at an object level and a local level;
and S204, respectively training an object level classification network and a local level classification network according to the salient region slices of the object level and the local level.
More preferably, S103 specifically includes the following steps:
s301, the object level classification network and the local level classification network respectively output the subordinate probability of the prediction input image to each category;
s302, obtaining the sum result of the subordinate probabilities of the same category output by the object level classification network and the local classification network, and selecting the category corresponding to the maximum sum result as a prediction classification result.
On the other hand, the invention provides an intelligent screening system for abnormal tongue pictures, which comprises a server, a shooting device, an intelligent terminal, a display unit and a shooting and recording unit;
the server includes: the abnormal tongue picture health examination module, the matching module and the information transmission module;
the shooting device is used for shooting the abnormal tongue picture of the detected person and transmitting the characteristic picture of the abnormal tongue picture symptom of the detected person to the server;
the display unit is used for displaying the tongue picture abnormal symptom characteristic picture shot by the shooting device;
the abnormal tongue picture health examination module analyzes the picture shot by the shooting device and outputs a picture marked with the abnormal tongue picture to the shooting and recording unit;
the recording unit marks and describes a suspected image and an acquired reflection image in the tongue picture abnormal symptom characteristic picture according to the tongue picture clinical abnormal symptom characteristic, and transmits the suspected image and the acquired reflection image to the matching module;
the matching module carries out picture retrieval and comparison according to the suspected picture and the reflected picture in each tongue picture abnormal symptom characteristic picture of the examined person to generate diagnosis content and a diagnosis conclusion;
the information transmission module transmits the picture output by the abnormal tongue picture health examination module, the disease diagnosis content and the disease diagnosis conclusion output by the matching module to the intelligent terminal;
the display unit is electrically connected with the server, the shooting device is electrically connected with the input end of the abnormal tongue picture health check module, the output end of the abnormal tongue picture health check module is electrically connected with the input end of the shooting unit, the output end of the shooting unit is electrically connected with the input end of the matching module, the output end of the matching module is electrically connected with the input end of the information transmission module, and the output end of the information transmission module is electrically connected with the intelligent terminal.
On the basis of the above technical solution, preferably, the server further includes: the device comprises a rechecking module and a transmission module;
the rechecking module generates corresponding confidence coefficient for the label in the image output by the abnormal tongue picture health examination module;
the transmission module sets the range of the confidence coefficient output by the rechecking module, and when the confidence coefficient is not in the preset range, the image output by the abnormal tongue picture health examination module is transmitted to the traditional Chinese medicine tongue diagnosis cloud;
the input end of the rechecking module is electrically connected with the output end of the abnormal tongue picture health examination module, the output end of the rechecking module is electrically connected with the input end of the transmission module, and the output end of the transmission module is electrically connected with the intelligent terminal.
On the basis of the above technical solution, preferably, an indication module for displaying the tongue image model to indicate the subject to shoot is installed inside the shooting device.
Compared with the prior art, the intelligent screening method for the abnormal tongue picture has the following beneficial effects:
(1) extracting abnormal symptom characteristics of the input human tongue picture image by adopting a fine-grained image classification algorithm and an artificial neural network algorithm, wherein the fine-grained image classification algorithm is very sensitive to the difference of local areas, can automatically position a high-identification-degree area of the image, effectively represents the detail information of a target, and has higher accuracy in image identification by using a surrounding frame for marking; by combining a fine-grained image classification algorithm and an artificial neural network algorithm, the characteristics can be automatically extracted, the high-level semantic characteristics of the original image can be obtained, the local information can be well characterized, and the limitation of an artificial characteristic extraction method is avoided;
(2) by constructing a knowledge platform of the tongue diagnosis of the traditional Chinese medicine and an expert auxiliary diagnosis system, an objective and quantitative measurement means can be provided for the judgment of the tongue diagnosis information, and the diagnosis accuracy is improved;
(3) adopting a fine-grained image classification algorithm based on an attention mechanism, wherein the attention mechanism reflects the perception difference of a human visual system to the surrounding environment, namely the attention is focused on a salient region in the environment; the method uses the convolutional neural network to simulate the visual attention characteristic of human beings, thereby fully utilizing the information of a classification target and being beneficial to improving the accuracy of tongue picture fine-grained image classification;
(4) inspired by a visual attention mechanism, in the process of utilizing a fine-grained image classification algorithm, the spatial position characteristics are introduced into classification and modeling, namely, a plurality of key areas of the tongue picture are positioned, modeling is carried out according to the spatial relationship among all parts of the tongue picture, and the integrity and the discrimination of the part characteristics can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of a method for intelligently screening abnormal tongue manifestation according to the present invention;
FIG. 2 is a flowchart of tongue image feature extraction in an intelligent abnormal tongue image screening method according to the present invention;
FIG. 3 is a flow chart of the method for intelligently screening abnormal tongue manifestation according to the present invention, wherein the method for intelligently screening abnormal tongue manifestation positions the salient regions according to the attention distribution map;
FIG. 4 is a flowchart illustrating the fusion of the outputs of the object-level classification network and the local classification network in the intelligent abnormal tongue picture screening method according to the present invention;
FIG. 5 is a block diagram of an intelligent screening system for abnormal tongue manifestation in accordance with the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
Example 1
As shown in fig. 1, the intelligent abnormal tongue screening method of the present invention includes the following steps:
s1, collecting abnormal tongue picture symptom data, carrying out professional tongue picture evaluation on the collected tongue picture data according to tongue pictures and physical health conditions of healthy or sub-healthy people in different regions and ages, and constructing a knowledge platform and an expert-aided diagnosis system of the tongue diagnosis of the traditional Chinese medicine by a consistent evaluation method;
preferably, the method further comprises preprocessing and enhancing the input human tongue picture image before proceeding to the next step. The method comprises the following specific steps: the mirror image of the human tongue image is used for carrying out shape and color space conversion, and the image histogram is used for adjusting the contrast of the human tongue image, so that the contrast of the abnormal tongue symptom image in the human tongue image is enhanced.
S2, extracting abnormal symptom features of the input human tongue picture by using a fine-grained image classification algorithm and an artificial neural network algorithm, and mapping the abnormal symptom features of the human tongue picture to a neural network for image recognition and classification to obtain a classification result of the human tongue picture;
s3, inputting the classification result of the human tongue picture images into a knowledge platform of the tongue diagnosis of the traditional Chinese medicine and an expert auxiliary diagnosis system to obtain knowledge related to the tongue picture of different areas, ages, healthy or sub-healthy people and physical health conditions.
The beneficial effect of this embodiment does: extracting abnormal symptom characteristics of the input human tongue picture image by adopting a fine-grained image classification algorithm and an artificial neural network algorithm, wherein the fine-grained image classification algorithm is very sensitive to the difference of local areas, can automatically position a high-identification-degree area of the image, effectively represents the detail information of a target, and has higher accuracy in image identification by using a surrounding frame for marking; by combining a fine-grained image classification algorithm and an artificial neural network algorithm, the characteristics can be automatically extracted, the high-level semantic characteristics of the original image can be obtained, the local information can be well characterized, and the limitation of an artificial characteristic extraction method is avoided;
by constructing a knowledge platform of the tongue diagnosis of the traditional Chinese medicine and an expert auxiliary diagnosis system, an objective and quantitative measurement means can be provided for the judgment of the tongue diagnosis information, and the diagnosis accuracy is improved.
Example 2
On the basis of embodiment 1, the present embodiment provides a process for extracting abnormal symptom features of an input human tongue image by using a fine-grained image classification algorithm and an artificial neural network algorithm. The method comprises the following specific steps:
s101, completing training of an attention network model through a training set sample with a category label, a marking frame, a local area space position and description, inputting a human body tongue image with the category label, the marking frame, the local area space position and the description into the trained attention network model, and generating an attention distribution map corresponding to the attention network model;
preferably, in the traditional model training and testing process, the input human body tongue picture image needs to be subjected to scale transformation according to the network structure, so that in the transformation process, local information is easily lost, and a candidate area optionally contains larger redundant information, thereby further influencing the classification performance of the fine-grained image classification algorithm. Therefore, the present embodiment employs a fine-grained image classification algorithm based on an attention mechanism, which reflects the perceptual difference of the human visual system to the surrounding environment, i.e., the attention is focused on the salient region in the environment. In the embodiment, the convolutional neural network is used for simulating the visual attention characteristic of human beings, so that the information of the classification target is fully utilized, and the accuracy of tongue picture fine-grained image classification is improved.
In this embodiment, the attention network model includes 12 convolutional layers, 5 pooling layers, 2 fully-connected layers, 1 global average pooling layer, 1 bilinear operation layer, and 1 softmax layer; wherein, 2 layers of convolution layer, 1 layer of pooling layer, 3 layers of convolution layer, 1 layer of pooling layer, 1 layer of bilinear operation layer, 1 layer of global average pooling layer, 2 layers of full connection layer and 1 layer of softmax layer are connected in series in sequence. The global average pooling layer is mainly used for keeping the one-to-one correspondence between the pooled nodes and the feature map after the last layer of convolution, facilitating the subsequent calculation of the attention distribution map and simultaneously completing the average pooling operation of the input whole feature map; the bilinear operation layer is used for producing a bilinear feature map, the attention distribution map is also extracted on the basis of the bilinear feature map, and the bilinear operation layer enables the model to have stronger feature expression capability, so that the problem of local information loss is solved; the bilinear feature map obtains the feature vector through the global average pooling layer, and the corresponding relation between the bilinear feature map and the global average pooling layer can be ensured.
Further preferably, the attention network model extracts the attention distribution map by a class activation mapping algorithm.
S102, positioning of the saliency areas is completed according to the attention distribution map, and an object level classification network and a local level classification network are respectively trained according to the saliency areas of different levels;
more preferably, S102 specifically includes the following steps:
s201, converting the attention distribution map into an HSV color space;
s202, setting different H value threshold values to carry out binarization processing on the attention distribution map to obtain significance regions of different levels;
s203, corresponding the salient region coordinates with the coordinates of the input image, and cutting the input image according to the corresponding coordinates to obtain salient region slices at an object level and a local level;
and S204, respectively training an object level classification network and a local level classification network according to the salient region slices of the object level and the local level.
And S203, fusing the outputs of the object level classification network and the local level classification network to obtain a classification result of the input human tongue picture image.
S103, merging the output of the object level classification network and the output of the local level classification network to obtain a classification result of the input human tongue picture image.
More preferably, S103 specifically includes the following steps:
s301, the object level classification network and the local level classification network respectively output the subordinate probability of the prediction input image to each category;
s302, obtaining the sum result of the subordinate probabilities of the same category output by the object level classification network and the local classification network, and selecting the category corresponding to the maximum sum result as a prediction classification result.
The beneficial effect of this embodiment does: adopting a fine-grained image classification algorithm based on an attention mechanism, wherein the attention mechanism reflects the perception difference of a human visual system to the surrounding environment, namely the attention is focused on a salient region in the environment; in the embodiment, the convolutional neural network is used for simulating the visual attention characteristic of human beings, so that the information of a classification target is fully utilized, and the accuracy of tongue picture fine-grained image classification is improved;
inspired by a visual attention mechanism, in the process of utilizing a fine-grained image classification algorithm, the spatial position characteristics are introduced into classification and modeling, namely, a plurality of key areas of the tongue picture are positioned, modeling is carried out according to the spatial relationship among all parts of the tongue picture, and the integrity and the discrimination of the part characteristics can be improved.
Example 3
On the basis of embodiment 1 or embodiment 2, the present embodiment provides an intelligent screening system for abnormal tongue picture, which includes a server, a shooting device, a display unit, an intelligent terminal and a recording unit; the server includes: the abnormal tongue picture health examination module, the matching module, the information transmission module, the rechecking module and the transmission module.
The shooting device is used for shooting the abnormal tongue picture of the detected person and transmitting the characteristic picture of the abnormal tongue picture symptom of the detected person to the server; further preferably, the photographing device is internally provided with an indicating module for displaying the tongue image model to indicate photographing of the subject. In the embodiment, the indication module is installed in the shooting device, so that the user can shoot conveniently, and common users without medical knowledge can use the device. The photographing device may be any one of a mobile phone, a camera, or a tablet mounted with a camera, but is not limited thereto. The pictures taken by the camera may be transferred to the server via a wireless and/or wired network.
The display unit is used for displaying the tongue picture abnormal symptom characteristic picture shot by the shooting device;
the abnormal tongue picture health examination module analyzes the picture shot by the shooting device and outputs a picture marked with the abnormal tongue picture to the shooting and recording unit; in addition, the rechecking module generates a corresponding confidence coefficient for the label in the image output by the abnormal tongue picture health examination module; the transmission module sets the range of the confidence coefficient output by the rechecking module, and when the confidence coefficient is not in the preset range, the image output by the abnormal tongue picture health examination module is transmitted to the traditional Chinese medicine tongue diagnosis cloud. By arranging the rechecking module and the transmission module, a doctor can conveniently analyze the photographing result and avoid the conditions of misdiagnosis and the like.
The recording unit marks and describes a suspected image and an acquired reflection image in the tongue picture abnormal symptom characteristic picture according to the tongue picture clinical abnormal symptom characteristic, and transmits the suspected image and the acquired reflection image to the matching module;
the matching module carries out picture retrieval and comparison according to the suspected picture and the reflected picture in each tongue picture abnormal symptom characteristic picture of the examined person to generate diagnosis content and a diagnosis conclusion;
the information transmission module transmits the picture output by the abnormal tongue picture health examination module, the disease diagnosis content and the disease diagnosis conclusion output by the matching module to the intelligent terminal. The information transfer module can transfer information through communication tools such as mails and short messages, but is not limited thereto. The information transmission module is arranged, so that the examinee can obtain the detection result in time.
In this embodiment, the connection relationship of each module is as follows: the display unit is electrically connected with the server, the shooting device is electrically connected with the input end of the abnormal tongue picture health check module, the output end of the abnormal tongue picture health check module is electrically connected with the input end of the shooting unit and the input end of the rechecking module respectively, the output end of the rechecking module is electrically connected with the input end of the transmission module, the output end of the transmission module is electrically connected with the intelligent terminal, the output end of the shooting unit is electrically connected with the input end of the matching module, the output end of the matching module is electrically connected with the input end of the information transmission module, and the output end of the information transmission module is electrically connected with the intelligent terminal.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. An intelligent abnormal tongue picture screening method is characterized in that: the method comprises the following steps:
s1, collecting abnormal tongue picture symptom data, carrying out professional tongue picture evaluation on the collected tongue picture data according to tongue pictures and physical health conditions of healthy or sub-healthy people in different regions and ages, and constructing a knowledge platform and an expert-aided diagnosis system of the tongue diagnosis of the traditional Chinese medicine by a consistent evaluation method;
s2, extracting abnormal symptom features of the input human tongue picture by using a fine-grained image classification algorithm and an artificial neural network algorithm, and mapping the abnormal symptom features of the human tongue picture to a neural network for image recognition and classification to obtain a classification result of the human tongue picture;
s3, inputting the classification result of the human tongue picture images into a knowledge platform of the tongue diagnosis of the traditional Chinese medicine and an expert auxiliary diagnosis system to obtain knowledge related to the tongue picture of different areas, ages, healthy or sub-healthy people and physical health conditions.
2. The intelligent abnormal tongue picture screening method as claimed in claim 1, wherein: further including, prior to performing S2: the mirror image of the human tongue image is used for carrying out shape and color space conversion, and the image histogram is used for adjusting the contrast of the human tongue image, so that the contrast of the abnormal tongue symptom image in the human tongue image is enhanced.
3. The intelligent abnormal tongue picture screening method as claimed in claim 1, wherein: the specific steps of S2 are as follows:
s101, completing training of an attention network model through a training set sample with a category label, a marking frame, a local area space position and description, inputting a human body tongue image with the category label, the marking frame, the local area space position and the description into the trained attention network model, and generating an attention distribution map corresponding to the attention network model;
s102, positioning of the saliency areas is completed according to the attention distribution map, and an object level classification network and a local level classification network are respectively trained according to the saliency areas of different levels;
s103, merging the output of the object level classification network and the output of the local level classification network to obtain a classification result of the input human tongue picture image.
4. The intelligent abnormal tongue picture screening method as claimed in claim 3, wherein: the attention network model in the S101 comprises 12 convolutional layers, 5 pooling layers, 2 fully-connected layers, 1 global average pooling layer, 1 bilinear operation layer and 1 softmax layer;
2 layers of convolution layer, 1 layer of pooling layer, 3 layers of convolution layer, 1 layer of pooling layer, 1 layer of bilinear operation layer, 1 layer of global average pooling layer, 2 layers of full-connection layer and 1 layer of softmax layer are connected in series in sequence.
5. The intelligent abnormal tongue picture screening method as claimed in claim 3, wherein: in the step S101, the attention network model extracts the attention distribution map through a class activation mapping algorithm.
6. The intelligent abnormal tongue picture screening method as claimed in claim 3, wherein: s102 specifically comprises the following steps:
s201, converting the attention distribution map into an HSV color space;
s202, setting different H value threshold values to carry out binarization processing on the attention distribution map to obtain significance regions of different levels;
s203, corresponding the salient region coordinates with the coordinates of the input image, and cutting the input image according to the corresponding coordinates to obtain salient region slices at an object level and a local level;
and S204, respectively training an object level classification network and a local level classification network according to the salient region slices of the object level and the local level.
7. The intelligent abnormal tongue picture screening method as claimed in claim 3, wherein: the step S103 specifically includes the following steps:
s301, the object level classification network and the local level classification network respectively output the subordinate probability of the prediction input image to each category;
s302, obtaining the sum result of the subordinate probabilities of the same category output by the object level classification network and the local classification network, and selecting the category corresponding to the maximum sum result as a prediction classification result.
8. The utility model provides an intelligent screening system of unusual tongue picture, its includes server, shooting device, intelligent terminal and display element, its characterized in that: the device also comprises a shooting and recording unit;
the server includes: the abnormal tongue picture health examination module, the matching module and the information transmission module;
the shooting device is used for shooting the abnormal tongue picture of the examinee and transmitting the characteristic picture of the abnormal tongue picture symptom of the examinee to the server;
the display unit is used for displaying the characteristic picture of the abnormal tongue symptom shot by the shooting device;
the abnormal tongue picture health examination module analyzes the picture shot by the shooting device and outputs the picture marked with the abnormal tongue picture to the shooting and recording unit;
the recording unit marks and describes a suspected image and an acquired reflection image in the tongue abnormal symptom characteristic picture according to the tongue disease clinical abnormal symptom characteristic, and transmits the suspected image and the acquired reflection image to the matching module;
the matching module carries out picture retrieval and comparison according to the suspected picture and the reflected picture in each tongue picture abnormal symptom characteristic picture of the examined person to generate diagnosis content and a diagnosis conclusion;
the information transmission module transmits the picture output by the abnormal tongue picture health examination module, the disease diagnosis content and the disease diagnosis conclusion output by the matching module to the intelligent terminal;
the display unit is electrically connected with the server, the shooting device is electrically connected with the input end of the abnormal tongue picture health check module, the output end of the abnormal tongue picture health check module is electrically connected with the input end of the shooting unit, the output end of the shooting unit is electrically connected with the input end of the matching module, the output end of the matching module is electrically connected with the input end of the information transmission module, and the output end of the information transmission module is electrically connected with the intelligent terminal.
9. The intelligent abnormal tongue picture screening system of claim 8, wherein: the server further comprises: the device comprises a rechecking module and a transmission module;
the rechecking module generates corresponding confidence degrees for the labels in the images output by the abnormal tongue picture health examination module;
the transmission module is used for setting the range of the confidence coefficient output by the rechecking module, and when the confidence coefficient is not in the preset range, the image output by the abnormal tongue picture health examination module is transmitted to the traditional Chinese medicine tongue diagnosis cloud end;
the input end of the rechecking module is electrically connected with the output end of the abnormal tongue picture health examination module, the output end of the rechecking module is electrically connected with the input end of the transmission module, and the output end of the transmission module is electrically connected with the intelligent terminal.
10. The intelligent abnormal tongue picture screening system of claim 8, wherein: the photographic device is internally provided with an indicating module for displaying the tongue image model to indicate the photographic subject to shoot.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010209218.3A CN111524093A (en) | 2020-03-23 | 2020-03-23 | Intelligent screening method and system for abnormal tongue picture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010209218.3A CN111524093A (en) | 2020-03-23 | 2020-03-23 | Intelligent screening method and system for abnormal tongue picture |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111524093A true CN111524093A (en) | 2020-08-11 |
Family
ID=71910531
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010209218.3A Pending CN111524093A (en) | 2020-03-23 | 2020-03-23 | Intelligent screening method and system for abnormal tongue picture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111524093A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112133436A (en) * | 2020-10-13 | 2020-12-25 | 广州万孚健康科技有限公司 | Health warning method and system based on big data analysis and readable storage medium |
| CN112270323A (en) * | 2020-09-29 | 2021-01-26 | 西安电子科技大学 | Intelligent medical reinspection control method, system, medium, computer equipment and application |
| CN112529002A (en) * | 2020-11-11 | 2021-03-19 | 湖南老来网科技发展有限公司 | Tongue picture classification method and device, computer equipment and storage medium |
| CN113724237A (en) * | 2021-09-03 | 2021-11-30 | 平安科技（深圳）有限公司 | Tooth mark recognition method and device, computer equipment and storage medium |
| CN114565611A (en) * | 2022-04-28 | 2022-05-31 | 武汉大学 | Medical information acquisition method and related equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1830380A (en) * | 2006-04-20 | 2006-09-13 | 上海交通大学 | Tongue picture anulgis and diagnosis system of traditional Chinese medicine |
| CN104298983A (en) * | 2013-07-15 | 2015-01-21 | 清华大学 | Tongue fur image acquisition and analysis system with distributed user terminals |
| CN109685115A (en) * | 2018-11-30 | 2019-04-26 | 西北大学 | A kind of the fine granularity conceptual model and learning method of bilinearity Fusion Features |
| CN110299193A (en) * | 2019-06-27 | 2019-10-01 | 合肥云诊信息科技有限公司 | Chinese medicine health cloud service method based on artificial intelligence lingual diagnosis |
| CN110766665A (en) * | 2019-09-30 | 2020-02-07 | 天聚星信息科技(深圳)有限公司 | Tongue picture data analysis method based on strong supervision algorithm and deep learning network |
| CN110827304A (en) * | 2018-08-10 | 2020-02-21 | 清华大学 | Traditional Chinese medicine tongue image positioning method and system based on deep convolutional network and level set method |
| CN110826629A (en) * | 2019-11-08 | 2020-02-21 | 华南理工大学 | Otoscope image auxiliary diagnosis method based on fine-grained classification |
-
2020
- 2020-03-23 CN CN202010209218.3A patent/CN111524093A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1830380A (en) * | 2006-04-20 | 2006-09-13 | 上海交通大学 | Tongue picture anulgis and diagnosis system of traditional Chinese medicine |
| CN104298983A (en) * | 2013-07-15 | 2015-01-21 | 清华大学 | Tongue fur image acquisition and analysis system with distributed user terminals |
| CN110827304A (en) * | 2018-08-10 | 2020-02-21 | 清华大学 | Traditional Chinese medicine tongue image positioning method and system based on deep convolutional network and level set method |
| CN109685115A (en) * | 2018-11-30 | 2019-04-26 | 西北大学 | A kind of the fine granularity conceptual model and learning method of bilinearity Fusion Features |
| CN110299193A (en) * | 2019-06-27 | 2019-10-01 | 合肥云诊信息科技有限公司 | Chinese medicine health cloud service method based on artificial intelligence lingual diagnosis |
| CN110766665A (en) * | 2019-09-30 | 2020-02-07 | 天聚星信息科技(深圳)有限公司 | Tongue picture data analysis method based on strong supervision algorithm and deep learning network |
| CN110826629A (en) * | 2019-11-08 | 2020-02-21 | 华南理工大学 | Otoscope image auxiliary diagnosis method based on fine-grained classification |
Non-Patent Citations (2)
| Title |
|---|
| 张阳: "细粒度图像分类算法研究", no. 09, pages 30 - 43 * |
| 柳杨: "数字图像物体识别理论详解与实战", 北京邮电大学出版社, pages: 16 - 18 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112270323A (en) * | 2020-09-29 | 2021-01-26 | 西安电子科技大学 | Intelligent medical reinspection control method, system, medium, computer equipment and application |
| CN112270323B (en) * | 2020-09-29 | 2024-02-02 | 西安电子科技大学 | Intelligent medical review control method, system, medium, computer equipment and application |
| CN112133436A (en) * | 2020-10-13 | 2020-12-25 | 广州万孚健康科技有限公司 | Health warning method and system based on big data analysis and readable storage medium |
| CN112529002A (en) * | 2020-11-11 | 2021-03-19 | 湖南老来网科技发展有限公司 | Tongue picture classification method and device, computer equipment and storage medium |
| CN112529002B (en) * | 2020-11-11 | 2023-04-07 | 智慧眼科技股份有限公司 | Tongue picture classification method and device, computer equipment and storage medium |
| CN113724237A (en) * | 2021-09-03 | 2021-11-30 | 平安科技（深圳）有限公司 | Tooth mark recognition method and device, computer equipment and storage medium |
| CN114565611A (en) * | 2022-04-28 | 2022-05-31 | 武汉大学 | Medical information acquisition method and related equipment |
| CN114565611B (en) * | 2022-04-28 | 2022-07-19 | 武汉大学 | Medical information acquisition method and related equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108230296B (en) | Image feature recognition method and device, storage medium and electronic device | |
| CN111524093A (en) | Intelligent screening method and system for abnormal tongue picture | |
| JP7058373B2 (en) | Lesion detection and positioning methods, devices, devices, and storage media for medical images | |
| CN109190540B (en) | Biopsy region prediction method, image recognition device, and storage medium | |
| KR101977174B1 (en) | Apparatus, method and computer program for analyzing image | |
| CN111563887B (en) | Intelligent analysis method and device for oral cavity image | |
| CN111325745B (en) | Fracture region analysis method and device, electronic equipment and readable storage medium | |
| KR20170016778A (en) | Method and program for computing bone age by deep neural network | |
| CN110584618B (en) | Infectious disease machine recognition system based on artificial intelligence | |
| CN109117890B (en) | Image classification method and device and storage medium | |
| KR102045223B1 (en) | Apparatus, method and computer program for analyzing bone age | |
| US20230005251A1 (en) | Diagnostic assistance apparatus and model generation apparatus | |
| CN112614133B (en) | Three-dimensional pulmonary nodule detection model training method and device without anchor point frame | |
| CN110731773A (en) | abnormal electrocardiogram screening method based on fusion of global and local depth features of electrocardiogram | |
| CN115862831B (en) | Intelligent online reservation diagnosis and treatment management system and method | |
| CN112508884A (en) | Comprehensive detection device and method for cancerous region | |
| CN110456050B (en) | Portable intelligent digital parasite in vitro diagnostic instrument | |
| Kaushal et al. | An IoMT‐based smart remote monitoring system for healthcare | |
| CN117392470B (en) | Fundus image multi-label classification model generation method and system based on knowledge graph | |
| CN111047590A (en) | Hypertension classification method and device based on fundus images | |
| CN113096811A (en) | Diabetic foot image processing and risk early warning equipment based on infrared thermal imaging | |
| CN117237351A (en) | Ultrasonic image analysis method and related device | |
| WO2023155488A1 (en) | Fundus image quality evaluation method and device based on multi-source multi-scale feature fusion | |
| CN116797572A (en) | Rheumatoid arthritis activity grading device based on multi-mode data | |
| Sun et al. | Liver tumor segmentation and subsequent risk prediction based on Deeplabv3+ |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |