CN113781488A - Tongue picture image segmentation method, apparatus and medium - Google Patents
Tongue picture image segmentation method, apparatus and medium Download PDFInfo
- Publication number
- CN113781488A CN113781488A CN202110881511.9A CN202110881511A CN113781488A CN 113781488 A CN113781488 A CN 113781488A CN 202110881511 A CN202110881511 A CN 202110881511A CN 113781488 A CN113781488 A CN 113781488A
- Authority
- CN
- China
- Prior art keywords
- picture
- tongue
- network
- tongue picture
- color
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000003709 image segmentation Methods 0.000 title claims description 10
- 238000012549 training Methods 0.000 claims abstract description 40
- 238000012545 processing Methods 0.000 claims abstract description 39
- 230000011218 segmentation Effects 0.000 claims abstract description 32
- 238000006243 chemical reaction Methods 0.000 claims abstract description 22
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 18
- 238000005520 cutting process Methods 0.000 claims abstract description 13
- 238000012360 testing method Methods 0.000 claims abstract description 13
- 238000004364 calculation method Methods 0.000 claims abstract description 12
- 238000001514 detection method Methods 0.000 claims abstract description 11
- 230000009467 reduction Effects 0.000 claims abstract description 4
- 230000001537 neural effect Effects 0.000 claims description 21
- 230000006870 function Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 12
- 239000003086 colorant Substances 0.000 claims description 8
- 238000012937 correction Methods 0.000 claims description 7
- 238000013528 artificial neural network Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 5
- 238000012795 verification Methods 0.000 claims description 5
- 239000011159 matrix material Substances 0.000 claims description 4
- 230000003044 adaptive effect Effects 0.000 claims description 3
- 238000013434 data augmentation Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 230000003416 augmentation Effects 0.000 claims description 2
- 238000011176 pooling Methods 0.000 claims description 2
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 238000013527 convolutional neural network Methods 0.000 abstract description 2
- 238000003745 diagnosis Methods 0.000 description 13
- 239000003814 drug Substances 0.000 description 6
- 201000010099 disease Diseases 0.000 description 5
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 5
- 238000005286 illumination Methods 0.000 description 5
- 238000011160 research Methods 0.000 description 5
- 238000012897 Levenberg–Marquardt algorithm Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000018109 developmental process Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 210000004072 lung Anatomy 0.000 description 2
- 210000001835 viscera Anatomy 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 239000011248 coating agent Substances 0.000 description 1
- 238000000576 coating method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 230000001351 cycling effect Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 229940079593 drug Drugs 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000001575 pathological effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000008733 trauma Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a technical scheme of a segmentation method, a segmentation device and a segmentation medium of a tongue picture image, which comprises the following steps: acquiring a tongue picture, and performing standard color processing on the tongue picture; acquiring tongue information of the tongue picture, performing frame selection on the tongue information, performing color space conversion processing on the tongue picture, and further performing tongue picture detection to obtain a rough tongue picture position of the tongue picture; determining a corresponding threshold range according to a Y-Cb-Cr color space conversion algorithm, intersecting the threshold range with a frame selection range, and performing tongue picture cutting processing; performing multi-detail reduction to generate a tongue picture with higher definition; training and testing are carried out, and a complete segmentation tongue picture is output. The invention has the beneficial effects that: improved confidence of tongue picture results; the calculation pressure of the deep convolutional neural network is reduced, the calculation speed is increased, the accuracy is also improved, the hardware cost is reduced, and the applicability of the model is enhanced.
Description
Technical Field
The invention relates to the field of computers and medical treatment, in particular to a tongue picture image segmentation method, electronic equipment and a medium.
Background
For research in the medical field, it has become an important research direction to focus on the comprehensive physical condition of people rather than only on a single disease. The method is consistent with the idea that the human body is regarded as a whole in traditional Chinese medicine and the human body is in a healthy state by adjusting the circulation of the whole system. For a single individual, along with the continuous improvement of living standard, people pay more and more attention to the health, and pay more attention to the noninvasive and painless detection of diseases. The theory of traditional Chinese medicine holds that the body is an organic whole, and the tongue is connected with the viscera of the five viscera and the lung through the meridians and collaterals. The traditional Chinese medicine considers that the tongue inspection can understand the deficiency and excess of the lung, the disease nature cold and heat, the location of disease evil and the abundance and insufficiency of qi and blood of a patient, and plays an important role in disease condition evaluation and prescription development medication. Tongue diagnosis the pathological condition of a patient is diagnosed and analyzed by observing tongue picture. Therefore, the most obvious advantage of tongue diagnosis in traditional Chinese medicine is painlessness and no trauma. These conjunction points provide opportunities for the future development of tongue diagnosis in traditional Chinese medicine.
In recent years, image processing techniques have been rapidly developed, and more researchers are dedicated to quantification and standardization of tongue diagnosis. The tongue diagnosis is objectively performed in a plurality of aspects of research from tongue picture collection, tongue body segmentation, tongue coating separation, tongue picture characteristic extraction and attribute identification, and related products such as a tongue picture diagnosis system, a tongue diagnosis instrument and the like are published, and even are practically applied in clinic.
Although related devices such as tongue picture diagnosis systems and tongue diagnosis instruments are available at present, the tongue picture collection is generally required to be completed in a closed space with stable illumination, and specific collection devices are required, although a high-quality tongue picture can be obtained by means of constant illumination and professional devices, and the difficulty in subsequent tongue picture processing is reduced, on one hand, most of the collection or diagnosis devices are expensive and not easy to carry, and only some hospitals and some families have related equipment, so that the collection or diagnosis devices have certain limitations, and thus the collection or diagnosis devices are not widely applied. On the other hand, in consideration of popularization of smart phones, mobile phone camera shooting technology is greatly improved. Therefore, the tongue picture collection and intelligent analysis by using mobile devices in natural environment are gradually becoming new development directions. The segmentation and identification of tongue images taken by different devices under natural light conditions also become one of the main research works of computer tongue diagnosis, which is also the main content of research. In order to more accurately identify and analyze the tongue picture, two preprocessing operations of color correction and tongue body segmentation are carried out on the tongue picture.
At present, the tongue picture processing needs to train according to the tongue picture colors under different environmental illumination to obtain an illumination condition classifier, then train a color correction matrix under different illumination, finally pass the corrected picture through a tongue body segmentation network, and finally output the segmented tongue picture.
On one hand, hardware cost is increased by training a plurality of classifiers, and a large amount of time is consumed for accurate information marking, so that a better classifier can be obtained; on the other hand, the algorithm model is not an end-to-end model, and a plurality of models and a plurality of decomposition steps are adopted, so that the computational complexity and the time complexity of the model are increased. The standard U-Net network is a lightweight end-to-end network with a coding and decoding structure, because the coding model and the decoding model use the same symmetrical structure, and image processing is performed in a down-sampling mode in the coding stage, and image processing is performed in an up-sampling mode in the decoding stage, so that the whole network shape presents a U-shaped structure, which is called as the U-Net network. The unique mirror image operation of the U-Net network and the simple network model have quick and effective segmentation on a small amount of data, but the difficulty of network design is increased due to the fact that the input size and the output size are not equal, meanwhile, the feature graph merging and cutting operation is complicated, and the picture edge cutting and feature graph mismatching exist in an up-sampling mode, so that improvement needs to be conducted on the U-Net network to achieve a better segmentation effect.
Disclosure of Invention
The present invention is directed to solve at least one of the problems of the prior art, and provides a method, an apparatus and a medium for segmenting a tongue image, which overcome the disadvantages of the prior art.
The technical scheme of the invention comprises a segmentation method of a tongue picture image, which is characterized by comprising the following steps: acquiring a tongue picture, and performing standard color processing on the tongue picture to obtain a first picture; creating a VGG16 classification regression network, classifying a first picture, acquiring tongue information of the first picture, performing frame selection on the tongue information and performing color space conversion processing on the first picture, and further performing tongue picture detection to obtain a second picture comprising a rough tongue picture position; determining a corresponding threshold range of the second picture according to a Y-Cb-Cr color space conversion algorithm, intersecting the threshold range with a frame selection range, and performing tongue picture cutting processing to obtain a third picture; creating an SRGAN super-resolution generation network, and performing multi-detail reduction on the third picture to generate a fourth image with higher definition; and (3) creating a neural connection network based on an encoder-decoder structure, training and testing the fourth picture, and outputting a complete segmentation tongue picture.
The segmentation method of the tongue picture comprises the following steps of: creating a BP neural connection network, training the tongue picture by taking a standard color card value as a known input, wherein the known input color card value is taken as a supervision value, and executing supervision training to obtain a trained parameter value; initializing the BP neural connection network through the parameter values, inputting the tongue picture for calculation, and outputting a corrected first picture, wherein the first picture is the corrected tongue picture.
The tongue picture segmentation method comprises the following steps of training a tongue picture by taking a standard color card value as a known input: taking 24 colors of the standard color card values as sample values of the BP neural connection network, performing a plurality of hidden layer calculations on RGB channel values of each sample value, and outputting the corrected RGB channel values; comparing the corrected RGB channel value with the standard color card value, calculating an error, performing back propagation, and correcting all parameter values on a path according to an LM algorithm; and circulating the previous step until the errors of all the parameter values are reduced to be within the set range.
The segmentation method of the tongue picture image, wherein the BP neural connection network comprises: the standard color processing network learns the nonlinear relation between neural networks through the difference between the 24-color real color chip values and the color chip values in different scene pictures, and trains a weight model of the network by updating gradient values through back propagation; training the network through the VGG16 classification regression network through the existing tongue picture labels, and training the network through the existing data through a picture augmentation strategy to obtain results of 5 dimensions, namely whether the results are tongue pictures and position information; adopting Y-Cb-Cr color space conversion algorithm processing, including roughly determining the position information of the tongue picture through a color clustering strategy; generating a network and a tongue segmentation network through the SRGAN super resolution; using a ReLU (rectified Linear Unit) function as an activation function of the BP neural connection network, wherein the ReLU function is set as:
and x represents the output result of each forward propagation, the output result is mapped into a fixed range through an activation function, and the output result is used as the output function of the BP neural connection network through an overall average pooling layer and a Softmax layer.
The tongue picture image segmentation method comprises the following steps of: converting the first picture into a Y-Cb-Cr color space after passing through a VGG16 classification regression network, calculating the similarity of all pixels in the first picture according to the two-dimensional Gaussian distribution of Cb and Cr of the chroma of different tongue colors in the first picture, and calculating the tongue color probability in a way of
Wherein x isaFor the tongue sample value of each pixel a, t denotes the pixel mean value C as a covariance matrix, and m denotes the total number of training samples.
The tongue picture image segmentation method comprises the following steps of: calculating the first image by an AdaBoost algorithm to obtain the second image of the relatively rough tongue picture position, further executing normalization, and extracting a 10x10 pixel area from the detected tongue picture center as a tongue picture color reference, wherein the calculation mode is as follows:
tm＝[Cbm,Crm]
tmexpressing the average value of different components of all the pixel points, and judging the probability value of the pixel points by calculating the Euclidean distance D between the pixel points and the average value, wherein
Sorting the Euclidean distances D of all the extracted pixels from small to large, taking the pixel points with the closer distances to obtain an extraction proportion, setting a threshold range according to different extraction proportions, and repeating the steps to obtain t as a finally obtained adaptive parameter; and obtaining the position of the corresponding tongue picture by displaying in different color spaces, and cutting the position of the tongue picture to obtain the third picture comprising the whole complete tongue picture.
The segmentation method of the tongue picture image, wherein the cutting of the position of the tongue picture comprises the following steps: and intersecting the tongue picture range selected by the VGG16 network frame with the tongue picture range identified by the Y-Cb-Cr color space, taking the center point of the intersection as the center of a circle, and cutting by taking the boundary distance of 1.25 times as the radius.
The tongue image segmentation method according to, wherein creating a neural connection network based on an encoder-decoder structure comprises: a standard color processing strategy, wherein the network performs standardized correction on the color of the picture by learning the difference between a standard color card and a color card in the picture; roughly positioning the position of the tongue picture by using a VGG16 classification regression network and a Y-Cb-Cr color space conversion algorithm; the texture definition of the cut tongue image is improved through an SRGAN super-resolution generation network; generating a network model by using the countermeasure, generating similar available data according to the data, and expanding the data set in a data augmentation mode; performing convolution processing and deconvolution processing on the third picture, wherein a same filling mode is used in the convolution processing, and the size of the image is kept unchanged after the image is subjected to convolution operation; in the deconvolution process, extending picture information by using bilinear difference values; meanwhile, a ResNet18 residual module is used as a convolution layer in each convolution block, and 1/2 random abandon processing is performed during convolution processing to obtain a tongue picture comprising tongue texture information and high-level information.
The segmentation method of the tongue picture image, wherein the training and testing of the third picture comprises: in the training stage, the existing data are divided into a training set and a verification set according to a proportion, and final model parameters are obtained through multiple groups of cross verification; and in the testing stage, the tongue picture obtained under the color space conversion is used as the input of the neural connection network, and a complete tongue picture segmentation picture is obtained.
The technical scheme of the invention comprises a segmentation device of a tongue image, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, and is characterized in that the processor realizes any one of the steps of the method when executing the computer program.
An aspect of the present invention is a computer-readable storage medium storing a computer program, wherein the computer program implements any of the method steps when executed by a processor.
The invention has the beneficial effects that: the original tongue picture is subjected to color restoration and color space conversion, so that the acquired tongue picture does not depend on hardware for acquiring the tongue picture, and the reliability of the tongue picture result is improved. And through the conversion of the color space, the position of the complete tongue picture is rapidly and roughly obtained, most of useless information is removed, the calculation pressure of the deep convolutional neural network is reduced, the calculation speed is increased, the accuracy is also improved, the hardware cost is reduced, and the applicability of the model is enhanced.
Drawings
The invention is further described below with reference to the accompanying drawings and examples;
FIG. 1 is a flow chart illustrating segmentation of a tongue image according to an embodiment of the present invention;
FIG. 2 is an overall flow diagram according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a BP neural connection network process according to an embodiment of the present invention;
fig. 4 shows a schematic view of an apparatus according to an embodiment of the invention.
Detailed Description
Reference will now be made in detail to the present preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout.
In the description of the present invention, the meaning of a plurality of means is one or more, the meaning of a plurality of means is two or more, and larger, smaller, larger, etc. are understood as excluding the number, and larger, smaller, inner, etc. are understood as including the number.
In the description of the present invention, the consecutive reference numbers of the method steps are for convenience of examination and understanding, and the implementation order between the steps is adjusted without affecting the technical effect achieved by the technical solution of the present invention by combining the whole technical solution of the present invention and the logical relationship between the steps.
In the description of the present invention, unless otherwise explicitly defined, terms such as set, etc. should be broadly construed, and those skilled in the art can reasonably determine the specific meanings of the above terms in the present invention in combination with the detailed contents of the technical solutions.
Fig. 1 is a flowchart illustrating segmentation of a tongue image according to an embodiment of the present invention, the flowchart including: acquiring a tongue picture from acquisition equipment, performing standard color processing on the tongue picture, and outputting a first picture; performing VGG16 classification regression network detection on the first picture to detect whether the tongue picture exists, selecting a position, performing color space conversion processing, and further performing tongue picture detection to obtain a second picture comprising the rough position of the tongue picture; determining a corresponding threshold range by a Y-Cb-Cr color space conversion algorithm of the second picture, intersecting the corresponding threshold range with the frame selection range, taking the center point of the intersection as the circle center, taking the boundary distance of 1.25 times as the radius, and performing tongue picture cutting processing to obtain a third picture; and creating a Connect-Net neural network, training and testing the third picture, and outputting a complete segmentation tongue picture.
Fig. 2 is an overall flow chart of the embodiment of the invention, and the RGB color correction is because the display of the three primary colors of RGB is the same as the hardware device, and different devices have corresponding error values, so in order to make the result objectively evaluated, the picture must be processed with standard color. The calibration picture is divided into two stages, a training stage and a testing stage:
wherein the training phase comprises:
i. training by using standard color card value as known input, training by using color card value, and performing supervised training by using known color card value as supervised value
ii.24 color card for Connect-Net neural network is 24 sample values
Each sample has R, G, B channel values, the final output is R, G, B corrected values after calculation through multiple hidden layers, comparison with 24 color cards, error calculation, back propagation, and correction of all parameter values on the path according to LM (Levenberg-Marquardt algorithm)
Cycling the iii step until the error stabilizes and falls within the desired range
Training VGG16 classification regression network
vi training SRGAN super resolution generation network
Training GAN countermeasure generation network
Training Connect-Net segmentation network
The testing stage comprises the following steps:
i. initializing Connect-Net neural network by trained parameter values
ii, calculating each pixel value of the tongue picture to be corrected, and outputting to obtain a corrected picture
iii, sequentially passing the picture after color correction through VGG16, Y-Cb-Cr color space conversion algorithm, SRGAN super-resolution generation network and Connect-Net segmentation network
The connection-Net segmentation neural network model has fewer training samples and a single output result dimension, so that the network structure is relatively simple to design, in order to prevent the over-fitting situation, a Dropout strategy is used for randomly discarding part of neuron information, a ReLU function is used as an activation function of the network, and Global Average Power and Softmax layers are used as output functions of the connection-Net neural network. Wherein x represents the output result of each forward propagation and is mapped into a fixed range through a formula.
Wherein x represents the output result of each forward propagation and is mapped into a fixed range through a formula.
In the RGB color space, the tongue color and the non-tongue color are overlapped on R, G, B three components, and generally, the image needs to be subjected to color space conversion to be converted into a Y-Cb-Cr color space which is irrelevant to brightness and chroma, so as to improve the detection rate. In the Y-Cb-Cr color space, the tongue colors have certain clustering characteristics, and the Cb and Cr distribution of the chroma of different tongue colors is approximately equal to two-dimensional Gaussian distribution. The tongue color probability can be estimated by calculating the similarity of all pixels, and the probability estimation formula is as follows:
in the formula: x is the number ofaFor the tongue sample value of each pixel a, t denotes the pixel mean value C as a covariance matrix, and m denotes the total number of training samples.
In order to improve the robustness of the detection model, the following improvement is made to the above formula.
Obtaining a relatively rough tongue picture position through a VGG16 classification regression network algorithm and a Y-Cb-Cr color space algorithm, carrying out normalization, and extracting a 10x10 pixel region from the detected tongue picture center as a tongue picture color reference, then:
tm＝[Cbm，Crm]
in the formula: t is tmExpressing the average value of different components of all pixel points by calculating the Euclidean between the pixel points and the average valueAnd judging the probability value of the pixel point by the distance D.
Sorting the Euclidean distances of all pixels from small to large, taking the pixel points with the closer distances to obtain an extraction ratio sigma, and setting a threshold range according to different ratios.
And repeating the steps to obtain t as the finally obtained adaptive parameter.
And determining a corresponding threshold range through a Y-Cb-Cr color space conversion algorithm, intersecting the threshold range with the frame selection range, taking the center point of the intersection as the center of a circle, taking the boundary distance of 1.25 times as the radius, and performing tongue picture cutting processing to obtain a picture containing the whole complete tongue picture.
FIG. 3 is a flowchart illustrating a BP neural connection network processing according to an embodiment of the present invention, which uses the Connect-Net network to obtain a precise segmentation tongue image in three stages:
stage of designing network
i. A network model is redesigned aiming at the problems;
firstly, generating similar available data according to the existing small amount of data by using a confrontation generation network model, and expanding a data set in a data augmentation mode;
using a same filling mode in the convolution step to enable the image to keep the original size after the convolution operation;
the deconvolution process uses bilinear difference values to expand the picture information;
v. using the ResNet18 residual module as a convolutional layer in each block module, preventing the network from training overfitting by random drop1/2 parametric magnitude operation, and more tongue texture information and high-level information can be obtained.
Training phase
i. Dividing the existing data into a training set and a testing set according to the ratio of (8: 2);
dividing the training set into a training set and a verification set according to the ratio of (5: 1);
and ii, obtaining final model parameters through 5 sets of cross validation.
Testing phase
i. Taking the tongue picture obtained by color space conversion as the input of an improved network;
and ii, obtaining a complete segmentation picture.
Fig. 4 shows a schematic view of an apparatus according to an embodiment of the invention. The apparatus comprises a memory 100 and a processor 200, wherein the processor 200 stores a computer program, and the computer program implements the following method flows when executed by the processor 200: acquiring a tongue picture, and performing standard color processing on the tongue picture to obtain a first picture; establishing a VGG16 classification regression network, classifying the first picture, acquiring tongue information of the first picture, performing frame selection on the tongue information and performing color space conversion processing on the first picture, and further performing tongue picture detection to obtain a second picture comprising a rough tongue picture position; determining a corresponding threshold range of the second picture according to a Y-Cb-Cr color space conversion algorithm, intersecting the threshold range with the frame selection range, and performing tongue picture cutting processing to obtain a third picture; creating an SRGAN super-resolution generation network, and performing multi-detail reduction on the third picture to generate a fourth image with higher definition; and (4) creating a neural connection network based on an encoder-decoder structure, training and testing the fourth picture, and outputting a complete segmentation tongue picture.
It should be recognized that the method steps in embodiments of the present invention may be embodied or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer readable memory. The method may use standard programming techniques. Each program may be implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Furthermore, the program can be run on a programmed application specific integrated circuit for this purpose.
Further, the operations of processes described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and/or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) collectively executed on one or more processors, by hardware, or combinations thereof. The computer program includes a plurality of instructions executable by one or more processors.
Further, the method may be implemented in any type of computing platform operatively connected to a suitable interface, including but not limited to a personal computer, mini computer, mainframe, workstation, networked or distributed computing environment, separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, and the like. Aspects of the invention may be embodied in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optically read and/or write storage medium, RAM, ROM, or the like, such that it may be read by a programmable computer, which when read by the storage medium or device, is operative to configure and operate the computer to perform the procedures described herein. Further, the machine-readable code, or portions thereof, may be transmitted over a wired or wireless network. The invention described herein includes these and other different types of non-transitory computer-readable storage media when such media include instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. The invention also includes the computer itself when programmed according to the methods and techniques described herein.
A computer program can be applied to input data to perform the functions described herein to transform the input data to generate output data that is stored to non-volatile memory. The output information may also be applied to one or more output devices, such as consumers. In a preferred embodiment of the present invention, the transformed data represents physical and tangible objects, including particular visual depictions of physical and tangible objects produced on the consumer.
The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the gist of the present invention.
Claims (11)
1. A method for segmenting a tongue image, the method comprising:
acquiring a tongue picture, and performing standard color processing on the tongue picture to obtain a first picture;
creating a VGG16 classification regression network, classifying a first picture, acquiring tongue information of the first picture, performing frame selection on the tongue information and performing color space conversion processing on the first picture, and further performing tongue picture detection to obtain a second picture comprising a rough tongue picture position;
determining a corresponding threshold range of the second picture according to a Y-Cb-Cr color space conversion algorithm, intersecting the threshold range with a frame selection range, and performing tongue picture cutting processing to obtain a third picture;
creating an SRGAN super-resolution generation network, and performing multi-detail reduction on the third picture to generate a fourth image with higher definition;
and (3) creating a neural connection network based on an encoder-decoder structure, training and testing the fourth picture, and outputting a complete segmentation tongue picture.
2. The method for segmenting the tongue picture according to claim 1, wherein the performing standard color processing on the tongue picture comprises:
creating a BP neural connection network, training the tongue picture by taking a standard color card value as a known input, wherein the known input color card value is taken as a supervision value, and executing supervision training to obtain a trained parameter value;
initializing the BP neural connection network through the parameter values, inputting the tongue picture for calculation, and outputting a corrected first picture, wherein the first picture is the corrected tongue picture.
3. The method for segmenting the tongue picture according to claim 2, wherein the training of the tongue picture with the standard color card value as the known input comprises:
taking 24 colors of the standard color card values as sample values of the BP neural connection network, performing a plurality of hidden layer calculations on RGB channel values of each sample value, and outputting the corrected RGB channel values;
comparing the corrected RGB channel value with the standard color card value, calculating an error, performing back propagation, and correcting all parameter values on a path according to an LM algorithm;
and circulating the previous step until the errors of all the parameter values are reduced to be within the set range.
4. The tongue image segmentation method according to claim 2, wherein the BP neural connection network comprises:
the standard color processing network learns the nonlinear relation between neural networks through the difference between the 24-color real color chip values and the color chip values in different scene pictures, and trains a weight model of the network by updating gradient values through back propagation;
training the network through the VGG16 classification regression network through the existing tongue picture labels, and training the network through the existing data through a picture augmentation strategy to obtain results of 5 dimensions, namely whether the results are tongue pictures and position information;
adopting Y-Cb-Cr color space conversion algorithm processing, including roughly determining the position information of the tongue picture through a color clustering strategy;
generating a network and a tongue segmentation network through the SRGAN super resolution;
using a ReLU (rectified Linear Unit) function as an activation function of the BP neural connection network, wherein the ReLU function is set as:
and x represents the output result of each forward propagation, the output result is mapped into a fixed range through an activation function, and the output result is used as the output function of the BP neural connection network through an overall average pooling layer and a Softmax layer.
5. The tongue image segmentation method according to claim 1, wherein the performing tongue image detection comprises:
converting the first picture into a Y-Cb-Cr color space after passing through a VGG16 classification regression network, calculating the similarity of all pixels in the first picture according to the two-dimensional Gaussian distribution of Cb and Cr of the chroma of different tongue colors in the first picture, and calculating the tongue color probability in a way of
Wherein x isaFor the tongue sample value of each pixel a, t denotes the pixel mean value C as a covariance matrix, and m denotes the total number of training samples.
6. The tongue image segmentation method according to claim 4, wherein the performing tongue image detection further comprises:
calculating the first image by an AdaBoost algorithm to obtain the second image of the relatively rough tongue picture position, further executing normalization, and extracting a 10x10 pixel area from the detected tongue picture center as a tongue picture color reference, wherein the calculation mode is as follows:
tm＝[Cbm,Crm]
tmexpressing the average value of different components of all the pixel points, and judging the probability value of the pixel points by calculating the Euclidean distance D between the pixel points and the average value, wherein
Sorting the Euclidean distances D of all the extracted pixels from small to large, taking the pixel points with the closer distances to obtain an extraction proportion, setting a threshold range according to different extraction proportions, and repeating the steps to obtain t as a finally obtained adaptive parameter;
and obtaining the position of the corresponding tongue picture by displaying in different color spaces, and cutting the position of the tongue picture to obtain the third picture comprising the whole complete tongue picture.
7. The method for segmenting the tongue image according to claim 6, wherein the clipping the position of the tongue image comprises: and intersecting the tongue picture range selected by the VGG16 network frame with the tongue picture range identified by the Y-Cb-Cr color space, taking the center point of the intersection as the center of a circle, and cutting by taking the boundary distance of 1.25 times as the radius.
8. The tongue image segmentation method according to claim 1, wherein the creating of the encoder-decoder structure-based neural connection network comprises:
a standard color processing strategy, wherein the network performs standardized correction on the color of the picture by learning the difference between a standard color card and a color card in the picture;
roughly positioning the position of the tongue picture by using a VGG16 classification regression network and a Y-Cb-Cr color space conversion algorithm;
the texture definition of the cut tongue image is improved through an SRGAN super-resolution generation network;
generating a network model by using the countermeasure, generating similar available data according to the data, and expanding the data set in a data augmentation mode;
performing convolution processing and deconvolution processing on the third picture, wherein a same filling mode is used in the convolution processing, and the size of the image is kept unchanged after the image is subjected to convolution operation;
in the deconvolution process, extending picture information by using bilinear difference values;
meanwhile, a ResNet18 residual module is used as a convolution layer in each convolution block, and 1/2 random abandon processing is performed during convolution processing to obtain a tongue picture comprising tongue texture information and high-level information.
9. The method for segmenting the tongue image according to claim 1, wherein the training and testing the third picture comprises:
in the training stage, the existing data are divided into a training set and a verification set according to a proportion, and final model parameters are obtained through multiple groups of cross verification;
and in the testing stage, the tongue picture obtained under the color space conversion is used as the input of the neural connection network, and a complete tongue picture segmentation picture is obtained.
10. A device for segmenting a tongue image, the device comprising a memory, a processor and a computer program stored in said memory and executable on said processor, characterized in that said processor implements the method steps of any of claims 1-9 when executing said computer program.
11. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method steps of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110881511.9A CN113781488A (en) | 2021-08-02 | 2021-08-02 | Tongue picture image segmentation method, apparatus and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110881511.9A CN113781488A (en) | 2021-08-02 | 2021-08-02 | Tongue picture image segmentation method, apparatus and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113781488A true CN113781488A (en) | 2021-12-10 |
Family
ID=78836501
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110881511.9A Pending CN113781488A (en) | 2021-08-02 | 2021-08-02 | Tongue picture image segmentation method, apparatus and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113781488A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115601358A (en) * | 2022-12-01 | 2023-01-13 | 合肥云诊信息科技有限公司（Cn） | Tongue picture image segmentation method under natural light environment |
| CN117094966A (en) * | 2023-08-21 | 2023-11-21 | 青岛美迪康数字工程有限公司 | Tongue image identification method and device based on image amplification and computer equipment |
| CN117197139A (en) * | 2023-11-07 | 2023-12-08 | 天津市肿瘤医院（天津医科大学肿瘤医院） | Tongue diagnosis image multi-label classification method based on AI |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103106386A (en) * | 2011-11-10 | 2013-05-15 | 华为技术有限公司 | Dynamic self-adaption skin color segmentation method and device |
| KR20140104558A (en) * | 2013-02-19 | 2014-08-29 | 대전대학교 산학협력단 | New tongue diagnosis model system developed by analyzing the color of the tongue |
| CN105303152A (en) * | 2014-07-15 | 2016-02-03 | 中国人民解放军理工大学 | Human body re-recognition method |
| CN106909883A (en) * | 2017-01-17 | 2017-06-30 | 北京航空航天大学 | A kind of modularization hand region detection method and device based on ROS |
| CN107316307A (en) * | 2017-06-27 | 2017-11-03 | 北京工业大学 | A kind of Chinese medicine tongue image automatic segmentation method based on depth convolutional neural networks |
| CN107977671A (en) * | 2017-10-27 | 2018-05-01 | 浙江工业大学 | A kind of tongue picture sorting technique based on multitask convolutional neural networks |
| CN108734108A (en) * | 2018-04-24 | 2018-11-02 | 浙江工业大学 | A kind of fissured tongue recognition methods based on SSD networks |
| CN109815860A (en) * | 2019-01-10 | 2019-05-28 | 中国科学院苏州生物医学工程技术研究所 | TCM tongue diagnosis image color correction method, electronic equipment, storage medium |
| CN110136062A (en) * | 2019-05-10 | 2019-08-16 | 武汉大学 | A kind of super resolution ratio reconstruction method of combination semantic segmentation |
| US20210038198A1 (en) * | 2019-08-07 | 2021-02-11 | Siemens Healthcare Gmbh | Shape-based generative adversarial network for segmentation in medical imaging |
-
2021
- 2021-08-02 CN CN202110881511.9A patent/CN113781488A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103106386A (en) * | 2011-11-10 | 2013-05-15 | 华为技术有限公司 | Dynamic self-adaption skin color segmentation method and device |
| KR20140104558A (en) * | 2013-02-19 | 2014-08-29 | 대전대학교 산학협력단 | New tongue diagnosis model system developed by analyzing the color of the tongue |
| CN105303152A (en) * | 2014-07-15 | 2016-02-03 | 中国人民解放军理工大学 | Human body re-recognition method |
| CN106909883A (en) * | 2017-01-17 | 2017-06-30 | 北京航空航天大学 | A kind of modularization hand region detection method and device based on ROS |
| CN107316307A (en) * | 2017-06-27 | 2017-11-03 | 北京工业大学 | A kind of Chinese medicine tongue image automatic segmentation method based on depth convolutional neural networks |
| CN107977671A (en) * | 2017-10-27 | 2018-05-01 | 浙江工业大学 | A kind of tongue picture sorting technique based on multitask convolutional neural networks |
| CN108734108A (en) * | 2018-04-24 | 2018-11-02 | 浙江工业大学 | A kind of fissured tongue recognition methods based on SSD networks |
| CN109815860A (en) * | 2019-01-10 | 2019-05-28 | 中国科学院苏州生物医学工程技术研究所 | TCM tongue diagnosis image color correction method, electronic equipment, storage medium |
| CN110136062A (en) * | 2019-05-10 | 2019-08-16 | 武汉大学 | A kind of super resolution ratio reconstruction method of combination semantic segmentation |
| US20210038198A1 (en) * | 2019-08-07 | 2021-02-11 | Siemens Healthcare Gmbh | Shape-based generative adversarial network for segmentation in medical imaging |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115601358A (en) * | 2022-12-01 | 2023-01-13 | 合肥云诊信息科技有限公司（Cn） | Tongue picture image segmentation method under natural light environment |
| CN117094966A (en) * | 2023-08-21 | 2023-11-21 | 青岛美迪康数字工程有限公司 | Tongue image identification method and device based on image amplification and computer equipment |
| CN117094966B (en) * | 2023-08-21 | 2024-04-05 | 青岛美迪康数字工程有限公司 | Tongue image identification method and device based on image amplification and computer equipment |
| CN117197139A (en) * | 2023-11-07 | 2023-12-08 | 天津市肿瘤医院（天津医科大学肿瘤医院） | Tongue diagnosis image multi-label classification method based on AI |
| CN117197139B (en) * | 2023-11-07 | 2024-02-02 | 天津市肿瘤医院（天津医科大学肿瘤医院） | Tongue diagnosis image multi-label classification method based on AI |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10482603B1 (en) | Medical image segmentation using an integrated edge guidance module and object segmentation network | |
| Pinaya et al. | Unsupervised brain imaging 3D anomaly detection and segmentation with transformers | |
| CN110120040B (en) | Slice image processing method, slice image processing device, computer equipment and storage medium | |
| US20200320685A1 (en) | Automated classification and taxonomy of 3d teeth data using deep learning methods | |
| CN113781488A (en) | Tongue picture image segmentation method, apparatus and medium | |
| US11935213B2 (en) | Laparoscopic image smoke removal method based on generative adversarial network | |
| CN109492668A (en) | MRI based on multichannel convolutive neural network not same period multi-mode image characterizing method | |
| CN114399510B (en) | Skin focus segmentation and classification method and system combining image and clinical metadata | |
| CN111539911A (en) | Mouth breathing face recognition method, device and storage medium | |
| CN117975170B (en) | Medical information processing method and system based on big data | |
| CN112862745B (en) | Training method and training system for tissue lesion recognition based on artificial neural network | |
| Wang et al. | Optic disc detection based on fully convolutional neural network and structured matrix decomposition | |
| Codella et al. | Segmentation of both diseased and healthy skin from clinical photographs in a primary care setting | |
| CN115761226A (en) | Oral cavity image segmentation identification method and device, electronic equipment and storage medium | |
| CN112862787B (en) | CTA image data processing method, device and storage medium | |
| CN114648512A (en) | Sublingual image analysis method, apparatus, computer device and storage medium | |
| CN115578400A (en) | Image processing method, and training method and device of image segmentation network | |
| CN112862785A (en) | CTA image data identification method, device and storage medium | |
| CN117315378B (en) | Grading judgment method for pneumoconiosis and related equipment | |
| US20220254012A1 (en) | Methods, devices, and systems for determining presence of appendicitis | |
| Zou et al. | Research on Object Recognition Detection Based on Image Processing and Machine Learning | |
| Koe | Design of Predictive Model for TCM Tongue Diagnosis In Malaysia Using Machine Learning | |
| Aydoghmishi | Skin Cancer Detection by Deep Learning Algorithms | |
| Masood et al. | Optimized Skin Lesion Segmentation: Analysing DeepLabV3+ and ASSP Against Generative AI-Based Deep Learning Approach | |
| Carvajal | AN END TO END PIPELINE TO LOCALIZE NUCLEI IN MICROSCOPIC ZEBRAFISH EMBRYO IMAGES |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |