CN113706515B - Tongue image anomaly determination method, tongue image anomaly determination device, computer equipment and storage medium - Google Patents
Tongue image anomaly determination method, tongue image anomaly determination device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN113706515B CN113706515B CN202111016133.4A CN202111016133A CN113706515B CN 113706515 B CN113706515 B CN 113706515B CN 202111016133 A CN202111016133 A CN 202111016133A CN 113706515 B CN113706515 B CN 113706515B
- Authority
- CN
- China
- Prior art keywords
- tongue
- image
- brightness
- sub
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000005286 illumination Methods 0.000 claims description 39
- 230000002159 abnormal effect Effects 0.000 claims description 35
- 230000008859 change Effects 0.000 claims description 34
- 238000012545 processing Methods 0.000 claims description 22
- 230000011218 segmentation Effects 0.000 claims description 19
- 238000003709 image segmentation Methods 0.000 claims description 16
- 238000001914 filtration Methods 0.000 claims description 15
- 238000002310 reflectometry Methods 0.000 claims description 14
- 210000000746 body region Anatomy 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 12
- 238000003706 image smoothing Methods 0.000 claims description 11
- 239000011159 matrix material Substances 0.000 claims description 10
- 238000012549 training Methods 0.000 claims description 9
- 238000012706 support-vector machine Methods 0.000 claims description 7
- 230000005856 abnormality Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000002708 enhancing effect Effects 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims description 2
- 238000013473 artificial intelligence Methods 0.000 abstract description 3
- 239000011248 coating agent Substances 0.000 description 12
- 238000000576 coating method Methods 0.000 description 12
- 238000003745 diagnosis Methods 0.000 description 9
- 239000003814 drug Substances 0.000 description 9
- 206010043949 Tongue discolouration Diseases 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 210000004185 liver Anatomy 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000012935 Averaging Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000002405 diagnostic procedure Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 238000009499 grossing Methods 0.000 description 3
- 210000005181 root of the tongue Anatomy 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013145 classification model Methods 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 238000004195 computer-aided diagnosis Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000004141 dimensional analysis Methods 0.000 description 1
- 238000012850 discrimination method Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 239000003623 enhancer Substances 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 210000002216 heart Anatomy 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 210000003734 kidney Anatomy 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 210000004072 lung Anatomy 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000005316 response function Methods 0.000 description 1
- 210000000952 spleen Anatomy 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 210000002784 stomach Anatomy 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
- 230000004304 visual acuity Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration by the use of histogram techniques
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Abstract
The application relates to the field of artificial intelligence and digital medical treatment, and provides a tongue image anomaly determination method, a tongue image anomaly determination device, a tongue image anomaly determination computer device and a tongue image storage medium.
Description
Technical Field
The present disclosure relates to the field of artificial intelligence and digital medical technology, and in particular, to a tongue image anomaly determination method, apparatus, computer device, and storage medium.
Background
The diagnostic method of traditional Chinese medicine mainly comprises four diagnostic methods of inspection, smelling, asking and cutting. The tongue diagnosis is a method for observing the state and change characteristics of the tongue, tongue coating, tongue shape, etc. of a patient to examine the disease, and is an important content of inspection and one of the characteristics of the diagnostic methods of traditional Chinese medicine. With the development of science and technology, doctors can carry out tongue diagnosis with the aid of a computer, in the traditional Chinese medicine tongue diagnosis of the computer-aided diagnosis, a clear tongue image is usually acquired by a personal mobile phone or a special tongue diagnostic instrument, and then a disease symptom existing in the tongue image is analyzed by a computer intelligent algorithm program, so that the purpose of the traditional Chinese medicine diagnosis is achieved.
The existing intelligent algorithm mostly takes the tongue image as a two-dimensional image to carry out intelligent analysis, and the judgment of a plurality of abnormal tongue images in traditional Chinese medicine tongue diagnosis needs to judge the three-dimensional shape of the tongue, and human eyes have the natural advantages of identifying the three-dimensional depth and shape of a target in the image. For example, whether teeth marks are formed on two sides of the tongue, whether pits are formed on the front, middle and rear parts of the tongue, whether liver depression tables are formed on two sides of the tongue or not and the like are judged, and the three-dimensional analysis is carried out on the two-dimensional image according to human eyes. However, the deep learning of the existing popular artificial intelligence technology requires a large amount of data annotation, and is difficult to annotate three-dimensional information, and no such annotation data set exists, so that the conventional deep learning technology is difficult to reconstruct the three-dimensional surface of the tongue in tongue diagnosis of traditional Chinese medicine.
Disclosure of Invention
The main purpose of the application is to provide a tongue image anomaly determination method, a tongue image anomaly determination device, computer equipment and a storage medium, and aims to solve the technical problem that tongue images are inconvenient to perform anomaly analysis on a three-dimensional surface.
In order to achieve the above object, the present application provides a tongue image anomaly determination method, including the following steps:
acquiring an image to be detected, and performing image segmentation on the image to be detected to obtain a tongue image;
dividing the tongue image according to a preset rule to correspondingly obtain a tongue left side, a tongue right side, a tongue root, a tongue middle part and a tongue tip;
processing the tongue image through a CIELab color space to obtain a plurality of tongue sub-images;
determining an initial illumination direction from the tongue sub-image;
determining whether shadow areas exist in the tongue root, the tongue middle part and the tongue tip according to the tongue sub-image;
if yes, carrying out super-pixel segmentation on the tongue sub-image to obtain a super-pixel image;
detecting whether a crack exists in the tongue sub-image, if so, extracting a crack area, and forming a first superpixel set by superpixels corresponding to each pixel positioned in the crack area;
judging whether reflecting points exist in the tongue image according to the tongue sub-image, if so, extracting reflecting areas, and forming super pixels corresponding to the reflecting points into a second super pixel set;
Classifying the super-pixel image through a preset classifier to obtain a classification result; dividing the tongue sub-image according to the classification result, the crack area and the reflection area; the preset classifier is obtained based on training of a support vector machine; the segmented tongue sub-image comprises a plurality of target areas, and the reflectivity of the same target area is the same;
determining a third superpixel set according to the segmented tongue sub-image, calculating superpixel brightness of the third superpixel set, and taking the superpixel brightness and the initial illumination direction into a preset illumination model to calculate the light source direction;
calculating the normal direction of the super pixel of each target area according to the segmented tongue sub-image and the light source direction;
and determining whether the tongue image is abnormal according to the normal direction.
Further, the tongue sub-image includes an original luminance image; the step of determining an initial illumination direction according to the tongue sub-image comprises the following steps:
expanding a preset number of pixels outwards on the original brightness image along the image segmentation boundary;
dividing the expanded original brightness image by an otsu method to obtain a first area and a second area;
calculating the average brightness of each of the first area and the second area; if the difference value of the average brightness of the second area and the first area is larger than a preset value, determining the second area as a shadow area;
An initial illumination direction is determined from the shadow region.
Further, the plurality of tongue sub-images includes a tongue brightness image, a tongue red component image, a tongue yellow component image, a tongue green component image, and a tongue blue component image; the step of determining whether a shadow area exists in the tongue root, the tongue middle part and the tongue tip according to the tongue sub-image comprises the following steps:
forming a tongue body region by the tongue root, the tongue middle part and the tongue tip, and carrying out histogram statistics on pixels of the tongue body region;
determining an initial reflection brightness threshold according to the result of histogram statistics, and respectively carrying out seed point growth on a tongue brightness image, a tongue red component image, a tongue yellow component image, a tongue green component image and a tongue blue component image according to the initial reflection brightness threshold to obtain respective corresponding reflection candidate areas;
if the values of the reflection candidate areas corresponding to the tongue red component image, the tongue yellow component image, the tongue green component image and the tongue blue component image are smaller than the preset threshold value, removing the reflection candidate areas on the tongue brightness image to obtain a new tongue brightness image;
dividing the new tongue brightness image by an otsu method to obtain a target first region and a target second region;
And determining whether a shadow area exists in the tongue body area according to the target first area and the target second area.
Further, the tongue sub-image includes a tongue brightness image; the step of detecting whether a crack exists in the tongue sub-image comprises the following steps:
carrying out image smoothing on the tongue brightness image by utilizing Gaussian filtering;
enhancing the tongue brightness image after the image smoothing treatment by using Gabor filtering to obtain a target tongue brightness image;
and calculating a black plug matrix of each pixel point in the target tongue brightness, determining whether a ridge line exists in the tongue brightness image according to the black plug matrix, and if so, determining that the ridge line is a crack.
Further, the tongue sub-image includes an original color image; the step of classifying the super-pixel images by a preset classifier to obtain a classification result comprises the following steps:
processing the original color image through a Leung-Malik filter bank to obtain a corresponding feature vector;
and inputting the feature vector into a preset classifier to judge the category included in the original color image.
Further, the step of determining whether the tongue image is abnormal according to the normal direction includes:
Calculating an average normal line of a normal line direction between adjacent super pixels in a horizontal direction in a tongue body region formed by a tongue root part, a tongue middle part and a tongue tip part;
calculating the included angles between the normal directions of two adjacent super pixels and the average normal respectively;
and determining whether the tongue image is abnormal or not according to the included angle.
Further, the step of determining whether the tongue image is abnormal according to the normal direction includes:
acquiring the normal direction of each super pixel in the left side or the right side of the tongue;
calculating a first rate of change in a normal direction between adjacent superpixels along a boundary of the image segmentation; if the first change rate is larger than a preset first change rate threshold value, determining that the tongue image is abnormal;
or calculating a second rate of change in the normal direction between adjacent superpixels in the horizontal direction; and if the second change rate is larger than a preset second change rate threshold value, determining that the tongue image is abnormal.
The application also provides a tongue image abnormity determining device, which comprises:
the acquisition unit is used for acquiring an image to be detected and carrying out image segmentation on the image to be detected to obtain a tongue image;
the dividing unit is used for dividing the tongue image according to a preset rule to correspondingly obtain a tongue left side, a tongue right side, a tongue root, a tongue middle part and a tongue tip;
The processing unit is used for processing the tongue image through a CIELab color space to obtain a plurality of tongue sub-images;
a first determining unit for determining an initial illumination direction according to the tongue sub-image;
a second determining unit for determining whether a shadow area exists in the tongue root, the tongue middle part and the tongue tip according to the tongue sub-image;
the super-pixel segmentation unit is used for performing super-pixel segmentation on the tongue sub-image if the tongue sub-image exists, so as to obtain a super-pixel image;
the detection unit is used for detecting whether a crack exists in the tongue sub-image, if so, extracting a crack area, and forming a first superpixel set by superpixels corresponding to each pixel positioned in the crack;
the judging unit is used for judging whether reflecting points exist in the tongue image according to the tongue sub-image, if so, extracting reflecting areas, and forming super pixels corresponding to the reflecting points into a second super pixel set;
the classification unit is used for classifying the super-pixel images through a preset classifier to obtain classification results; dividing the tongue sub-image according to the classification result, the crack area and the reflection area; the preset classifier is obtained based on training of a support vector machine; the segmented tongue sub-image comprises a plurality of target areas, and the reflectivity of the same target area is the same;
The first computing unit is used for determining a third superpixel set according to the segmented tongue sub-image, computing superpixel brightness of the third superpixel set, and taking the superpixel brightness and the initial illumination direction into a preset illumination model to compute the light source direction;
the second calculating unit is used for calculating the normal direction of the super pixel of each target area according to the segmented tongue sub-image and the light source direction;
and a third determining unit for determining whether the tongue image is abnormal according to the normal direction.
The application also provides a computer device, which comprises a memory and a processor, wherein the memory stores a computer program, and the processor realizes the steps of the tongue image abnormity determination method when executing the computer program.
The present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the tongue image anomaly determination method of any one of the above.
According to the tongue image anomaly determination method, device, computer equipment and storage medium, a plurality of tongue sub-images are obtained from tongue images based on colors, whether shadows, reflections, cracks and the like exist in the images or not is determined on the basis of the tongue sub-images, then the super-pixel images are classified by means of the classification model to obtain target areas, fine segmentation of the images on the basis of textures and brightness is achieved, a light source method is further calculated by means of the illumination model, the influence of the shadows, the reflections and the cracks is eliminated by calculation of the light source direction, the light source direction is more accurate, the normal direction of each super-pixel obtained according to the light source direction can be more accurate, three-dimensional reconstruction of the tongue can be achieved according to the normal direction, whether the tongue images are abnormal or not is accurately judged on the basis of three dimensions, and more accurate results are provided for subsequent traditional Chinese medicine tongue diagnosis.
Drawings
FIG. 1 is a schematic diagram showing steps of a method for determining tongue image anomalies according to an embodiment of the present application;
FIG. 2 is a block diagram of a tongue image abnormality determination apparatus according to an embodiment of the present application;
fig. 3 is a block diagram schematically illustrating a structure of a computer device according to an embodiment of the present application.
The realization, functional characteristics and advantages of the present application will be further described with reference to the embodiments, referring to the attached drawings.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
Referring to fig. 1, an embodiment of the present application provides a tongue image anomaly determination method, including the following steps:
step S1, obtaining an image to be detected, and carrying out image segmentation on the image to be detected to obtain a tongue image;
step S2, dividing the tongue image according to a preset rule, and correspondingly obtaining a tongue left side, a tongue right side, a tongue root, a tongue middle part and a tongue tip;
s3, processing the tongue image through a CIELab color space to obtain a plurality of tongue sub-images;
S4, determining an initial illumination direction according to the tongue sub-image;
step S5, determining whether shadow areas exist in the tongue root, the tongue middle part and the tongue tip according to the tongue sub-image;
step S6, if yes, performing super-pixel segmentation on the tongue sub-image to obtain a super-pixel image;
step S7, detecting whether a crack exists in the tongue sub-image, if so, extracting a crack area, and forming a first superpixel set by superpixels corresponding to each pixel positioned in the crack area;
step S8, judging whether reflecting points exist in the tongue image according to the tongue sub-image, if so, extracting reflecting areas, and forming super pixels corresponding to the reflecting points into a second super pixel set;
step S9, classifying the super-pixel image through a preset classifier to obtain a classification result; dividing the tongue sub-image according to the classification result, the crack area and the reflection area; the preset classifier is obtained based on training of a support vector machine; the segmented tongue sub-image comprises a plurality of target areas, and the reflectivity of the same target area is the same;
step S10, determining a third superpixel set according to the segmented tongue sub-image, calculating superpixel brightness of the third superpixel set, and taking the superpixel brightness and the initial illumination direction into a preset illumination model to calculate a light source direction;
Step S11, calculating the normal direction of the super pixel of each target area according to the segmented tongue sub-image and the light source direction;
step S12, determining whether the tongue image is abnormal according to the normal direction.
In this embodiment, as described in step S1, the image to be detected is obtained by the camera, and during the obtaining, the lower half area of the face is certainly collected, even the non-face area includes the unknown environment area, so that the image to be detected is subjected to image segmentation, the tongue ROI (region of interest, the region of interest) is extracted, and the tongue area is outlined from the processed image in a square, circular, elliptical or irregular polygon manner, so as to obtain the tongue image. Specifically, the UNet neural network can be used for dividing the tongue region, the UNet neural network is trained on hundreds of tongue dividing and labeling results, the dividing results of the UNet neural network obtain a black-white binary image mask, the white region is the tongue region, and the region of the corresponding white region in the image to be detected is the tongue image.
As described in the above step S2, the tongue is divided according to the tongue image obtained in the step S1, and the mask is generally elliptical, so that the two long and narrow regions of the tongue are divided into left side and right side of the tongue according to the geometric definition, and correspond to the liver and gall region; the remaining area is divided into three blocks from top to bottom, namely a tongue root part, a tongue middle part and a tongue tip part, corresponding to kidney, spleen and stomach and heart lung.
As described in the above step S3, since the tongue diagnosis in traditional Chinese medicine needs to carefully examine the different colors and distribution of tongue coating, the tongue image is processed through the CIELab color space to obtain 7 tongue sub-images, which are the original color image I, the original brightness image IL, the tongue brightness image tlil, the tongue red component image tIR, the tongue yellow component image tIY, the tongue green component image tIG and the tongue blue component image tIB, respectively.
CIELab is a color system of CIE, a color system, based on CIELab means based on the color system, for determining numerical information of a certain color. The CIELab color space is used to convert a color RGB image into an image in Lab space, wherein for each pixel L, the brightness is taken, the part of a larger than zero is taken as a red component, the part of a smaller than zero is taken as a green component, the part of b larger than zero is taken as a yellow component, the part of b smaller than zero is taken as a blue component, for example, after one pixel is converted into Lab, L is 78, a is 36, b is-53, the red component is 36, the green component is 0, the yellow component is 0, and the blue component is 53.
As described in the above step S4, when the image to be detected is captured, shadows appear around the tongue due to the ambient light source, and we first determine whether there are obvious shadows around the tongue on the original brightness image, if so, the initial illumination direction can be roughly determined due to the ambient light source. Considering that when an image to be detected is actually shot, the root of the tongue is generally blocked by the mouth, so that the root of the tongue is always shaded, therefore, we firstly examine whether the left side, the right side and the outer edge below the tongue are obviously shaded, and when the left side edge of the tongue is shaded, the initial illumination direction is roughly judged to be right to left in sequence; if the edge on the right side of the tongue is shaded, the initial illumination direction is roughly judged to be from left to right in sequence; if the edge below the tongue is shaded, the initial illumination direction is roughly judged to be from top to bottom in sequence.
As described in the above step S5, whether shadow areas exist in the tongue root, the tongue middle part and the tongue tip is determined according to the tongue brightness image, and there are several possible reasons for these shadows: a. when the ambient light irradiates in certain directions, the ambient light is blocked by other parts of the human face, which are not the tongue; b. when the ambient light irradiates in certain directions, the tongue surface is concave or convex; c. when a tongue image is shot, the shooting equipment such as a mobile phone shields the ambient light. The occurrence of the depressions or protrusions on the tongue surface indicates that the tongue image is abnormal, and whether the shadow areas of the tongue surface are caused by the depressions or protrusions on the tongue surface or not, that is, whether the shadow areas are caused by the reason b, needs to be judged by means of the subsequent steps.
As described in the above step S6, the tongue brightness image is subjected to superpixel segmentation, and the superpixel is a small region composed of a series of pixel points which are adjacent in position and have similar characteristics of color, brightness, texture and the like. These small areas mostly retain the effective information for further image segmentation and do not generally destroy the boundary information of the objects in the image.
Specifically, a Normalized Cut method capable of simultaneously utilizing texture and brightness is adopted to perform superpixel segmentation on a tongue brightness image, so that a superpixel image is obtained, the Normalized Cut is a grouping technology, and the method has wide application in the aspects of data processing and image processing, a picture is regarded as a graph, then a weighted graph is calculated, and then the weighted graph is segmented into a plurality of areas with the same characteristics (texture, color, brightness and the like). The parameter k is 200, that is, at least 200 superpixels are divided, and the average brightness of the inner pixels in each superpixel is calculated and is used as the brightness of the corresponding superpixel.
As described in the above step S7, when the brightness of a super pixel is significantly different from that of an adjacent super pixel, it is possible that the normal direction of the super pixel is different from that of the adjacent super pixel, or that the reflectivity of the super pixel is different from that of the adjacent super pixel, and that the tongue crack is a structure with very different reflectivity from that of other regions, so that we need to extract the crack. According to the visual characteristics of the tongue crack, a ridge line (namely the tongue crack) is extracted through a ridge line extraction algorithm, wherein the ridge line is composed of a plurality of points, after the ridge line is extracted, super pixels corresponding to each pixel on the ridge line are formed into a first super pixel set, and for each super pixel in the first super pixel set, the brightness of the super pixel is replaced by the brightness average value of the adjacent super pixels in the periphery, which are not in the first super pixel set.
As described in the above step S8, the extraction of the reflection area is performed, the histogram statistics is performed on the pixels of the whole tongue area, a threshold T is found, once T is found, all pixels with brightness greater than T are found on the tongue brightness image, and seed point growth is performed to obtain alternative reflection points, then alternative reflection points are obtained from the tongue red component image, the tongue yellow component image, the tongue green component image and the tongue blue component image, and these alternative reflection points are discriminated, and if the pixel values on the corresponding two sub-images are less than 5, the true reflection points are determined. The reflection point can be positioned to the corresponding super-pixel to obtain a second super-pixel set, and for each super-pixel in the second super-pixel set, if more than half of the points in the second super-pixel set are judged to be reflection points, the brightness value of the super-pixel is replaced by the average value of the brightness of the super-pixel in the non-first super-pixel set adjacent to the periphery, otherwise, the brightness value of the super-pixel in the non-reflection point in the second super-pixel set is replaced by the average value of the brightness of the pixels in the inner non-reflection point.
As described in the above step S9, the super-pixel image is accurately segmented, that is, the tongue surface area is segmented into areas with different reflectivities, because different texture substances correspond to different reflectivities. The method comprises the steps of training a classifier in advance, marking a hundred tongue images during training, dividing each tongue image into a plurality of areas such as light red tongue white coating, light red tongue yellow coating, red tongue white coating, red tongue yellow coating, red tongue black coating, red tongue no coating, light white tongue white coating, light white tongue black coating, purple tongue white coating, purple tongue black coating, pricking and blood stasis, extracting texture and brightness characteristics of each pixel of each area by using a filter bank, and obtaining samples of the categories of the areas according to the characteristics of each corresponding pixel, wherein one pixel is one sample, so that too many samples do not need to be marked. The classifier can identify the regions belonging to the same class in the super-pixel image, the regions belonging to the same class are used as a target region, and the crack region and the reflection region are used as a target region.
As described in the above step S10, for the target area obtained in the above step S9, the superpixels corresponding to the shadow area obtained in the step S5 are removed, and the third superpixel set with the largest homogeneous connectivity is found, that is, all the superpixels in the third superpixel set are of the same texture, that is, the reflectivity is substantially the same and not shadow, crack and reflection, and the difference between the normal directions of the adjacent superpixels is small.
Substituting the superpixel brightness and the initial illumination direction in the third superpixel set into a preset illumination model, and estimating the accurate light source direction L by using a least square method. Specifically, let the brightness of the pixel on the tongue image (I, j) coordinate be I (i,j) The actual three-dimensional tongue surface shape is Z (x, y), i.e., an unknown value on the Z axis. Let the normal direction at the point (i, j) beIs the reflectivity at that point. The tongue surface is assumed to be Lambertian plane, with no secondary reflections from each other. In tongue images, there is generally a more common irradiance I of ambient light a In addition, there is irradiance I of point light source p Assume that the light source direction is +.>We can approximate the luminance value of this pixel with the following model:
wherein->
As can be seen, for each particular tongue image,I p and I a Is fixed, the variation factor is mainly the normal direction +>And reflectance ρ d 。
As described in the above steps S11-S12, according to the division of different target areas and the corresponding super-pixel brightness obtained in the step S9, and the light source direction obtained in the step S10, the normal direction of all super-pixels of each target area is calculated by taking into the preset illumination model, and whether the tongue image has a depression, a protrusion, a tooth trace or a liver depression is determined according to the normal direction, if so, the tongue image is abnormal.
In the embodiment, the normal direction of each super pixel is calculated through the illumination model, three-dimensional reconstruction of the tongue is realized, whether the tongue image is abnormal or not is judged on the basis of three dimensions, whether the tongue surface has pits or protrusions, tooth marks and liver depression tables can be judged, and a lot of useful information is provided for the subsequent traditional Chinese medicine tongue diagnosis. Meanwhile, the method does not need to mark the three-dimensional shape of the tongue, only uses a small amount of segmentation marks of texture primitives, and only uses a single tongue image, so that the method can be applied to very wide application scenes.
In one embodiment, the tongue sub-image comprises an original luminance image; the step S2 of determining the initial illumination direction according to the tongue sub-image comprises the following steps:
step S201, expanding a preset number of pixels outwards along the image segmentation boundary on the original brightness image;
step S202, segmenting the expanded original brightness image by an otsu method to obtain a first region and a second region;
step S203, calculating the average brightness of each of the first area and the second area;
step S204, if the difference value of the average brightness of the second area and the first area is larger than the preset value, determining the second area as a shadow area;
Step S205, determining an initial illumination direction according to the shadow area.
In this embodiment, considering that when a tongue image is actually photographed, the root of a tongue is blocked by the mouth, so that the root of the tongue is always shaded, we first examine whether there is a distinct shadow on the two sides of the tongue and the outer edge under the tongue, and then, by computing the sum of the specific gravity and the variance to obtain the value of the inter-class variance of the image under the threshold, then, computing the inter-class variance of the image based on the other threshold (if the distribution of the image is spread over 256 gray values, then, computing 256 gray values in sequence), and finally, obtaining the sub-class variance of the image under the maximum T-class variance by performing the sub-class variance segmentation on the image under the threshold. An image with a pixel statistic value greater than the threshold value is taken as a first area, and an image with a pixel statistic value less than the threshold value is taken as a second area. If the average brightness of the first area is more than 10 less than the average brightness of the second area, the second area is considered as a shadow area, and the corresponding illumination directions are roughly judged to be right-to-left, left-to-right and top-to-bottom in sequence.
In an embodiment, the plurality of tongue sub-images includes a tongue brightness image, a tongue red component image, a tongue yellow component image, a tongue green component image, and a tongue blue component image; the step S5 of determining whether a shadow area exists in the tongue root, the tongue middle part and the tongue tip according to the tongue sub-image includes:
step S501, forming a tongue body area by the tongue root, the tongue middle part and the tongue tip, and carrying out histogram statistics on pixels of the tongue body area;
step S502, determining an initial reflection brightness threshold according to the result of histogram statistics, and respectively carrying out seed point growth on a tongue brightness image, a tongue red component image, a tongue yellow component image, a tongue green component image and a tongue blue component image according to the initial reflection brightness threshold to obtain respective corresponding reflection candidate areas;
step S503, if the values of the reflection candidate areas corresponding to the tongue red component image, the tongue yellow component image, the tongue green component image and the tongue blue component image are smaller than the preset threshold value, removing the reflection candidate areas on the tongue brightness image to obtain a new tongue brightness image;
step S504, dividing the new tongue brightness image by an otsu method to obtain a target first area and a target second area;
Step S505, determining whether a shadow area exists in the tongue area according to the target first area and the target second area.
In this embodiment, only the tongue root, middle and tip regions in step S2 are considered, three regions are formed into a tongue region, the pixels in the tongue region are subjected to histogram statistics, and a threshold is found between 90 and 100 as an initial reflection brightness threshold, and the discrimination method is to minimize the intra-class variance of the pixels larger than T. And (3) carrying out seed point growth according to the initial reflection brightness threshold value to obtain reflection candidate areas, judging each candidate reflection area again on four sub-images of the tongue red component image, the tongue yellow component image, the tongue green component image and the tongue blue component image, and if four corresponding values of the areas are smaller than 5, namely the corresponding red component, green component, yellow component and blue component are smaller than 5, judging the light-emitting candidate area as a real reflection area, and removing the reflection areas to obtain a new tongue brightness image. The otsu method is performed on the new tongue brightness image to obtain a target first area and a target second area, the judgment of the target first area and the target second area is the same as that of the first area and the second area in the previous embodiment, if the area of the segmented target second area is too small, the tongue area is considered to be free of obvious shadow, namely, the number of pixels in the target second area is smaller than a preset area threshold (such as 100), and otherwise, the tongue area is considered to be free of shadow.
In another embodiment, if the average luminance of the segmented target second region and the average luminance of the target first region differ less, then no distinct shadows are considered to exist, otherwise.
In one embodiment, the tongue sub-image comprises a tongue intensity image; the step S7 of detecting whether a crack exists in the tongue sub-image includes:
step S701, performing image smoothing processing on the tongue brightness image by Gaussian filtering;
step S702, enhancing the tongue brightness image after the image smoothing processing by using Gabor filtering to obtain a target tongue brightness image;
step S703, calculating a black matrix of each pixel point in the target tongue brightness, determining whether a ridge line exists in the tongue brightness image according to the black matrix, and if so, determining that the ridge line is a crack.
In this embodiment, as described in the above step S701, the tongue luminance image is subjected to image smoothing processing by gaussian filtering, and the image smoothing is an area enhancement algorithm, and the smoothing algorithm includes a neighborhood averaging method, middle finger filtering, boundary preserving class filtering, and the like. During image generation, transmission and copying, it is often disturbed by noise or data loss occurs for various reasons, reducing the quality of the image (a pixel is infected by noise if it is significantly different from surrounding pixels). This requires some enhancement of the image to reduce the impact of these defects. In this embodiment, the image smoothing is performed by using a gaussian filtering method, which is a method for smoothing an image based on the idea of neighborhood averaging, and pixels at different positions are given different weights when the image is averaged.
As described in step S702, the tongue brightness image is processed by Gabor filtering, and the smoothed image is enhanced by Gabor filtering of 9*9 in six directions of 0 degrees, 30 degrees, 60 degrees, 90 degrees, 120 degrees, and 150 degrees, and since the cracks are relatively dark areas, the minimum value after the 6 filter values is taken as the enhanced image pixel value for each pixel in the image.
The main ideas of Gabor filtering method are: different textures generally have different center frequencies and bandwidths, according to which a set of Gabor filters can be designed to filter the texture image, each Gabor filter only allows the texture corresponding to its frequency to pass smoothly, while the energy of other textures is suppressed, and the texture features are analyzed and extracted from the output results of the filters for later classification or segmentation tasks. The Gabor filter extraction of texture features mainly comprises two processes: (1) filters (e.g., functions, numbers, directions, and spacing) are designed; (2) and extracting an effective texture feature set from the output result of the filter. The Gabor filter is a band-pass filter whose unit impulse response function (Gabor function) is the product of a gaussian function and a complex exponential function. The method is a function reaching the lower bound of the time-frequency measurement inaccuracy relationship, and has the resolving power of the best considering the time-frequency domain of the signal.
As described in step S703, for each pixel in the processed image, a black plug (Hessian) matrix is calculated from the surrounding pixel values:two sets of eigenvalues and eigenvectors can be obtained from matrix H: lambda (lambda) 1 ,λ 2 ,ξ 1 ,ξ 2 ；λ 1 <-10, construct a feature +.>Wherein->Is the first derivative. Any point (i, j) and its eight neighbors (i+k, j+l) obey Q (i, j) Q (i+k, j+l) (ζ 1 (i,j)·ξ 1 (i+k,j+l))<0; for any point (i, j) and its eight neighbors (i+k, j+l), satisfy +.>
When the above conditions are satisfied at the same time, the point (i, j) is a point on the ridge line, thereby obtaining all the points located on the ridge line.
In one embodiment, the tongue sub-image comprises an original color image; the step S9 of classifying the super-pixel image by a preset classifier to obtain a classification result includes:
processing the original color image through a Leung-Malik filter bank to obtain a corresponding feature vector;
and inputting the feature vector into a preset classifier to judge the category included in the original color image.
In this embodiment, a Leung-Malik filter bank is used, and the obtained filter response values form a feature vector, and multiple classification is performed by using a classifier. And (3) removing the superpixels of the first superpixel set and the second superpixel set for the superpixel image obtained in the step S6, calculating the Leung-Malik filter group characteristic of the central pixel of each superpixel, and judging which tongue image region category belongs to by using a trained classifier.
In an embodiment, the step S12 of determining whether the tongue image is abnormal according to the normal direction includes:
step S12a, calculating an average normal of normal directions between adjacent super pixels in the horizontal direction in a tongue body region formed by a tongue root part, a tongue middle part and a tongue tip part;
step S12b, calculating the included angles between the normal directions of two adjacent super pixels and the average normal respectively;
and step S12c, determining whether the tongue image is abnormal or not according to the included angle.
In this embodiment, each superpixel calculates a corresponding normal direction, the root portion, the middle portion and the tip portion of the tongue form a tongue body region, whether the tongue body region has a recess or a protrusion is determined according to the change condition of the normal directions of all the superpixels in the tongue body region, the normal direction is a normal vector, average normal is obtained by averaging the normal directions of two horizontally adjacent superpixels, the average normal can represent a plane, the included angle between the two superpixels and the average normal is calculated respectively, in the horizontal direction, the superpixel located at the left side is named as a, the superpixel located at the left side is named as B, when the included angle between the A and the average normal is greater than 90 degrees, the included angle between the B and the average normal is smaller than 90 degrees, the protrusion is determined when the included angle between the A and the average normal is greater than 90 degrees, the recess or the protrusion is determined to be present, and the tongue image is abnormal.
In an embodiment, the step S12 of determining whether the tongue image is abnormal according to the normal direction includes:
step S1201, obtaining the normal direction of each superpixel in the left side or right side of the tongue;
step S1202 of calculating a first rate of change in the normal direction between adjacent superpixels along the boundary of image division; if the first change rate is larger than a preset first change rate threshold value, determining that the tongue image is abnormal;
step S1203, or calculate a second rate of change in the normal direction between adjacent super pixels in the horizontal direction; and if the second change rate is larger than a preset second change rate threshold value, determining that the tongue image is abnormal.
In this embodiment, according to the physical feature distribution characteristics of the two sides of the tongue, we can consider that the reflectivity of the tongue after removing the superpixels of the reflective regions obtained in step S8 is the same, and the average brightness of the superpixels of each non-reflective region is known, according to the estimated light source directionAnd obtaining the normal direction of the superpixels on the two sides of the tongue.
And (2) looking at the first change rate of the normal direction of the adjacent super pixels according to the image segmentation edge direction in the step (S1), and if the first change rate is overlarge, namely the first change rate is larger than a preset first change rate threshold value, determining that tooth marks exist, and determining that the tongue image is abnormal.
And (3) observing a second change rate of the normal direction of the adjacent super pixels in the horizontal direction, and if the second change rate is overlarge, namely the second change rate is larger than a preset second change rate threshold value, considering that a liver depression table exists, and determining that the tongue image is abnormal.
Referring to fig. 2, an embodiment of the present application provides a tongue image anomaly determination device, including:
the acquisition unit 10 is used for acquiring an image to be detected and carrying out image segmentation on the image to be detected to obtain a tongue image;
the dividing unit 20 is configured to divide the tongue image according to a preset rule, and correspondingly obtain a left tongue side, a right tongue side, a root tongue, a middle tongue and a tip tongue;
a processing unit 30, configured to process the tongue image through a CIELab color space to obtain a plurality of tongue sub-images;
a first determining unit 40 for determining an initial illumination direction from the tongue sub-image;
a second determining unit 50 for determining whether a shadow area exists in the tongue root, the tongue middle part and the tongue tip according to the tongue sub-image;
a superpixel segmentation unit 60, configured to perform superpixel segmentation on the tongue sub-image, if any, to obtain a superpixel image;
the detecting unit 70 is configured to detect whether a crack exists in the tongue sub-image, and if so, extract a crack area, and form a first superpixel set from superpixels corresponding to each pixel located in the crack;
The judging unit 80 is configured to judge whether a reflection point exists in the tongue image according to the tongue sub-image, and if so, extract a reflection area, and form a second superpixel set by superpixels corresponding to each reflection point;
a classification unit 90, configured to classify the super-pixel image by using a preset classifier, so as to obtain a classification result; dividing the tongue sub-image according to the classification result, the crack area and the reflection area; the preset classifier is obtained based on training of a support vector machine; the segmented tongue sub-image comprises a plurality of target areas, and the reflectivity of the same target area is the same;
a first calculating unit 100, configured to determine a third superpixel set according to the segmented tongue sub-image, calculate superpixel brightness of the third superpixel set, and bring the superpixel brightness and the initial illumination direction into a preset illumination model to calculate a light source direction;
a second calculating unit 110 for calculating a normal direction of the super pixel of each target area according to the segmented tongue sub-image and the light source direction;
the third determining unit 120 is configured to determine whether the tongue image is abnormal according to the normal direction.
In an embodiment, the first determining unit 40 includes:
An expansion subunit, configured to expand a preset number of pixels outwards on the original luminance image along the boundary of image segmentation;
the first segmentation subunit is used for segmenting the expanded original brightness image by an otsu method to obtain a first region and a second region;
a first calculating subunit for calculating average brightness of each of the first region and the second region; if the difference value of the average brightness of the second area and the first area is larger than a preset value, determining the second area as a shadow area;
a first determination subunit for determining an initial illumination direction according to the shadow area.
In an embodiment, the second determining unit 50 includes:
the statistics subunit is used for forming a tongue body area from the tongue root, the tongue middle part and the tongue tip, and carrying out histogram statistics on pixels of the tongue body area;
the second determining subunit is used for determining an initial reflection brightness threshold according to the result of histogram statistics, and respectively carrying out seed point growth on the tongue brightness image, the tongue red component image, the tongue yellow component image, the tongue green component image and the tongue blue component image according to the initial reflection brightness threshold to obtain respective corresponding reflection candidate areas;
The removing subunit is used for removing the reflection candidate areas on the tongue brightness image to obtain a new tongue brightness image if the values of the reflection candidate areas corresponding to the tongue red component image, the tongue yellow component image, the tongue green component image and the tongue blue component image are smaller than a preset threshold value;
the second segmentation subunit is used for segmenting the new tongue brightness image through an otsu method to obtain a target first region and a target second region;
and the third determination subunit is used for determining whether the tongue body area has a shadow area according to the target first area and the target second area.
In one embodiment, the detecting unit 70 includes:
a smoothing processing subunit for performing image smoothing processing on the tongue brightness image by using Gaussian filtering;
the enhancer unit is used for enhancing the tongue brightness image after the image smoothing processing by utilizing Gabor filtering to obtain a target tongue brightness image;
and the second calculating subunit is used for calculating a black plug matrix of each pixel point in the target tongue brightness, determining whether a ridge line exists in the tongue brightness image according to the black plug matrix, and determining that the ridge line is a crack if the ridge line exists.
In one embodiment, the classifying unit 90 includes the steps of:
the processing subunit is used for processing the original color image through a Leung-Malik filter bank to obtain a corresponding feature vector;
and the input subunit is used for inputting the characteristic vector into a preset classifier to judge the category included in the original color image.
In an embodiment, the third determining unit 120 includes:
a third calculation subunit, configured to calculate an average normal line in a normal line direction between adjacent super pixels in a horizontal direction in a tongue body region formed by a tongue root portion, a tongue middle portion, and a tongue tip portion;
a fourth calculating subunit, configured to calculate angles between the normal directions of two adjacent super pixels and the average normal respectively;
and the fourth determination subunit is used for determining whether the tongue image is abnormal or not according to the included angle.
In an embodiment, the third determining unit 120 includes:
an acquisition subunit, configured to acquire a normal direction of each superpixel in the left side or the right side of the tongue;
a fifth calculation subunit for calculating a first rate of change in the normal direction between adjacent superpixels along the boundary of the image division; if the first change rate is larger than a preset first change rate threshold value, determining that the tongue image is abnormal;
A sixth calculation subunit configured to or calculate a second rate of change in the normal direction between adjacent super pixels in the horizontal direction; and if the second change rate is larger than a preset second change rate threshold value, determining that the tongue image is abnormal.
In this embodiment, the specific implementation of each unit and subunit is described in the foregoing method embodiment, and will not be described herein.
Referring to fig. 3, a computer device is further provided in the embodiment of the present application, where the computer device may be a server, and the internal structure of the computer device may be as shown in fig. 3. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the computer is configured to provide computing and control capabilities. The memory of the computer device includes a storage medium, an internal memory. The storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the storage media. The database of the computer device is used for storing data and the like. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program, when executed by a processor, implements a tongue image anomaly determination method.
Those skilled in the art will appreciate that the architecture shown in fig. 3 is merely a block diagram of a portion of the architecture in connection with the present application and is not intended to limit the computer device to which the present application is applied.
An embodiment of the present application further provides a computer readable storage medium having a computer program stored thereon, which when executed by a processor implements a tongue image anomaly determination method.
In summary, in the tongue image anomaly determination method, device, computer equipment and storage medium provided in the embodiments of the present application, an image to be detected is obtained, and image segmentation is performed on the image to be detected to obtain a tongue image; dividing the tongue image according to a preset rule to correspondingly obtain a tongue left side, a tongue right side, a tongue root, a tongue middle part and a tongue tip; processing the tongue image through a CIELab color space to obtain a plurality of tongue sub-images; determining an initial illumination direction from the tongue sub-image; determining whether shadow areas exist in the tongue root, the tongue middle part and the tongue tip according to the tongue sub-image; if yes, carrying out super-pixel segmentation on the tongue sub-image to obtain a super-pixel image; detecting whether a crack exists in the tongue sub-image, if so, extracting a crack area, and forming a first superpixel set by superpixels corresponding to each pixel positioned in the crack area; judging whether reflecting points exist in the tongue image according to the tongue sub-image, if so, extracting reflecting areas, and forming super pixels corresponding to the reflecting points into a second super pixel set; classifying the super-pixel image through a preset classifier to obtain a classification result; dividing the tongue sub-image according to the classification result, the crack area and the reflection area; the preset classifier is obtained based on training of a support vector machine; the segmented tongue sub-image comprises a plurality of target areas, and the reflectivity of the same target area is the same; determining a third superpixel set according to the segmented tongue sub-image, calculating superpixel brightness of the third superpixel set, and taking the superpixel brightness and the initial illumination direction into a preset illumination model to calculate the light source direction; calculating the normal direction of the super pixel of each target area according to the segmented tongue sub-image and the light source direction; and determining whether the tongue image is abnormal according to the normal direction. According to the method, whether shadows, reflections, ridge lines and the like exist in the image or not is determined, the super-pixel image is classified by means of the classification model, a target area is obtained, the normal direction of each super-pixel is further obtained by means of the illumination model, three-dimensional reconstruction of the tongue is achieved, and whether the tongue image is abnormal or not is judged on the basis of three dimensions.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by hardware associated with a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium provided herein and used in embodiments may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, apparatus, article or method that comprises the element.
The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the scope of the claims, and all equivalent structures or equivalent processes using the descriptions and drawings of the present application, or direct or indirect application in other related technical fields are included in the scope of the claims of the present application.
Claims (8)
1. The tongue image abnormality determination method is characterized by comprising the following steps:
acquiring an image to be detected, and performing image segmentation on the image to be detected to obtain a tongue image;
dividing the tongue image according to a preset rule to correspondingly obtain a tongue left side, a tongue right side, a tongue root, a tongue middle part and a tongue tip;
Processing the tongue image through a CIELab color space to obtain a plurality of tongue sub-images;
determining an initial illumination direction from the tongue sub-image;
determining whether shadow areas exist in the tongue root, the tongue middle part and the tongue tip according to the tongue sub-image;
if yes, carrying out super-pixel segmentation on the tongue sub-image to obtain a super-pixel image;
detecting whether a crack exists in the tongue sub-image, if so, extracting a crack area, and forming a first superpixel set by superpixels corresponding to each pixel positioned in the crack area;
judging whether reflecting points exist in the tongue image according to the tongue sub-image, if so, extracting reflecting areas, and forming super pixels corresponding to the reflecting points into a second super pixel set;
classifying the super-pixel image through a preset classifier to obtain a classification result; dividing the tongue sub-image according to the classification result, the crack area and the reflection area; the preset classifier is obtained based on training of a support vector machine; the segmented tongue sub-image comprises a plurality of target areas, and the reflectivity of the same target area is the same;
determining a third superpixel set according to the segmented tongue sub-image, calculating superpixel brightness of the third superpixel set, and taking the superpixel brightness and the initial illumination direction into a preset illumination model to calculate the light source direction;
Calculating the normal direction of the super pixel of each target area according to the segmented tongue sub-image and the light source direction;
determining whether the tongue image is abnormal according to the normal direction;
the plurality of tongue sub-images includes a tongue brightness image, a tongue red component image, a tongue yellow component image, a tongue green component image, and a tongue blue component image; the step of determining whether a shadow area exists in the tongue root, the tongue middle part and the tongue tip according to the tongue sub-image comprises the following steps:
forming a tongue body region by the tongue root, the tongue middle part and the tongue tip, and carrying out histogram statistics on pixels of the tongue body region;
determining an initial reflection brightness threshold according to the result of histogram statistics, and respectively carrying out seed point growth on a tongue brightness image, a tongue red component image, a tongue yellow component image, a tongue green component image and a tongue blue component image according to the initial reflection brightness threshold to obtain respective corresponding reflection candidate areas;
if the values of the reflection candidate areas corresponding to the tongue red component image, the tongue yellow component image, the tongue green component image and the tongue blue component image are smaller than the preset threshold value, removing the reflection candidate areas on the tongue brightness image to obtain a new tongue brightness image;
Dividing the new tongue brightness image by an otsu method to obtain a target first region and a target second region;
determining whether a shadow area exists in the tongue area according to the target first area and the target second area; the step of determining whether the tongue image is abnormal according to the normal direction comprises the following steps:
calculating an average normal line of a normal line direction between adjacent super pixels in a horizontal direction in a tongue body region formed by a tongue root part, a tongue middle part and a tongue tip part;
calculating the included angles between the normal directions of two adjacent super pixels and the average normal respectively;
and determining whether the tongue image is abnormal or not according to the included angle.
2. The tongue image abnormality determination method according to claim 1, wherein the tongue sub-image includes an original luminance image; the step of determining an initial illumination direction according to the tongue sub-image comprises the following steps:
expanding a preset number of pixels outwards on the original brightness image along the image segmentation boundary;
dividing the expanded original brightness image by an otsu method to obtain a first area and a second area;
calculating the average brightness of each of the first area and the second area; if the difference value of the average brightness of the second area and the first area is larger than a preset value, determining the second area as a shadow area;
An initial illumination direction is determined from the shadow region.
3. The tongue image anomaly determination method of claim 1, wherein the tongue sub-image comprises a tongue brightness image; the step of detecting whether a crack exists in the tongue sub-image comprises the following steps:
carrying out image smoothing on the tongue brightness image by utilizing Gaussian filtering;
enhancing the tongue brightness image after the image smoothing treatment by using Gabor filtering to obtain a target tongue brightness image;
and calculating a black plug matrix of each pixel point in the target tongue brightness, determining whether a ridge line exists in the tongue brightness image according to the black plug matrix, and if so, determining that the ridge line is a crack.
4. The tongue image anomaly determination method of claim 1, wherein the tongue sub-image comprises an original color image; the step of classifying the super-pixel image through a preset classifier to obtain a classification result comprises the following steps:
processing the original color image through a Leung-Malik filter bank to obtain a corresponding feature vector;
and inputting the feature vector into a preset classifier to judge the category included in the original color image.
5. The tongue image abnormality determination method according to claim 1, wherein the step of determining whether the tongue image is abnormal or not according to the normal direction includes:
Acquiring the normal direction of each super pixel in the left side or the right side of the tongue;
calculating a first rate of change in a normal direction between adjacent superpixels along a boundary of the image segmentation; if the first change rate is larger than a preset first change rate threshold value, determining that the tongue image is abnormal;
or calculating a second rate of change in the normal direction between adjacent superpixels in the horizontal direction; and if the second change rate is larger than a preset second change rate threshold value, determining that the tongue image is abnormal.
6. A tongue image abnormality determination apparatus, comprising:
the acquisition unit is used for acquiring an image to be detected and carrying out image segmentation on the image to be detected to obtain a tongue image;
the dividing unit is used for dividing the tongue image according to a preset rule to correspondingly obtain a tongue left side, a tongue right side, a tongue root, a tongue middle part and a tongue tip;
the processing unit is used for processing the tongue image through a CIELab color space to obtain a plurality of tongue sub-images;
a first determining unit for determining an initial illumination direction according to the tongue sub-image;
a second determining unit for determining whether a shadow area exists in the tongue root, the tongue middle part and the tongue tip according to the tongue sub-image;
The super-pixel segmentation unit is used for performing super-pixel segmentation on the tongue sub-image if the tongue sub-image exists, so as to obtain a super-pixel image;
the detection unit is used for detecting whether a crack exists in the tongue sub-image, if so, extracting a crack area, and forming a first superpixel set by superpixels corresponding to each pixel positioned in the crack;
the judging unit is used for judging whether reflecting points exist in the tongue image according to the tongue sub-image, if so, extracting reflecting areas, and forming super pixels corresponding to the reflecting points into a second super pixel set;
the classification unit is used for classifying the super-pixel images through a preset classifier to obtain classification results; dividing the tongue sub-image according to the classification result, the crack area and the reflection area; the preset classifier is obtained based on training of a support vector machine; the segmented tongue sub-image comprises a plurality of target areas, and the reflectivity of the same target area is the same;
the first computing unit is used for determining a third superpixel set according to the segmented tongue sub-image, computing superpixel brightness of the third superpixel set, and taking the superpixel brightness and the initial illumination direction into a preset illumination model to compute the light source direction;
The second calculating unit is used for calculating the normal direction of the super pixel of each target area according to the segmented tongue sub-image and the light source direction;
a third determining unit for determining whether the tongue image is abnormal according to the normal direction;
the second determination unit includes:
the statistics subunit is used for forming a tongue body area from the tongue root, the tongue middle part and the tongue tip, and carrying out histogram statistics on pixels of the tongue body area;
the second determining subunit is used for determining an initial reflection brightness threshold according to the result of histogram statistics, and respectively carrying out seed point growth on the tongue brightness image, the tongue red component image, the tongue yellow component image, the tongue green component image and the tongue blue component image according to the initial reflection brightness threshold to obtain respective corresponding reflection candidate areas;
the removing subunit is used for removing the reflection candidate areas on the tongue brightness image to obtain a new tongue brightness image if the values of the reflection candidate areas corresponding to the tongue red component image, the tongue yellow component image, the tongue green component image and the tongue blue component image are smaller than a preset threshold value;
the second segmentation subunit is used for segmenting the new tongue brightness image through an otsu method to obtain a target first region and a target second region;
The third determination unit includes:
a third calculation subunit, configured to calculate an average normal line in a normal line direction between adjacent super pixels in a horizontal direction in a tongue body region formed by a tongue root portion, a tongue middle portion, and a tongue tip portion;
a fourth calculating subunit, configured to calculate angles between the normal directions of two adjacent super pixels and the average normal respectively;
and the fourth determination subunit is used for determining whether the tongue image is abnormal or not according to the included angle.
7. A computer device comprising a memory and a processor, the memory having stored therein a computer program, characterized in that the processor, when executing the computer program, implements the steps of the tongue image anomaly determination method of any one of claims 1 to 5.
8. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the tongue image anomaly determination method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111016133.4A CN113706515B (en) | 2021-08-31 | 2021-08-31 | Tongue image anomaly determination method, tongue image anomaly determination device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111016133.4A CN113706515B (en) | 2021-08-31 | 2021-08-31 | Tongue image anomaly determination method, tongue image anomaly determination device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113706515A CN113706515A (en) | 2021-11-26 |
| CN113706515B true CN113706515B (en) | 2023-07-18 |
Family
ID=78658317
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111016133.4A Active CN113706515B (en) | 2021-08-31 | 2021-08-31 | Tongue image anomaly determination method, tongue image anomaly determination device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113706515B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114820603B (en) * | 2022-06-27 | 2022-09-16 | 深圳中易健康科技有限公司 | Intelligent health management method based on AI tongue diagnosis image processing and related device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1908984A (en) * | 2006-08-18 | 2007-02-07 | 清华大学 | Coated tongue division and extracting method for colored digital photo of tongue |
| CN102389293A (en) * | 2011-07-07 | 2012-03-28 | 中国中医科学院中医基础理论研究所 | Tongue diagnosis spectral image processing system |
| CN110298829A (en) * | 2019-06-19 | 2019-10-01 | 上海国民集团健康科技有限公司 | A kind of lingual diagnosis method, apparatus, system, computer equipment and storage medium |
| CN110807775A (en) * | 2018-08-06 | 2020-02-18 | 深圳市前海安测信息技术有限公司 | Traditional Chinese medicine tongue image segmentation device and method based on artificial intelligence and storage medium |
| CN111329452A (en) * | 2020-03-27 | 2020-06-26 | 上海祉云医疗科技有限公司 | Face-tongue diagnosis instrument |
| CN111860538A (en) * | 2020-07-20 | 2020-10-30 | 河海大学常州校区 | Tongue color identification method and device based on image processing |
-
2021
- 2021-08-31 CN CN202111016133.4A patent/CN113706515B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1908984A (en) * | 2006-08-18 | 2007-02-07 | 清华大学 | Coated tongue division and extracting method for colored digital photo of tongue |
| CN102389293A (en) * | 2011-07-07 | 2012-03-28 | 中国中医科学院中医基础理论研究所 | Tongue diagnosis spectral image processing system |
| CN110807775A (en) * | 2018-08-06 | 2020-02-18 | 深圳市前海安测信息技术有限公司 | Traditional Chinese medicine tongue image segmentation device and method based on artificial intelligence and storage medium |
| CN110298829A (en) * | 2019-06-19 | 2019-10-01 | 上海国民集团健康科技有限公司 | A kind of lingual diagnosis method, apparatus, system, computer equipment and storage medium |
| CN111329452A (en) * | 2020-03-27 | 2020-06-26 | 上海祉云医疗科技有限公司 | Face-tongue diagnosis instrument |
| CN111860538A (en) * | 2020-07-20 | 2020-10-30 | 河海大学常州校区 | Tongue color identification method and device based on image processing |
Non-Patent Citations (1)
| Title |
|---|
| 光谱法舌诊度量化中光源影响的研究;林凌;解鑫;李刚;;生命科学仪器(第05期);第1-5页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113706515A (en) | 2021-11-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Vidya et al. | Skin cancer detection using machine learning techniques | |
| WO2021129323A1 (en) | Ultrasound image lesion describing method and apparatus, computer device, and storage medium | |
| US10839510B2 (en) | Methods and systems for human tissue analysis using shearlet transforms | |
| Yang-Mao et al. | Edge enhancement nucleus and cytoplast contour detector of cervical smear images | |
| Olugbara et al. | Segmentation of melanoma skin lesion using perceptual color difference saliency with morphological analysis | |
| CN110119710A (en) | Cell sorting method, device, computer equipment and storage medium | |
| Sanchez-Torres et al. | Automatic measurement of fish weight and size by processing underwater hatchery images | |
| TWI673683B (en) | System and method for identification of symptom image | |
| CN107038704B (en) | Retina image exudation area segmentation method and device and computing equipment | |
| WO2022198898A1 (en) | Picture classification method and apparatus, and device | |
| CN110675464A (en) | Medical image processing method and device, server and storage medium | |
| CN108601509B (en) | Image processing apparatus, image processing method, and program-recorded medium | |
| CN110866932A (en) | Multi-channel tongue edge detection device and method and storage medium | |
| Ghazal et al. | Automated framework for accurate segmentation of leaf images for plant health assessment | |
| Kiruthika et al. | Matching of different rice grains using digital image processing | |
| CN113706515B (en) | Tongue image anomaly determination method, tongue image anomaly determination device, computer equipment and storage medium | |
| Priyadharsini et al. | Optic disc and cup segmentation in fundus retinal images using feature detection and morphological techniques | |
| Visalaxi et al. | Lesion extraction of endometriotic images using open computer vision | |
| Sibi Chakkaravarthy et al. | Automatic leaf vein feature extraction for first degree veins | |
| CN111222371A (en) | Sublingual vein feature extraction device and method | |
| CN110648336B (en) | Method and device for dividing tongue texture and tongue coating | |
| Kim et al. | Tongue diagnosis method for extraction of effective region and classification of tongue coating | |
| CN114359279B (en) | Image processing method, image processing device, computer equipment and storage medium | |
| Khan et al. | Segmentation of single and overlapping leaves by extracting appropriate contours | |
| CN115705748A (en) | Facial feature recognition system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |