CN113706515A - Tongue image abnormality determination method, tongue image abnormality determination device, computer device, and storage medium - Google Patents

Tongue image abnormality determination method, tongue image abnormality determination device, computer device, and storage medium Download PDF

Info

Publication number
CN113706515A
CN113706515A CN202111016133.4A CN202111016133A CN113706515A CN 113706515 A CN113706515 A CN 113706515A CN 202111016133 A CN202111016133 A CN 202111016133A CN 113706515 A CN113706515 A CN 113706515A
Authority
CN
China
Prior art keywords
tongue
image
brightness
sub
area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111016133.4A
Other languages
Chinese (zh)
Other versions
CN113706515B (en
Inventor
陈超
周宸
陈远旭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Priority to CN202111016133.4A priority Critical patent/CN113706515B/en
Publication of CN113706515A publication Critical patent/CN113706515A/en
Application granted granted Critical
Publication of CN113706515B publication Critical patent/CN113706515B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/40—Image enhancement or restoration using histogram techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Public Health (AREA)
  • Primary Health Care (AREA)
  • Evolutionary Biology (AREA)
  • Epidemiology (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The application relates to the field of artificial intelligence and digital medical treatment, and provides a tongue image abnormity determining method, a tongue image abnormity determining device, computer equipment and a storage medium.

Description

Tongue image abnormality determination method, tongue image abnormality determination device, computer device, and storage medium
Technical Field
The present application relates to the field of artificial intelligence and digital medical technology, and in particular, to a method and an apparatus for determining abnormal tongue image, a computer device, and a storage medium.
Background
The diagnosis methods of traditional Chinese medicine mainly include inspection, auscultation, inquiry and cutting. The tongue diagnosis is a method of observing the state and change characteristics of the tongue proper, tongue coating, tongue shape, etc. of a patient to diagnose the disease, is an important part of inspection and is one of the features of the diagnosis methods of traditional Chinese medicine. With the development of science and technology, doctors can perform tongue diagnosis with the aid of computer, and in the computer-aided diagnosis of traditional Chinese medicine tongue diagnosis, a clear tongue image is collected by a personal mobile phone or a special tongue diagnosis instrument, and disease symptoms existing in the tongue image are analyzed by using a computer intelligent algorithm program, so that the purpose of assisting the traditional Chinese medicine diagnosis is achieved.
Most of the existing intelligent algorithms take tongue images as two-dimensional images to be subjected to intelligent analysis, the judgment of a plurality of abnormal tongue images in the tongue diagnosis in traditional Chinese medicine needs to judge the three-dimensional shape of the tongue, and human eyes have the natural advantages of identifying the three-dimensional depth and the shape of a target in the images. For example, whether teeth marks exist on two sides of the tongue, whether depressions exist on the front, middle and rear parts of the tongue, whether liver depression exists on two sides of the tongue, and the like are obtained by performing three-dimensional analysis on a two-dimensional image according to human eyes. However, the current popular artificial intelligence technology deep learning needs a large amount of data labeling, the three-dimensional information is difficult to label, and such labeled data sets do not exist, so that the three-dimensional surface reconstruction of the tongue is difficult to perform in the traditional Chinese medicine tongue diagnosis by the common deep learning technology.
Disclosure of Invention
The application mainly aims to provide a tongue image abnormality determination method, a tongue image abnormality determination device, a computer device and a storage medium, and aims to solve the technical problem that a tongue image is inconvenient to analyze abnormalities on a three-dimensional surface.
In order to achieve the above object, the present application provides a tongue image abnormality determination method, including the steps of:
acquiring an image to be detected, and carrying out image segmentation on the image to be detected to obtain a tongue image;
dividing the tongue image according to a preset rule to correspondingly obtain the left side of the tongue, the right side of the tongue, the root of the tongue, the middle of the tongue and the tip of the tongue;
processing the tongue image through a CIELab color space to obtain a plurality of tongue sub-images;
determining an initial illumination direction according to the tongue subimage;
determining whether shadow regions exist in the tongue root, the tongue middle and the tongue tip according to the tongue sub-images;
if so, carrying out super-pixel segmentation on the tongue sub-image to obtain a super-pixel image;
detecting whether cracks exist in the tongue sub-images or not, if so, extracting crack regions, and forming a first superpixel set by superpixels corresponding to all pixels in the crack regions;
judging whether the tongue image has reflection points according to the tongue sub-image, if so, extracting a reflection area, and forming the super pixels corresponding to the reflection points into a second super pixel set;
classifying the super-pixel image through a preset classifier to obtain a classification result; segmenting the tongue subimage according to the classification result, the crack area and the light reflection area; the preset classifier is obtained based on training of a support vector machine; the segmented tongue sub-image comprises a plurality of target areas, and the reflectivity of the same target area is the same;
determining a third superpixel set according to the segmented tongue subimage, calculating superpixel brightness of the third superpixel set, and substituting the superpixel brightness and the initial illumination direction into a preset illumination model to calculate a light source direction;
calculating the normal direction of the super pixel of each target area according to the segmented tongue sub-image and the light source direction;
and determining whether the tongue image is abnormal according to the normal direction.
Further, the tongue sub-image comprises an original luminance image; the step of determining an initial illumination direction from the tongue sub-image comprises:
expanding a preset number of pixels outwards on the original brightness image along the boundary of image segmentation;
segmenting the expanded original brightness image by an otsu method to obtain a first region and a second region;
calculating the average brightness of each of the first region and the second region; if the difference value of the average brightness of the second area and the average brightness of the first area is larger than a preset value, determining that the second area is a shadow area;
and determining an initial illumination direction according to the shadow area.
Further, the plurality of tongue sub-images includes a tongue brightness image, a tongue red component image, a tongue yellow component image, a tongue green component image, and a tongue blue component image; the step of determining whether shadow regions exist in the tongue root, the tongue middle and the tongue tip according to the tongue sub-images comprises the following steps:
forming a tongue body area by the tongue root, the tongue middle and the tongue tip, and carrying out histogram statistics on pixels of the tongue body area;
determining an initial reflection brightness threshold according to the result of the histogram statistics, and respectively performing seed point growth on a tongue brightness image, a tongue red component image, a tongue yellow component image, a tongue green component image and a tongue blue component image according to the initial reflection brightness threshold to obtain respective corresponding reflection alternative areas;
if the values of the light reflecting alternative areas corresponding to the tongue red component image, the tongue yellow component image, the tongue green component image and the tongue blue component image are all smaller than a preset threshold value, removing the light reflecting alternative areas on the tongue brightness image to obtain a new tongue brightness image;
segmenting the new tongue brightness image by an otsu method to obtain a target first region and a target second region;
and determining whether a shadow area exists in the tongue body area according to the target first area and the target second area.
Further, the tongue sub-image comprises a tongue brightness image; the step of detecting whether a crack is present in the tongue sub-image comprises:
carrying out image smoothing on the tongue brightness image by using Gaussian filtering;
enhancing the tongue brightness image after the image smoothing by using Gabor filtering to obtain a target tongue brightness image;
calculating a black plug matrix of each pixel point in the target tongue brightness, determining whether a ridge line exists in the tongue brightness image according to the black plug matrix, and if the ridge line exists, determining that the ridge line is a crack.
Further, the tongue sub-image comprises an original color image; the step of obtaining a classification result by classifying the super-pixel image in a preset classifier comprises the following steps:
processing the original color image through a Leung-Malik filter bank to obtain a corresponding feature vector;
and inputting the feature vector into a preset classifier to judge the category included in the original color image.
Further, the step of determining whether the tongue image is abnormal according to the normal direction includes:
calculating the average normal of the normal direction between adjacent superpixels in the horizontal direction in the area of the tongue body consisting of the tongue root part, the tongue middle part and the tongue tip part;
calculating included angles between the normal directions of two adjacent superpixels and the average normal respectively;
and determining whether the tongue image is abnormal or not according to the included angle.
Further, the step of determining whether the tongue image is abnormal according to the normal direction includes:
acquiring the normal direction of each super pixel in the left side or the right side of the tongue;
calculating a first rate of change in a normal direction between adjacent superpixels along a boundary of an image segmentation; if the first change rate is larger than a preset first change rate threshold value, determining that the tongue image is abnormal;
or calculating a second rate of change in the normal direction between adjacent superpixels in the horizontal direction; and if the second change rate is greater than a preset second change rate threshold value, determining that the tongue image is abnormal.
The present application also provides a tongue image abnormality determination device, including:
the tongue image acquisition unit is used for acquiring an image to be detected and carrying out image segmentation on the image to be detected to obtain a tongue image;
the dividing unit is used for dividing the tongue image according to a preset rule to correspondingly obtain the left side of the tongue, the right side of the tongue, the root of the tongue, the middle part of the tongue and the tip part of the tongue;
the processing unit is used for processing the tongue image through a CIELab color space to obtain a plurality of tongue sub-images;
the first determining unit is used for determining an initial illumination direction according to the tongue sub-image;
the second determining unit is used for determining whether shadow areas exist in the tongue root, the tongue middle and the tongue tip according to the tongue sub-images;
the super-pixel segmentation unit is used for carrying out super-pixel segmentation on the tongue sub-image if the tongue sub-image exists so as to obtain a super-pixel image;
the detection unit is used for detecting whether cracks exist in the tongue sub-images or not, if so, extracting a crack area, and forming the superpixels corresponding to the pixels positioned in the cracks into a first superpixel set;
the judging unit is used for judging whether the tongue image has the reflecting points or not according to the tongue sub-image, if so, extracting the reflecting area, and forming the super pixels corresponding to the reflecting points into a second super pixel set;
the classification unit is used for classifying the super-pixel images through a preset classifier to obtain a classification result; segmenting the tongue subimage according to the classification result, the crack area and the light reflection area; the preset classifier is obtained based on training of a support vector machine; the segmented tongue sub-image comprises a plurality of target areas, and the reflectivity of the same target area is the same;
the first calculation unit is used for determining a third superpixel set according to the segmented tongue subimages, calculating superpixel brightness of the third superpixel set, and substituting the superpixel brightness and the initial illumination direction into a preset illumination model to calculate a light source direction;
the second calculation unit is used for calculating the normal direction of the super-pixels of each target area according to the segmented tongue sub-images and the light source direction;
and the third determining unit is used for determining whether the tongue image is abnormal or not according to the normal direction.
The present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and the processor implements the steps of the tongue image abnormality determination method according to any one of the above methods when executing the computer program.
The present application also provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the tongue image abnormality determination method according to any one of the above.
The tongue image abnormity determining method, the tongue image abnormity determining device, the computer equipment and the storage medium provided by the application can be used for obtaining a plurality of tongue sub-images based on colors of a tongue image, determining whether shadows, reflection, cracks and the like exist in the tongue sub-images, classifying super-pixel images by means of a classification model to obtain a target area, realizing fine segmentation of the image on the basis of textures and brightness, further calculating a light source method by means of an illumination model, and eliminating the influences of the shadows, the reflection and the cracks by means of calculation of the light source direction, so that the light source direction is more accurate, the normal direction of each super-pixel obtained according to the light source direction can be more accurate, the three-dimensional reconstruction of the tongue can be realized according to the normal direction, whether the tongue image is abnormal or not can be accurately judged on the basis of three dimensions, and more accurate results can be provided for subsequent tongue diagnosis in traditional Chinese medicine.
Drawings
FIG. 1 is a schematic diagram illustrating a method for determining abnormal tongue image according to an embodiment of the present application;
FIG. 2 is a block diagram of a tongue abnormality determination apparatus according to an embodiment of the present application;
fig. 3 is a block diagram illustrating a structure of a computer device according to an embodiment of the present application.
The implementation, functional features and advantages of the objectives of the present application will be further explained with reference to the accompanying drawings.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Referring to fig. 1, an embodiment of the present application provides a method for determining a tongue image abnormality, including the following steps:
step S1, acquiring an image to be detected, and carrying out image segmentation on the image to be detected to obtain a tongue image;
step S2, dividing the tongue image according to a preset rule to correspondingly obtain the left side of the tongue, the right side of the tongue, the root of the tongue, the middle of the tongue and the tip of the tongue;
step S3, processing the tongue image through a CIELab color space to obtain a plurality of tongue sub-images;
step S4, determining an initial illumination direction according to the tongue sub-image;
step S5, determining whether shadow areas exist in the tongue root, the tongue middle part and the tongue tip according to the tongue sub-images;
step S6, if the tongue sub-image exists, carrying out super-pixel segmentation on the tongue sub-image to obtain a super-pixel image;
step S7, detecting whether cracks exist in the tongue sub-images, if so, extracting crack regions, and forming a first superpixel set by superpixels corresponding to each pixel in the crack regions;
step S8, judging whether the tongue image has a reflection point according to the tongue sub-image, if so, extracting a reflection area, and forming a second super-pixel set by the super-pixels corresponding to the reflection points;
step S9, classifying the super-pixel image through a preset classifier to obtain a classification result; segmenting the tongue subimage according to the classification result, the crack area and the light reflection area; the preset classifier is obtained based on training of a support vector machine; the segmented tongue sub-image comprises a plurality of target areas, and the reflectivity of the same target area is the same;
step S10, determining a third superpixel set according to the segmented tongue subimages, calculating superpixel brightness of the third superpixel set, and substituting the superpixel brightness and the initial illumination direction into a preset illumination model to calculate a light source direction;
step S11, calculating the normal direction of the superpixel of each target area according to the segmented tongue sub-image and the light source direction;
in step S12, it is determined whether the tongue image is abnormal or not based on the normal direction.
In this embodiment, as described in step S1 above, the image to be detected is obtained by the camera, and during the obtaining, the lower half area of the face is collected definitely, and even the non-face area includes an unknown environment area, so that the image to be detected is segmented, a tongue ROI (area of interest) is extracted, and the area of the tongue is outlined from the processed image in a manner of a square frame, a circle, an ellipse, an irregular polygon, or the like, so as to obtain a tongue image. Specifically, the tongue region can be segmented by using a UNet neural network, the Unet neural network is trained on hundreds of tongue segmentation and labeling results, the segmentation result of the Unet neural network obtains a black-and-white binary image mask, the white region is the tongue region, and the region of the corresponding white region in the image to be detected is the tongue image.
As described in step S2, since the mask is generally elliptical, the tongue is divided into two narrow regions on both sides of the tongue, which are geometrically defined as the left side and the right side of the tongue, corresponding to the hepatobiliary region, based on the tongue image obtained in step S1; the remaining area is divided into three parts from top to bottom, which are the root part, middle part and tip part of the tongue, corresponding to the kidney, spleen and stomach, heart and lung.
As described in the step S3, in the analysis of tongue color by tongue inspection in the traditional Chinese medicine, different colors and distributions of tongue coating need to be carefully checked, and the tongue image is processed through the CIELab color space to obtain 7 tongue sub-images, which are the original color image I, the original luminance image IL, the tongue luminance image tli, the tongue red component image tIR, the tongue yellow component image tIY, the tongue green component image tIG, and the tongue blue component image tIB.
CIELab is a color system of the CIE, the color system, based on which means that numerical information for determining a certain color is based on the color system. Using the CIELab color space, that is, converting a color RGB image into an image in the Lab space, for each pixel L, brightness, a part greater than zero of a is a red component, a part less than zero of a is a green component, a part greater than zero of b is a yellow component, and a part less than zero of b is a blue component, for example, after a pixel is converted into Lab, L is 78, a is 36, b is-53, the red component is 36, the green component is 0, the yellow component is 0, and the blue component is 53.
As described in step S4, when the image to be detected is captured, the light source in the environment may cause shadows around the tongue, and we first determine whether there is an obvious shadow around the tongue on the original luminance image, and if so, the light source in the environment may cause the shadows, so as to roughly determine the initial illumination direction. Considering that the root of a tongue is blocked by the mouth when an image to be detected is actually shot, so that shadows often appear on the root of the tongue, the left side and the right side of the tongue and the outer edge below the tongue are firstly examined to see whether obvious shadows exist, and when the shadows appear on the edge of the left side of the tongue, the initial illumination direction is roughly judged to be from right to left in sequence; if the edge of the right side of the tongue is shaded, roughly judging the initial illumination direction to be from left to right in sequence; if the edge below the tongue is shaded, the initial illumination direction is roughly determined to be from top to bottom in sequence.
As described in the above step S5, it is determined whether there are shadow areas in the tongue root, the tongue middle and the tongue tip according to the tongue brightness image, and there are several possible reasons for these shadows: a. when the ambient light in certain directions is irradiated, the ambient light is shielded by other parts of the human face, which are not the tongue; b. when the environment light in certain directions irradiates, the light is caused by the dent or the protrusion of the tongue surface; c. when shooting the tongue image, the shooting equipment such as a mobile phone shields the ambient light. The tongue image is abnormal when the tongue surface is concave or convex, and the shadow areas of the tongue surface are not caused by the concave or convex of the tongue surface, namely whether the shadow areas are caused by the reason b or not, and the judgment needs to be carried out by means of subsequent steps.
As described in step S6, the tongue luminance image is subjected to superpixel segmentation, where the superpixel is a small region composed of a series of pixels with adjacent positions and similar characteristics such as color, luminance, texture, and the like. Most of these small regions retain effective information for further image segmentation, and generally do not destroy the boundary information of objects in the image.
Specifically, the tongue luminance image is subjected to superpixel segmentation by using a Normalized Cut method capable of simultaneously utilizing texture and luminance to obtain a superpixel image, the Normalized Cut is a clustering (clustering) technology, and has wide application in data processing and image processing, a picture is regarded as a graph (graph), then a weighted graph (weighted graph) is calculated, and the graph is segmented into regions with the same characteristics (texture, color, brightness and the like). The parameter k is 200, that is, at least 200 super pixels are divided, and the average brightness of the internal pixels in each super pixel is calculated as the corresponding super pixel brightness.
As described in step S7, when the brightness of a super pixel is significantly different from that of an adjacent super pixel, the normal direction of the super pixel may be different from that of the adjacent super pixel, or the reflectivity of the super pixel may be different from that of the adjacent super pixel, and the tongue crack is a structure having a very different reflectivity from other regions, so that the crack needs to be extracted. According to the visual characteristics of the tongue cracks, a ridge line (namely the tongue cracks) is extracted through a ridge line extraction algorithm, the ridge line is composed of a plurality of points, after the ridge line is extracted, super pixels corresponding to each pixel on the ridge line are composed into a first super pixel set, and for each super pixel in the first super pixel set, the super pixel brightness is replaced by the brightness average value of the super pixels which are adjacent at the periphery and are not in the first super pixel set.
As described in step S8, the light reflection region is extracted, histogram statistics is performed on pixels in the entire tongue region, a threshold T is found, once T is found, all pixels with a luminance greater than T are found on the tongue luminance image, seed point growth is performed to obtain alternative light reflection points, then alternative light reflection points are obtained in the tongue red component image, the tongue yellow component image, the tongue green component image, and the tongue blue component image, and these alternative light reflection points are determined, and if the pixel values in the two corresponding sub-images are less than 5, the light reflection points are determined to be true light reflection points. The reflection points can be positioned to the corresponding super-pixels to obtain a second super-pixel set, and for each super-pixel in the second super-pixel set, if more than half of the points in the super-pixel set are judged as the reflection points, the brightness value of the super-pixel is replaced by the average brightness value of the super-pixels in the non-first super-pixel set adjacent to the periphery, otherwise, the average brightness value of the pixels of the non-reflection points in the super-pixel set is used.
As described in step S9 above, the super-pixel image is accurately segmented, i.e. the tongue surface region is segmented into regions of different reflectivity, since different texture materials correspond to different reflectivities. The method comprises the steps of training a classifier in advance, marking about one hundred tongue images during training, dividing each tongue image into multiple types of regions, namely possible light red tongue white fur, light red tongue yellow fur, red tongue white fur, red tongue yellow fur, red tongue black fur, red tongue no fur, light white tongue white fur, light white tongue black fur, purple tongue white fur, purple tongue black fur, prickling and blood stasis, performing a filter set on each pixel of each region to extract texture and brightness characteristics, and enabling each corresponding pixel characteristic to be a sample of the region types, wherein one pixel is a sample, so that too many samples are not required to be marked. Regions belonging to the same class in the super-pixel image can be identified through the classifier, the regions belonging to the same class are used as a target region, and meanwhile, the crack region and the light reflection region are both used as a target region.
As described in step S10, for the target region obtained in step S9, the superpixels corresponding to the shaded region obtained in step S5 are removed, and a third superpixel set with the largest homogeneous connectivity is found, that is, all the superpixels in the third superpixel set are the same texture, that is, the reflectivity is substantially the same and is not shadow, crack or reflection, and the difference between the normal directions of the adjacent superpixels is small.
And substituting the super-pixel brightness and the initial illumination direction in the third super-pixel set into a preset illumination model, and estimating the accurate light source direction L by using a least square method. Specifically, let the brightness of the pixel on the tongue image (I, j) coordinate be I(i,j)The actual three-dimensional tongue surface shape is Z (x, y), i.e., an unknown value on the Z-axis. Let the normal direction at this point (i, j) be
Figure BDA0003240273230000091
Is the reflectivity at that point. The tongue surface is assumed to be a Lambertian plane with no secondary reflections from each other. In tongue diagnosis, there is generally a relatively common ambient light irradiance IaIn addition, irradiance I of point light sourcepAssuming a light source direction of
Figure BDA0003240273230000092
We can approximate the luminance value of the pixel with the following model:
Figure BDA0003240273230000093
wherein
Figure BDA0003240273230000094
Figure BDA0003240273230000095
It can be seen that, for each particular tongue image,
Figure BDA0003240273230000096
Ipand IaIs fixed and invariant, and the variation factor is mainly the normal direction
Figure BDA0003240273230000097
And reflectance ρd。
As described in the above steps S11-S12, the division of different target areas and their corresponding superpixel luminances obtained in step S9 and the light source direction obtained in step S10 are substituted into the preset illumination model to calculate the normal directions of all superpixels in each target area, and whether the tongue image has a depression, a protrusion, a tooth mark or a depressed liver platform is determined according to the normal directions, if so, it is determined that the tongue image is abnormal.
In the embodiment, the normal direction of each super pixel is calculated through the illumination model, so that the three-dimensional reconstruction of the tongue is realized, whether the tongue image is abnormal or not is judged on the three-dimensional basis, whether the tongue surface has depression or protrusion, tooth marks and liver depression or not can be judged, and a great deal of useful information is provided for the follow-up tongue diagnosis in traditional Chinese medicine. Meanwhile, the tongue three-dimensional shape does not need to be marked, only a small number of texture primitives are used for segmentation and marking, and moreover, only a single tongue image is used in the method, so that the method can be applied to a very wide application scene.
In one embodiment, the tongue sub-image comprises an original luminance image; the step S2 of determining the initial lighting direction according to the tongue sub-image includes:
step S201, expanding a preset number of pixels outwards on the original brightness image along the boundary of image segmentation;
step S202, segmenting the expanded original brightness image by an otsu method to obtain a first region and a second region;
step S203, calculating the average brightness of the first area and the second area respectively;
step S204, if the difference value of the average brightness of the second area and the average brightness of the first area is larger than a preset value, determining that the second area is a shadow area;
step S205, determining an initial illumination direction according to the shadow area.
In the embodiment, considering that the root of the tongue is blocked by the mouth when the tongue image is actually shot, so that the root of the tongue often appears shadows, we first consider whether the two sides of the tongue and the outer edge below the tongue have obvious shadows or not, outwardly expand along the boundary of image segmentation on the original brightness image, namely, outwardly expand 30-50 pixels in the left, right and lower directions, segment the original brightness image by an otsu method, segment the original brightness image into a first area and a second area, calculate the average brightness of the two areas, specifically, firstly, calculate the distribution histogram of the expanded original brightness image, obtain the pixel statistical value corresponding to each gray value, search a threshold value between 90 and 100, divide the image into a 'first' binary image, then respectively calculate three values of the foreground and background specific gravity, the gray average value, the variance and the like based on the histogram, and obtain the image inter-class variance calculated value under the threshold value by calculating the sum of the specific gravity and the variance, and then, calculating the inter-class variance of the image based on other thresholds (if the gray distribution of the image extends over 256 gray values, 256 times of calculation are needed, and so on), and finally segmenting the image by taking the T with the maximum inter-class variance as a threshold to obtain an otsu segmentation result of the image. An image having a pixel statistic value larger than the threshold value is defined as a first area, and an image having a pixel statistic value smaller than the threshold value is defined as a second area. If the average brightness of the first area is less than that of the second area by more than 10, the second area is considered as a shadow area, and the corresponding illumination directions are roughly determined to be from right to left, from left to right and from top to bottom in sequence.
In an embodiment, the plurality of tongue sub-images includes a tongue brightness image, a tongue red component image, a tongue yellow component image, a tongue green component image, and a tongue blue component image; the step S5 of determining whether there is a shadow region in the tongue root, the tongue middle and the tongue tip according to the tongue sub-image includes:
step S501, a tongue body area is formed by the tongue root, the tongue middle and the tongue tip, and histogram statistics is carried out on pixels of the tongue body area;
step S502, determining an initial reflection brightness threshold according to the result of histogram statistics, and respectively performing seed point growth on a tongue brightness image, a tongue red component image, a tongue yellow component image, a tongue green component image and a tongue blue component image according to the initial reflection brightness threshold to obtain respective corresponding reflection alternative areas;
step S503, if the values of the light reflecting alternative areas corresponding to the tongue red component image, the tongue yellow component image, the tongue green component image and the tongue blue component image are all smaller than a preset threshold value, removing the light reflecting alternative areas on the tongue brightness image to obtain a new tongue brightness image;
step S504, the new tongue brightness image is segmented through an otsu method to obtain a target first region and a target second region;
step S505, determining whether the tongue body area has a shadow area according to the target first area and the target second area.
In this embodiment, only the tongue root region, the tongue middle region, and the tongue tip region in step S2 are considered, the three regions are combined into a tongue body region, histogram statistics is performed on pixels in the tongue body region, a threshold is found between 90 and 100 as an initial reflection brightness threshold, and the determination method is to minimize the intra-class variance of pixels larger than T. And growing seed points according to the initial reflecting brightness threshold to obtain a reflecting alternative region, judging each alternative reflecting region again on the four sub-images of the tongue red component image, the tongue yellow component image, the tongue green component image and the tongue blue component image, if the four values corresponding to the regions are all smaller than 5, namely the corresponding red component, green component, yellow component and blue component are smaller than 5, judging the light-emitting alternative region as a real reflecting region, and removing the reflecting regions to obtain a new tongue brightness image. And (3) carrying out otsu method segmentation on the new tongue brightness image to obtain a target first region and a target second region, wherein the judgment of the target first region and the target second region is the same as that of the first region and the second region in the previous embodiment, if the area of the segmented target second region is too small, the tongue region is considered to have no obvious shadow, namely the number of pixels in the target second region is smaller than a preset area threshold (such as 100), the tongue region is considered to have no shadow region, and otherwise, the tongue region exists.
In another embodiment, if the average brightness of the segmented target second region is smaller than the average brightness of the target first region, no obvious shadow is considered to exist, otherwise, the segmentation is performed.
In one embodiment, the tongue sub-image comprises a tongue brightness image; the step S7 of detecting whether there is a crack in the tongue sub-image includes:
step S701, carrying out image smoothing on the tongue brightness image by using Gaussian filtering;
step S702, enhancing the tongue brightness image after the image smoothing by utilizing Gabor filtering to obtain a target tongue brightness image;
step S703, calculating a black plug matrix of each pixel point in the target tongue brightness, determining whether a ridge line exists in the tongue brightness image according to the black plug matrix, and if so, determining that the ridge line is a crack.
In this embodiment, as described in step S701 above, the gaussian filtering is used to perform image smoothing on the tongue luminance image, where the image smoothing is a region enhancement algorithm, and the smoothing algorithm includes a neighborhood averaging method, middle-finger filtering, boundary preservation-like filtering, and the like. In the image generation, transmission and reproduction process, noise interference or data loss often occurs for various reasons, and the image quality is reduced (a certain pixel is infected by noise if it is significantly different from surrounding pixel points). This requires some enhancement of the image to reduce the effect of these defects. In the embodiment, the image smoothing is performed by adopting a gaussian filtering method, which is a method for smoothing an image by using a neighborhood averaging idea, and when the image is averaged, pixels at different positions are given different weights.
As described in step S702, the tongue luminance image is processed by using Gabor filtering, and the smoothed image is enhanced by using 9 × 9 Gabor filtering in six directions of 0 degree, 30 degrees, 60 degrees, 90 degrees, 120 degrees, and 150 degrees, and since cracks are relatively dark regions, the minimum value of the 6 filtered values is taken for each pixel in the image as the enhanced image pixel value.
The main idea of the Gabor filtering method is as follows: different textures generally have different central frequencies and bandwidths, a group of Gabor filters can be designed according to the frequencies and the bandwidths to filter texture images, each Gabor filter only allows textures corresponding to the frequency of the Gabor filter to pass through smoothly, energy of other textures is restrained, and texture features are analyzed and extracted from output results of the filters and used for subsequent classification or segmentation tasks. The extraction of the texture features by the Gabor filter mainly comprises two processes: design filters (e.g., functions, numbers, directions, and spacings); and secondly, extracting an effective texture feature set from an output result of the filter. A Gabor filter is a band-pass filter whose unit impulse response function (Gabor function) is the product of a gaussian function and a complex exponential function. The method is a function for reaching the lower bound of the time-frequency inaccurate measurement relation and has the resolution capability of giving the best consideration to signals in the time-frequency domain.
As described in step S703 above, for each pixel in the processed image, a black plug (Hessian) matrix is calculated from the peripheral pixel values:
Figure BDA0003240273230000131
two sets of eigenvalues and eigenvectors can be obtained from matrix H: lambda [ alpha ]1,λ2,ξ1,ξ2；λ1<-10, building features at each point
Figure BDA0003240273230000132
Wherein
Figure BDA0003240273230000133
Is the first derivative. Any point (i, j) and its eight neighbourhood points (i + k, j + l) all obey Q (i, j) × Q (i + k, j + l) × ξ1(i,j)·ξ1(i+k,j+l))<0; for any point (i, j) and its eight neighborhood points (i + k, j + l), satisfy
Figure BDA0003240273230000134
When the above conditions are satisfied simultaneously, the point (i, j) is a point on the ridge line, and all points located on the ridge line are obtained.
In one embodiment, the tongue sub-image comprises an original color image; the step S9 of classifying the super-pixel image by a preset classifier to obtain a classification result includes:
processing the original color image through a Leung-Malik filter bank to obtain a corresponding feature vector;
and inputting the feature vector into a preset classifier to judge the category included in the original color image.
In this embodiment, a Leung-Malik filter bank is adopted, the obtained filter response value forms a feature vector, and a classifier is used for multi-classification. For the superpixel image obtained in step S6, the superpixels of the first superpixel set and the second superpixel set are removed, the Leung-Malik filter bank characteristic of the center pixel of each superpixel is calculated, and the trained classifier is used to determine which type of tongue image region belongs to the superpixel image.
In an embodiment, the step S12 of determining whether the tongue image is abnormal according to the normal direction includes:
step S12a, calculating the average normal of the normal direction between adjacent superpixels in the horizontal direction in the tongue body area formed by the tongue root, the tongue middle and the tongue tip;
step S12b, calculating included angles between the normal directions of two adjacent superpixels and the average normal respectively;
and step S12c, determining whether the tongue image is abnormal according to the included angle.
In this embodiment, each super pixel calculates its corresponding normal direction, the tongue root, the tongue middle part, and the tongue tip part constitute a tongue body region, and it is determined whether there is a dent or a bulge in the tongue body region according to the change of the normal directions of all super pixels in the tongue body region, where the normal direction is a normal vector, the normal directions of two horizontally adjacent super pixels are averaged to obtain an average normal, the average normal may represent a plane, the included angles between the two super pixels and the average normal are calculated respectively, in the horizontal direction, the super pixel on the left side is named as a, the super pixel on the left side is named as B, when the included angle between a and the average normal is greater than 90 degrees and the included angle between B and the average normal is less than 90 degrees, the super pixel on the left side is judged to have a bulge, when the included angle between a and the average normal is less than 90 degrees and the included angle between B and the average normal is greater than 90 degrees, the super pixel on the left side is judged to have a dent, when there is a concavity or convexity, the tongue image is abnormal.
In an embodiment, the step S12 of determining whether the tongue image is abnormal according to the normal direction includes:
step S1201, acquiring the normal direction of each super pixel in the left side or the right side of the tongue;
step S1202 of calculating a first rate of change in a normal direction between adjacent superpixels along a boundary of image segmentation; if the first change rate is larger than a preset first change rate threshold value, determining that the tongue image is abnormal;
step S1203, or calculating a second rate of change in the normal direction between adjacent superpixels in the horizontal direction; and if the second change rate is greater than a preset second change rate threshold value, determining that the tongue image is abnormal.
In this embodiment, according to the physical feature distribution characteristics of both sides of the tongue, it can be considered that the reflectivities of the super-pixels in the areas on both sides of the tongue after the super-pixels in the reflection areas obtained in the step S8 are removed are the same, and the average brightness value of the super-pixels in each non-reflection area is known, and the light source direction is estimated
Figure BDA0003240273230000141
The normal directions of the superpixels at both sides of the tongue are obtained.
And (5) observing a first change rate of the normal direction of the adjacent super pixels according to the image segmentation edge direction of the step (S1), and if the first change rate is too large, namely the first change rate is larger than a preset first change rate threshold value, considering that tooth marks exist, and determining that the tongue image has abnormity.
And inspecting a second change rate of the normal direction of the adjacent super pixels in the horizontal direction, and if the second change rate is too large, namely the second change rate is larger than a preset second change rate threshold value, determining that the liver depression stage exists, and determining that the tongue image is abnormal.
Referring to fig. 2, an embodiment of the present application provides a tongue image abnormality determination apparatus, including:
the tongue image segmentation device comprises an acquisition unit 10, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring an image to be detected and carrying out image segmentation on the image to be detected to obtain a tongue image;
the dividing unit 20 is used for dividing the tongue image according to a preset rule to correspondingly obtain the left side of the tongue, the right side of the tongue, the root of the tongue, the middle of the tongue and the tip of the tongue;
the processing unit 30 is configured to process the tongue image through a CIELab color space to obtain a plurality of tongue sub-images;
a first determining unit 40, configured to determine an initial illumination direction according to the tongue sub-image;
a second determination unit 50 for determining whether there is a shadow region in the tongue root, the tongue middle and the tongue tip from the tongue sub-image;
a super-pixel segmentation unit 60, configured to perform super-pixel segmentation on the tongue sub-image if the tongue sub-image exists, to obtain a super-pixel image;
the detection unit 70 is configured to detect whether a crack exists in the tongue sub-image, extract a crack region if the crack exists, and form a first superpixel set with superpixels corresponding to each pixel located in the crack;
the judging unit 80 is configured to judge whether the tongue image has a reflection point according to the tongue sub-image, extract a reflection area if the tongue image has the reflection point, and form superpixels corresponding to the reflection points into a second superpixel set;
the classification unit 90 is used for classifying the super-pixel images through a preset classifier to obtain a classification result; segmenting the tongue subimage according to the classification result, the crack area and the light reflection area; the preset classifier is obtained based on training of a support vector machine; the segmented tongue sub-image comprises a plurality of target areas, and the reflectivity of the same target area is the same;
the first calculating unit 100 is configured to determine a third superpixel set according to the segmented tongue sub-images, calculate superpixel brightness of the third superpixel set, and bring the superpixel brightness and the initial illumination direction into a preset illumination model to calculate a light source direction;
the second calculating unit 110 is configured to calculate a normal direction of the superpixel of each target region according to the segmented tongue sub-image and the light source direction;
a third determining unit 120, configured to determine whether the tongue image is abnormal according to the normal direction.
In an embodiment, the first determining unit 40 includes:
the expansion subunit is used for expanding a preset number of pixels outwards along the boundary of image segmentation on the original brightness image;
the first segmentation subunit is used for segmenting the expanded original brightness image by an otsu method to obtain a first region and a second region;
a first calculating subunit, configured to calculate an average luminance of each of the first region and the second region; if the difference value of the average brightness of the second area and the average brightness of the first area is larger than a preset value, determining that the second area is a shadow area;
a first determining subunit, configured to determine an initial illumination direction according to the shadow area.
In an embodiment, the second determining unit 50 includes:
the statistics subunit is used for forming a tongue body area by the tongue root, the tongue middle part and the tongue tip part and carrying out histogram statistics on pixels of the tongue body area;
the second determining subunit is used for determining an initial reflection brightness threshold according to the result of the histogram statistics, and performing seed point growth on the tongue brightness image, the tongue red component image, the tongue yellow component image, the tongue green component image and the tongue blue component image respectively according to the initial reflection brightness threshold to obtain respective corresponding reflection alternative areas;
the removing subunit is configured to remove the light-reflecting alternative region on the tongue luminance image if the values of the light-reflecting alternative regions corresponding to the tongue red component image, the tongue yellow component image, the tongue green component image and the tongue blue component image are all smaller than a preset threshold value, so as to obtain a new tongue luminance image;
the second segmentation subunit is used for segmenting the new tongue brightness image by an otsu method to obtain a target first region and a target second region;
and the third determining subunit is used for determining whether a shadow area exists in the tongue body area according to the target first area and the target second area.
In one embodiment, the detection unit 70 includes:
the smoothing subunit is used for carrying out image smoothing on the tongue brightness image by utilizing Gaussian filtering;
the enhancement unit is used for enhancing the tongue brightness image after the image smoothing by utilizing Gabor filtering to obtain a target tongue brightness image;
and the second calculating subunit is used for calculating a black plug matrix of each pixel point in the target tongue brightness, determining whether a ridge line exists in the tongue brightness image according to the black plug matrix, and if the ridge line exists, determining that the ridge line is a crack.
In one embodiment, the classification unit 90 includes the steps of:
the processing subunit is used for processing the original color image through a Leung-Malik filter bank to obtain a corresponding feature vector;
and the input subunit is used for inputting the feature vectors into a preset classifier to judge the categories included in the original color image.
In an embodiment, the third determining unit 120 includes:
the third calculation subunit is used for calculating the average normal of the normal directions between adjacent superpixels in the horizontal direction in a tongue body area formed by the tongue root part, the tongue middle part and the tongue tip part;
the fourth calculating subunit is used for calculating included angles between the normal directions of two adjacent superpixels and the average normal respectively;
and the fourth determining subunit is used for determining whether the tongue image is abnormal or not according to the included angle.
In an embodiment, the third determining unit 120 includes:
an acquisition subunit, configured to acquire a normal direction of each super pixel in the left side or the right side of the tongue;
a fifth calculating subunit for calculating a first rate of change in a normal direction between adjacent superpixels along a boundary of the image segmentation; if the first change rate is larger than a preset first change rate threshold value, determining that the tongue image is abnormal;
a sixth calculating subunit configured to calculate a second rate of change in the normal direction between adjacent super pixels in the horizontal direction; and if the second change rate is greater than a preset second change rate threshold value, determining that the tongue image is abnormal.
In this embodiment, please refer to the above method embodiment for the specific implementation of each unit and sub-unit, which is not described herein again.
Referring to fig. 3, a computer device, which may be a server and whose internal structure may be as shown in fig. 3, is also provided in the embodiment of the present application. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the computer designed processor is used to provide computational and control capabilities. The memory of the computer device comprises a storage medium and an internal memory. The storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and computer programs in the storage medium to run. The database of the computer device is used for storing data and the like. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a tongue abnormality determination method.
Those skilled in the art will appreciate that the architecture shown in fig. 3 is only a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects may be applied.
An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored, the computer program, when executed by a processor, implementing a method for determining a tongue abnormality.
In summary, for the tongue image abnormality determination method, apparatus, computer device and storage medium provided in the embodiments of the present application, an image to be detected is obtained, and the image to be detected is subjected to image segmentation to obtain a tongue image; dividing the tongue image according to a preset rule to correspondingly obtain the left side of the tongue, the right side of the tongue, the root of the tongue, the middle of the tongue and the tip of the tongue; processing the tongue image through a CIELab color space to obtain a plurality of tongue sub-images; determining an initial illumination direction according to the tongue subimage; determining whether shadow regions exist in the tongue root, the tongue middle and the tongue tip according to the tongue sub-images; if so, carrying out super-pixel segmentation on the tongue sub-image to obtain a super-pixel image; detecting whether cracks exist in the tongue sub-images or not, if so, extracting crack regions, and forming a first superpixel set by superpixels corresponding to all pixels in the crack regions; judging whether the tongue image has reflection points according to the tongue sub-image, if so, extracting a reflection area, and forming the super pixels corresponding to the reflection points into a second super pixel set; classifying the super-pixel image through a preset classifier to obtain a classification result; segmenting the tongue subimage according to the classification result, the crack area and the light reflection area; the preset classifier is obtained based on training of a support vector machine; the segmented tongue sub-image comprises a plurality of target areas, and the reflectivity of the same target area is the same; determining a third superpixel set according to the segmented tongue subimage, calculating superpixel brightness of the third superpixel set, and substituting the superpixel brightness and the initial illumination direction into a preset illumination model to calculate a light source direction; calculating the normal direction of the super pixel of each target area according to the segmented tongue sub-image and the light source direction; and determining whether the tongue image is abnormal according to the normal direction. The method and the device have the advantages that whether shadows, reflection, ridge lines and the like exist in the image or not is determined, then the super-pixel image is classified by means of the classification model, the target area is obtained, the normal direction of each super-pixel is further obtained by means of the illumination model, the three-dimensional reconstruction of the tongue is achieved, and whether the tongue image is abnormal or not is judged on the basis of three dimensions.
It will be understood by those skilled in the art that all or part of the processes of the methods of the above embodiments may be implemented by hardware associated with instructions of a computer program, which may be stored on a non-volatile computer-readable storage medium, and when executed, may include processes of the above embodiments of the methods. Any reference to memory, storage, database, or other medium provided herein and used in the examples may include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double-rate SDRAM (SSRSDRAM), Enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that includes the element.
The above description is only for the preferred embodiment of the present application and not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application, or which are directly or indirectly applied to other related technical fields, are intended to be included within the scope of the present application.

Claims (10)

1. A tongue image abnormality determination method is characterized by comprising the following steps:
acquiring an image to be detected, and carrying out image segmentation on the image to be detected to obtain a tongue image;
dividing the tongue image according to a preset rule to correspondingly obtain the left side of the tongue, the right side of the tongue, the root of the tongue, the middle of the tongue and the tip of the tongue;
processing the tongue image through a CIELab color space to obtain a plurality of tongue sub-images;
determining an initial illumination direction according to the tongue subimage;
determining whether shadow regions exist in the tongue root, the tongue middle and the tongue tip according to the tongue sub-images;
if so, carrying out super-pixel segmentation on the tongue sub-image to obtain a super-pixel image;
detecting whether cracks exist in the tongue sub-images or not, if so, extracting crack regions, and forming a first superpixel set by superpixels corresponding to all pixels in the crack regions;
judging whether the tongue image has reflection points according to the tongue sub-image, if so, extracting a reflection area, and forming the super pixels corresponding to the reflection points into a second super pixel set;
classifying the super-pixel image through a preset classifier to obtain a classification result; segmenting the tongue subimage according to the classification result, the crack area and the light reflection area; the preset classifier is obtained based on training of a support vector machine; the segmented tongue sub-image comprises a plurality of target areas, and the reflectivity of the same target area is the same;
determining a third superpixel set according to the segmented tongue subimage, calculating superpixel brightness of the third superpixel set, and substituting the superpixel brightness and the initial illumination direction into a preset illumination model to calculate a light source direction;
calculating the normal direction of the super pixel of each target area according to the segmented tongue sub-image and the light source direction;
and determining whether the tongue image is abnormal according to the normal direction.
2. The tongue image abnormality determination method according to claim 1, wherein the tongue sub-image includes an original luminance image; the step of determining an initial illumination direction from the tongue sub-image comprises:
expanding a preset number of pixels outwards on the original brightness image along the boundary of image segmentation;
segmenting the expanded original brightness image by an otsu method to obtain a first region and a second region;
calculating the average brightness of each of the first region and the second region; if the difference value of the average brightness of the second area and the average brightness of the first area is larger than a preset value, determining that the second area is a shadow area;
and determining an initial illumination direction according to the shadow area.
3. The tongue image abnormality determination method according to claim 1, wherein the plurality of tongue sub-images include a tongue brightness image, a tongue red component image, a tongue yellow component image, a tongue green component image, and a tongue blue component image; the step of determining whether shadow regions exist in the tongue root, the tongue middle and the tongue tip according to the tongue sub-images comprises the following steps:
forming a tongue body area by the tongue root, the tongue middle and the tongue tip, and carrying out histogram statistics on pixels of the tongue body area;
determining an initial reflection brightness threshold according to the result of the histogram statistics, and respectively performing seed point growth on a tongue brightness image, a tongue red component image, a tongue yellow component image, a tongue green component image and a tongue blue component image according to the initial reflection brightness threshold to obtain respective corresponding reflection alternative areas;
if the values of the light reflecting alternative areas corresponding to the tongue red component image, the tongue yellow component image, the tongue green component image and the tongue blue component image are all smaller than a preset threshold value, removing the light reflecting alternative areas on the tongue brightness image to obtain a new tongue brightness image;
segmenting the new tongue brightness image by an otsu method to obtain a target first region and a target second region;
and determining whether a shadow area exists in the tongue body area according to the target first area and the target second area.
4. The tongue image abnormality determination method according to claim 1, wherein the tongue sub-image includes a tongue brightness image; the step of detecting whether a crack is present in the tongue sub-image comprises:
carrying out image smoothing on the tongue brightness image by using Gaussian filtering;
enhancing the tongue brightness image after the image smoothing by using Gabor filtering to obtain a target tongue brightness image;
calculating a black plug matrix of each pixel point in the target tongue brightness, determining whether a ridge line exists in the tongue brightness image according to the black plug matrix, and if the ridge line exists, determining that the ridge line is a crack.
5. The tongue image abnormality determination method according to claim 1, wherein the tongue sub-image includes an original color image; the step of classifying the super-pixel image through a preset classifier to obtain a classification result comprises the following steps:
processing the original color image through a Leung-Malik filter bank to obtain a corresponding feature vector;
and inputting the feature vector into a preset classifier to judge the category included in the original color image.
6. The method for determining abnormality of tongue image according to claim 1, wherein said step of determining whether the tongue image is abnormal or not based on the normal direction includes:
calculating the average normal of the normal direction between adjacent superpixels in the horizontal direction in the area of the tongue body consisting of the tongue root part, the tongue middle part and the tongue tip part;
calculating included angles between the normal directions of two adjacent superpixels and the average normal respectively;
and determining whether the tongue image is abnormal or not according to the included angle.
7. The method for determining abnormality of tongue image according to claim 1, wherein said step of determining whether the tongue image is abnormal or not based on the normal direction includes:
acquiring the normal direction of each super pixel in the left side or the right side of the tongue;
calculating a first rate of change in a normal direction between adjacent superpixels along a boundary of an image segmentation; if the first change rate is larger than a preset first change rate threshold value, determining that the tongue image is abnormal;
or calculating a second rate of change in the normal direction between adjacent superpixels in the horizontal direction; and if the second change rate is greater than a preset second change rate threshold value, determining that the tongue image is abnormal.
8. A tongue image abnormality determination device characterized by comprising:
the tongue image acquisition unit is used for acquiring an image to be detected and carrying out image segmentation on the image to be detected to obtain a tongue image;
the dividing unit is used for dividing the tongue image according to a preset rule to correspondingly obtain the left side of the tongue, the right side of the tongue, the root of the tongue, the middle part of the tongue and the tip part of the tongue;
the processing unit is used for processing the tongue image through a CIELab color space to obtain a plurality of tongue sub-images;
the first determining unit is used for determining an initial illumination direction according to the tongue sub-image;
the second determining unit is used for determining whether shadow areas exist in the tongue root, the tongue middle and the tongue tip according to the tongue sub-images;
the super-pixel segmentation unit is used for carrying out super-pixel segmentation on the tongue sub-image if the tongue sub-image exists so as to obtain a super-pixel image;
the detection unit is used for detecting whether cracks exist in the tongue sub-images or not, if so, extracting a crack area, and forming the superpixels corresponding to the pixels positioned in the cracks into a first superpixel set;
the judging unit is used for judging whether the tongue image has the reflecting points or not according to the tongue sub-image, if so, extracting the reflecting area, and forming the super pixels corresponding to the reflecting points into a second super pixel set;
the classification unit is used for classifying the super-pixel images through a preset classifier to obtain a classification result; segmenting the tongue subimage according to the classification result, the crack area and the light reflection area; the preset classifier is obtained based on training of a support vector machine; the segmented tongue sub-image comprises a plurality of target areas, and the reflectivity of the same target area is the same;
the first calculation unit is used for determining a third superpixel set according to the segmented tongue subimages, calculating superpixel brightness of the third superpixel set, and substituting the superpixel brightness and the initial illumination direction into a preset illumination model to calculate a light source direction;
the second calculation unit is used for calculating the normal direction of the super-pixels of each target area according to the segmented tongue sub-images and the light source direction;
and the third determining unit is used for determining whether the tongue image is abnormal or not according to the normal direction.
9. A computer device comprising a memory and a processor, the memory having stored therein a computer program, characterized in that the processor implements the steps of the tongue image abnormality determination method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the tongue abnormality determination method according to any one of claims 1 to 7.
CN202111016133.4A 2021-08-31 2021-08-31 Tongue image anomaly determination method, tongue image anomaly determination device, computer equipment and storage medium Active CN113706515B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111016133.4A CN113706515B (en) 2021-08-31 2021-08-31 Tongue image anomaly determination method, tongue image anomaly determination device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111016133.4A CN113706515B (en) 2021-08-31 2021-08-31 Tongue image anomaly determination method, tongue image anomaly determination device, computer equipment and storage medium

Publications (2)

Publication Number Publication Date
CN113706515A true CN113706515A (en) 2021-11-26
CN113706515B CN113706515B (en) 2023-07-18

Family

ID=78658317

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111016133.4A Active CN113706515B (en) 2021-08-31 2021-08-31 Tongue image anomaly determination method, tongue image anomaly determination device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113706515B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114820603A (en) * 2022-06-27 2022-07-29 深圳中易健康科技有限公司 Intelligent health management method based on AI tongue diagnosis image processing and related device

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1908984A (en) * 2006-08-18 2007-02-07 清华大学 Coated tongue division and extracting method for colored digital photo of tongue
CN102389293A (en) * 2011-07-07 2012-03-28 中国中医科学院中医基础理论研究所 Tongue diagnosis spectral image processing system
CN110298829A (en) * 2019-06-19 2019-10-01 上海国民集团健康科技有限公司 A kind of lingual diagnosis method, apparatus, system, computer equipment and storage medium
CN110807775A (en) * 2018-08-06 2020-02-18 深圳市前海安测信息技术有限公司 Traditional Chinese medicine tongue image segmentation device and method based on artificial intelligence and storage medium
CN111329452A (en) * 2020-03-27 2020-06-26 上海祉云医疗科技有限公司 Face-tongue diagnosis instrument
CN111860538A (en) * 2020-07-20 2020-10-30 河海大学常州校区 Tongue color identification method and device based on image processing

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1908984A (en) * 2006-08-18 2007-02-07 清华大学 Coated tongue division and extracting method for colored digital photo of tongue
CN102389293A (en) * 2011-07-07 2012-03-28 中国中医科学院中医基础理论研究所 Tongue diagnosis spectral image processing system
CN110807775A (en) * 2018-08-06 2020-02-18 深圳市前海安测信息技术有限公司 Traditional Chinese medicine tongue image segmentation device and method based on artificial intelligence and storage medium
CN110298829A (en) * 2019-06-19 2019-10-01 上海国民集团健康科技有限公司 A kind of lingual diagnosis method, apparatus, system, computer equipment and storage medium
CN111329452A (en) * 2020-03-27 2020-06-26 上海祉云医疗科技有限公司 Face-tongue diagnosis instrument
CN111860538A (en) * 2020-07-20 2020-10-30 河海大学常州校区 Tongue color identification method and device based on image processing

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
林凌;解鑫;李刚;: "光谱法舌诊度量化中光源影响的研究", 生命科学仪器, no. 05, pages 1 - 5 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114820603A (en) * 2022-06-27 2022-07-29 深圳中易健康科技有限公司 Intelligent health management method based on AI tongue diagnosis image processing and related device
CN114820603B (en) * 2022-06-27 2022-09-16 深圳中易健康科技有限公司 Intelligent health management method based on AI tongue diagnosis image processing and related device

Also Published As

Publication number Publication date
CN113706515B (en) 2023-07-18

Similar Documents

Publication Publication Date Title
Vidya et al. Skin cancer detection using machine learning techniques
WO2021129323A1 (en) Ultrasound image lesion describing method and apparatus, computer device, and storage medium
JP6660313B2 (en) Detection of nuclear edges using image analysis
US10839510B2 (en) Methods and systems for human tissue analysis using shearlet transforms
EP2888718B1 (en) Methods and systems for automatic location of optic structures in an image of an eye, and for automatic retina cup-to-disc ratio computation
Yang-Mao et al. Edge enhancement nucleus and cytoplast contour detector of cervical smear images
EP2847738B1 (en) Method and apparatus for image scoring and analysis
EP4070230A1 (en) System and method for determining damage on crops
CN108319973A (en) Citrusfruit detection method on a kind of tree
David Retinal image classification system for diagnosis of diabetic retinopathy using morphological edge detection and feature extraction techniques
Ghazal et al. Automated framework for accurate segmentation of leaf images for plant health assessment
WO2022198898A1 (en) Picture classification method and apparatus, and device
CN110648336B (en) Method and device for dividing tongue texture and tongue coating
US20130222767A1 (en) Methods and systems for detecting peripapillary atrophy
Visalaxi et al. Lesion extraction of endometriotic images using open computer vision
Sibi Chakkaravarthy et al. Automatic leaf vein feature extraction for first degree veins
CN113706515B (en) Tongue image anomaly determination method, tongue image anomaly determination device, computer equipment and storage medium
Rahman et al. Skin lesions classification based on color plane-histogram-image quality analysis features extracted from digital images
CN108629780B (en) Tongue image segmentation method based on color decomposition and threshold technology
Khan et al. Segmentation of single and overlapping leaves by extracting appropriate contours
Siddique et al. A robust segmentation of blood vessels in retinal images
Tariq et al. Computer aided diagnostic system for grading of diabetic retinopathy
Ma et al. Deep attention network for melanoma detection improved by color constancy
CN114359279A (en) Image processing method, image processing device, computer equipment and storage medium
Sahak et al. Iris localization using colour segmentation and circular Hough transform

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant