CN116230203A

CN116230203A - Traditional Chinese medicine facial diagnosis method, system, equipment and medium based on machine vision

Info

Publication number: CN116230203A
Application number: CN202211685852.XA
Authority: CN
Inventors: 赵罕; 汪宇通; 余少杰; 刘俊杰; 钱鑫江
Original assignee: Hangzhou Pulse Health Technology Co ltd
Current assignee: Hangzhou Pulse Health Technology Co ltd
Priority date: 2022-12-27
Filing date: 2022-12-27
Publication date: 2023-06-06

Abstract

The invention discloses a traditional Chinese medicine facial diagnosis method, a system, equipment and a medium based on machine vision, which relate to the technical field of facial diagnosis, and the method comprises the following steps: acquiring a face picture to be detected of a patient; dividing the face picture to be detected into a plurality of viscera areas according to the relation between the face and twelve viscera, and extracting color features and texture features of the viscera areas to obtain key viscera features; inputting the viscera key characteristics into a pre-trained facial diagnosis model to obtain a health score, and diagnosing according to the health score. The invention can divide the human face into a plurality of viscera regions according to the relationship between the human face and twelve viscera, extract the corresponding viscera key features, and input the viscera key features into the facial diagnosis model, thereby rapidly obtaining accurate diagnosis results without being influenced by the subjective effect.

Description

Traditional Chinese medicine facial diagnosis method, system, equipment and medium based on machine vision

Technical Field

The invention relates to the technical field of traditional Chinese medicine facial diagnosis, in particular to a traditional Chinese medicine facial diagnosis method, system, equipment and medium based on machine vision.

Background

In the theory of traditional Chinese medicine, the human body is regarded as an organic whole, and viscera and body surface tissue organs of the human body are closely related in structure and function and have the interaction effect in pathology. Therefore, the traditional Chinese medicine detects and diagnoses patients in a mode of combining four diagnosis modes of looking, smelling, asking and cutting.

However, since the traditional four diagnoses are mostly performed by the doctor's observation and communication of the patient to acquire the body surface biological characteristics of the patient and record and analyze, the traditional four diagnoses are easily affected by subjective factors of individuals, so that the acquisition and analysis of the body surface biological characteristics are inaccurate.

Disclosure of Invention

Aiming at the defects in the prior art, the invention provides a traditional Chinese medicine facial diagnosis method, a system, equipment and a medium based on machine vision.

In a first aspect, a machine vision-based facial diagnosis method of traditional Chinese medicine includes:

acquiring a face picture to be detected of a patient;

dividing the face picture to be detected into a plurality of viscera areas according to the relation between the face and twelve viscera, and extracting color features and texture features of the viscera areas to obtain key viscera features;

inputting the viscera key characteristics into a pre-trained facial diagnosis model to obtain a health score, and diagnosing according to the health score.

Preferably, dividing the face image to be detected into a plurality of viscera regions according to the relationship between the face and the twelve viscera comprises:

identifying face key points in the face picture to be detected by utilizing a face recognition model, wherein the face key points comprise eyebrow key points, eye key points, nose key points, mouth key points and outline key points;

dividing the face picture to be detected into a plurality of viscera areas according to the face key points and preset dividing parameters, wherein the dividing parameters are used for indicating the distance between the edge of each viscera area and the face key points;

wherein the viscera region comprises a liver left region, a liver right region, a lung region, a brain region, a heart region, a breast left region, a breast right region, a kidney left region, a kidney right region, a gall left region, a gall right region, a large intestine left region, a large intestine right region, a small intestine left region, a small intestine right region, a stomach left region, a stomach right region, a liver lower region, an upper reproduction region and a lower reproduction region.

Preferably, extracting the color feature and the texture feature of the viscera region, and obtaining the viscera key feature includes:

converting each viscera region into HSV (hue, saturation and saturation) and YCbCr (YCbCr) color space from RGB (red, green and blue) color space, dividing the RGB color space into a plurality of color channels, and carrying out Fourier change on each color channel to obtain the color characteristics of each viscera region;

extracting texture features from each viscera region by using a gray level co-occurrence matrix;

and fusing the color features and the texture features to obtain initial viscera key features.

Preferably, extracting the color feature and the texture feature of the viscera region, and obtaining the viscera key feature further includes:

inquiring whether the historical viscera key characteristics of the patient exist in a historical database;

if yes, acquiring a preset number of historical viscera key features in a preset time, and combining the historical viscera key features and the initial viscera key features to acquire viscera key features;

if not, taking the initial viscera key characteristics as viscera key characteristics.

Preferably, the method of training the facial diagnostic model comprises:

acquiring a sample set, wherein the sample set comprises viscera key characteristics and health score labels corresponding to face pictures in a large number of medical libraries, and dividing the sample set into a training set and a testing set;

constructing an LSTM model and a random forest regression model;

and performing joint training on the LSTM model and the random forest regression model by using the training set, and performing joint verification on the LSTM model and the random forest regression model by using the testing machine to obtain a facial diagnosis model meeting the accuracy.

In a second aspect, a machine vision based facial diagnosis system for traditional Chinese medicine comprises:

the acquisition module is used for acquiring a face picture to be detected;

the feature extraction module is used for dividing the face picture to be detected into a plurality of viscera areas according to the relation between the face and twelve viscera, and extracting color features and texture features of the viscera areas to obtain viscera key features;

and the facial diagnosis module is used for inputting the viscera key characteristics into a pre-trained facial diagnosis model to obtain a health score, and diagnosing according to the health score.

In a third aspect, an electronic device, comprising:

a processor; and

a memory arranged to store computer executable instructions which, when executed, cause the processor to perform the method of any of the preceding claims.

In a fourth aspect, a computer readable storage medium stores one or more programs, which when executed by an electronic device comprising a plurality of application programs, cause the electronic device to perform the method of any of the preceding claims.

The beneficial effects of the invention are as follows: the embodiment of the invention provides a traditional Chinese medicine facial diagnosis method based on machine vision, which can divide a human face into a plurality of viscera areas according to the relationship between the human face and twelve viscera, extract corresponding viscera key features, input the viscera key features into a facial diagnosis model, and obtain an accurate diagnosis result without being influenced by subjective influences.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. Like elements or portions are generally identified by like reference numerals throughout the several figures. In the drawings, elements or portions thereof are not necessarily drawn to scale.

Fig. 1 is a schematic flow chart of a traditional Chinese medicine facial diagnosis method based on machine vision according to an embodiment of the invention;

fig. 2 is a schematic sub-flowchart of a traditional Chinese medicine facial diagnosis method based on machine vision according to an embodiment of the present invention;

fig. 3 is a schematic sub-flowchart of a traditional Chinese medicine facial diagnosis method based on machine vision according to an embodiment of the present invention;

fig. 4 is a schematic flow chart of a traditional Chinese medicine facial diagnosis method based on machine vision according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of a traditional Chinese medicine facial diagnosis system based on machine vision according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

Embodiments of the technical scheme of the present invention will be described in detail below with reference to the accompanying drawings. The following examples are only for more clearly illustrating the technical aspects of the present invention, and thus are merely examples, and are not intended to limit the scope of the present invention.

It is noted that unless otherwise indicated, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

Example 1

As shown in fig. 1, fig. 1 is a schematic flow chart of a traditional Chinese medicine facial diagnosis method based on machine vision, which is provided by the embodiment of the invention, and the method includes:

step one: acquiring a face picture to be detected of a patient;

the embodiment of the invention does not limit equipment for acquiring the face picture, and the face picture can be acquired through equipment such as a mobile phone, a camera and the like.

It should be noted that, the photo of the face to be detected needs to satisfy some basic requirements, for example, the size of the photo is larger than a preset size, the photo must contain the face and the face of the person is not covered, and the average brightness value of the photo is larger than the preset brightness, so as to reduce the influence of the external environment of the photo and ensure the accuracy of the subsequent facial diagnosis.

Step two: dividing the face picture to be detected into a plurality of viscera areas according to the relation between the face and twelve viscera, and extracting color features and texture features of the viscera areas to obtain key viscera features;

referring to fig. 2, in an embodiment of the present invention, dividing the face image to be detected into a plurality of viscera regions according to a relationship between a face and twelve viscera includes: identifying face key points in the face picture to be detected by utilizing a face recognition model, wherein the face key points comprise eyebrow key points, eye key points, nose key points, mouth key points and outline key points; dividing the face picture to be detected into a plurality of viscera areas according to the face key points and preset dividing parameters, wherein the dividing parameters are used for indicating the distance between the edge of each viscera area and the face key points.

Specifically, the viscera region comprises a liver left region, a liver right region, a lung region, a brain region, a heart region, a breast left region, a breast right region, a kidney left region, a kidney right region, a gall left region, a gall right region, a large intestine left region, a large intestine right region, a small intestine left region, a small intestine right region, a stomach left region, a stomach right region, a liver lower region, a reproduction upper region and a reproduction lower region.

The face reflects physiological information of all parts of the whole body, so that the whole body of the face is formed into an integral reduction, the structure of the face belongs to different viscera and is the basis of facial inspection.

It should be noted that, in other embodiments, the wind parameter provided by the embodiment of the present invention is a distance, and the illustrated segmentation parameter may be a distance function, for example, the length of a human eye is determined by an inner corner key point and an outer corner key point of the human eye, the length of a small intestine region is determined according to the ratio of the length of the human eye to the small intestine region, the width of the human eye is determined according to the highest key zone you and the lowest key point of the human eye, and the width of the small intestine region is determined according to the ratio of the width of the human eye to the small intestine region, so as to divide the small intestine region with greater precision.

Referring to fig. 3, in an embodiment of the present invention, extracting color features and texture features of the viscera region includes: converting each viscera region into HSV (hue, saturation and saturation) and YCbCr (YCbCr) color space from RGB (red, green and blue) color space, dividing the RGB color space into a plurality of color channels, and carrying out Fourier change on each color channel to obtain the color characteristics of each viscera region; extracting texture features from each viscera region by using a gray level co-occurrence matrix; and fusing the color features and the texture features to obtain initial viscera key features.

Facial color is the most important information feature in inspection of traditional Chinese medicine. The facial color features are various, and the information reflected by different color spaces is also different.

The image corresponding to the color and viscera area contains a lot of information useful for health diagnosis. Although the facial color features have strong stability, the facial color features have weak differentiation on the skin, so that other features are required to be fused to enable the classifier to learn, and the facial diagnosis accuracy can be effectively improved. Texture is the extraction and analysis of the spatial distribution pattern of the grey scale of an image, describing the distribution law from pixel to pixel of the image, its presence being unaffected by the target colour or brightness variation, a characteristic of the surface sharing of an object. The gray level co-occurrence matrix is a classical second order statistical method for texture feature extraction that represents the relative frequency of gray levels present at a particular distance-to gray level present at a particular angle 0. In the embodiment of the invention, a gray formula matrix is adopted to extract the statistical characteristics of viscera regions in 4 directions of energy, contrast, entropy and inverse difference moment.

Referring to fig. 3, in an embodiment of the present invention, extracting the color feature and the texture feature of the viscera region, to obtain the viscera key feature further includes: inquiring whether the historical viscera key characteristics of the patient exist in a historical database; if yes, acquiring a preset number of historical viscera key features in a preset time, and combining the historical viscera key features and the initial viscera key features to acquire viscera key features; if not, taking the initial viscera key characteristics as viscera key characteristics.

The invention not only considers the facial viscera state of the patient during the secondary diagnosis, but also considers the state of the facial viscera region of the patient during the previous diagnosis, thereby effectively reducing the influence of the skin color of the patient on the diagnosis result.

Step three: inputting the viscera key characteristics into a pre-trained facial diagnosis model to obtain a health score, and diagnosing according to the health score.

Referring to fig. 4, in an embodiment of the present invention, a method of training the facial diagnostic model includes: acquiring a sample set, wherein the sample set comprises viscera key characteristics and health score labels corresponding to face pictures in a large number of medical libraries, and dividing the sample set into a training set and a testing set; constructing an LSTM model and a random forest regression model; and performing joint training on the LSTM model and the random forest regression model by using the training set, and performing joint verification on the LSTM model and the random forest regression model by using the testing machine to obtain a facial diagnosis model meeting the accuracy.

Various diagnostic logics can be set according to specific situations, for example, when the health score is lower than a first preset health value, the diagnostic patient is in a first-level unhealthy state, when the health score is lower than a second preset health value, the diagnostic patient is in a second-level unhealthy state, when the health score is lower than a third preset health value, the diagnostic patient is in a third-level unhealthy state, and when the health score is higher than the third preset health value, the diagnostic patient is in a healthy state.

In summary, the embodiment of the invention provides a traditional Chinese medicine facial diagnosis method based on machine vision, which can divide a human face into a plurality of viscera areas according to the relationship between the human face and twelve viscera, extract corresponding viscera key features, input the viscera key features into a facial diagnosis model, and obtain an accurate diagnosis result without being influenced by subjective influence.

Example two

As shown in fig. 5, fig. 5 is a schematic structural diagram of a traditional Chinese medicine facial diagnosis system based on machine vision according to an embodiment of the present invention, including:

the acquisition module is used for acquiring a face picture to be detected;

It should be understood that, for the same inventive concept, the more specific working principle of each module in the embodiment of the present invention may refer to the above embodiment, and details are not repeated in the embodiment of the present invention.

Example III

Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present specification. Referring to fig. 6, at the hardware level, the electronic device includes a processor, and optionally an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory (non-volatile Memory), such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.

The processor, network interface, and memory may be interconnected by an internal bus, which may be an ISA (Industry Standard Architecture ) bus, a PCI (PeripheralComponent Interconnect, peripheral component interconnect standard) bus, or EISA (Extended Industry StandardArchitecture ) bus, among others. The buses may be classified as address buses, data buses, control buses, etc. For ease of illustration, only one bi-directional arrow is shown in FIG. 3, but not only one bus or type of bus.

And the memory is used for storing programs. In particular, the program may include program code including computer-operating instructions. The memory may include memory and non-volatile storage and provide instructions and data to the processor.

The processor reads the corresponding computer program from the nonvolatile memory into the memory and then runs, and forms a shared resource access control device on a logic level. The processor is used for executing the programs stored in the memory and is specifically used for executing the following operations:

acquiring a face picture to be detected of a patient;

The above-mentioned auxiliary diagnosis method for facial diagnosis of traditional Chinese medicine based on face feature detection disclosed in the embodiment shown in fig. 1 to 4 of the present specification can be applied to or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or by instructions in the form of software. The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application SpecificIntegrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The methods, steps and logic blocks disclosed in the embodiments of the present specification may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present specification may be embodied directly in hardware, in a decoded processor, or in a combination of hardware and software modules in a decoded processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method.

Of course, in addition to the software implementation, the electronic device of the embodiments of the present disclosure does not exclude other implementations, such as a logic device or a combination of software and hardware, that is, the execution subject of the following processing flow is not limited to each logic unit, but may also be hardware or a logic device.

Example IV

The present description also proposes a computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a portable electronic device comprising a plurality of application programs, enable the portable electronic device to perform the method of the embodiments shown in fig. 1-4, and in particular to perform the method of: acquiring a face picture to be detected of a patient; dividing the face picture to be detected into a plurality of viscera areas according to the relation between the face and twelve viscera, and extracting color features and texture features of the viscera areas to obtain key viscera features; inputting the viscera key characteristics into a pre-trained facial diagnosis model to obtain a health score, and diagnosing according to the health score.

In summary, the foregoing description is only a preferred embodiment of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification should be included in the protection scope of the present specification.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.

Claims

1. The traditional Chinese medicine facial diagnosis method based on machine vision is characterized by comprising the following steps of:

acquiring a face picture to be detected of a patient;

2. The machine vision-based traditional Chinese medicine facial diagnosis method according to claim 1, wherein dividing the face image to be detected into a plurality of viscera regions according to the relationship between the face and twelve viscera comprises:

3. The machine vision based facial diagnosis method according to claim 1, wherein extracting color features and texture features of the viscera region to obtain viscera key features comprises:

4. The machine vision-based facial diagnosis method according to claim 3, wherein,

if yes, acquiring a preset number of historical viscera key features in a preset time, and combining the historical viscera key features and the initial viscera key features to acquire viscera key features; if not, taking the initial viscera key characteristics as viscera key characteristics.

5. The machine vision based facial diagnosis method according to claim 1, wherein the method for training the facial diagnosis model comprises:

constructing an LSTM model and a random forest regression model;

6. A machine vision based facial diagnosis system for traditional Chinese medicine, comprising:

the acquisition module is used for acquiring a face picture to be detected;

7. An electronic device, comprising:

a processor; and

a memory arranged to store computer executable instructions which, when executed, cause the processor to perform the method of any of claims 1-5.

8. A readable storage medium, characterized in that the computer readable storage medium stores one or more programs, which when executed by an electronic device comprising a plurality of application programs, cause the electronic device to perform the method of any of claims 1-5.