WO2023029500A1 - Health scheme recommendation method and apparatus based on deep learning, and device and medium - Google Patents

Abstract

The present application relates to the technical field of artificial intelligence. Provided are a health scheme recommendation method and apparatus based on deep learning, and a device and a medium. The method comprises: acquiring an initial similarity between an initial tongue image and a standard tongue image; extracting an initial tongue coating image and an initial tongue nature image from the initial tongue image, and inputting same into a preset tongue coating checking model and a preset tongue nature checking model, so as to obtain a tongue coating state and a tongue nature state; determining tongue association information, and thereby determining a plurality of health schemes to be selected; and determining a mental state of a user, so as to determine a recommended health scheme from among the health schemes to be selected. Further provided in the present application are a health scheme recommendation apparatus based on deep learning, and a device and a medium. A tongue nature state and a tongue coating state are obtained by means of a preset tongue coating checking model and a preset tongue nature checking model, and a suitable health scheme is recommended by combining two factors, i.e. a physical health condition and a mental state of a user, such that consideration dimensions are more comprehensive, thereby improving the user satisfaction degree. The present application is simple and feasible.

Description

Health plan recommendation method, device, equipment and medium based on deep learning

priority statement

This application claims the priority of the Chinese patent application submitted to the China Patent Office on August 30, 2021, with the application number 202111005735.X, and the title of the invention is "Health program recommendation method, device, equipment and medium based on deep learning". The entire contents are incorporated by reference in this application.

technical field

This application relates to the field of artificial intelligence technology, and provides a method, device, equipment and medium for recommending a health plan based on deep learning.

Background technique

Tongue coating, tongue color, shape and other information can represent a person’s current physical state to a certain extent. Traditional methods often require professionals such as doctors who have received professional training to obtain a person’s physical state through on-site diagnosis and other methods, and then Make some health management suggestions or prescriptions.

Due to the limited number of professionals, for some "small problems" such as getting angry, people often cannot get help from professionals for various reasons. Programs are in increasing demand. At the same time, the inventor realized that people's mood and mental state may also affect the physical state. Traditional health solutions often only focus on the user's physical basic detection data itself, such as blood pressure, heart rate, weight, etc., ignoring the user's mental state. The health management suggestions provided are not comprehensive enough, poorly targeted, and user satisfaction is poor.

Contents of the invention

This application provides a health plan recommendation method, device, equipment and medium based on deep learning. Its main purpose is to obtain the tongue quality and tongue coating status through the tongue image through the preset tongue coating detection model and the preset tongue quality detection model, so as to Determine the health plan to be selected, determine the recommended health plan based on the user's mental state, and recommend an appropriate health plan based on the user's physical health and mental state. The consideration dimension is more comprehensive and the user satisfaction is improved. It is simple and feasible.

In order to achieve the above purpose, the present application provides a method for recommending a health plan based on deep learning, the method includes: acquiring the user's initial tongue image, and comparing the initial tongue image with a standard tongue image to obtain the initial similarity;

If the initial similarity is lower than the preset initial similarity threshold, extracting an initial tongue coating image and an initial tongue quality image respectively from the initial tongue image;

Inputting the initial tongue coating image into a preset tongue coating detection model to obtain a tongue coating state, and inputting the initial tongue texture image into a preset tongue texture detection model to obtain a tongue texture state;

Determine tongue-related information according to the state of the tongue quality and the state of the tongue coating, and determine several health options to be selected, the health options to be selected include suggested foods and eating methods;

Obtain information related to the user's mental state and several user behavior images within a preset time period, and determine the user's mental state;

A recommended health plan is determined among the candidate health plans according to the mental state of the user.

In addition, in order to achieve the above purpose, the present application also provides a device for recommending a health plan based on deep learning, which includes:

An image acquisition module, configured to acquire an initial tongue image of the user, and compare the initial tongue image with a standard tongue image to obtain an initial similarity;

An image extraction module, configured to extract an initial tongue coating image and an initial tongue quality image from the initial tongue image if the initial similarity is lower than a preset initial similarity threshold;

A detection module, configured to input the initial tongue coating image into a preset tongue coating detection model to obtain a tongue coating state, and input the initial tongue texture image to a preset tongue texture detection model to obtain a tongue texture state;

Alternative options determination module, used to determine tongue-associated information according to the state of tongue quality and tongue coating state, and determine several health options to be selected, the health options to be selected include suggested foods and eating methods;

The mental state determination module is used to obtain the related information of the user's mental state and several user behavior images within a preset time period, and determine the user's mental state;

A recommendation module is configured to determine a recommended health plan among the candidate health plans according to the mental state of the user.

In addition, in order to achieve the above object, the present application also provides a computer device, including a memory, a processor, and a computer program stored on the memory and operable on the processor. When the processor executes the computer program, the above-mentioned The steps of the method described in one embodiment.

In addition, to achieve the above purpose, the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described in any one of the above embodiments are implemented.

The health plan recommendation method, device, equipment and medium based on deep learning proposed by this application, the method acquires the user's initial tongue image, and compares the initial tongue image with the standard tongue image to obtain the initial similarity, If the initial similarity is lower than the preset initial similarity threshold, the initial tongue coating image and the initial tongue texture image are respectively extracted from the initial tongue image, and the initial tongue coating image is input into the preset tongue coating detection model to obtain the tongue coating state, and the initial tongue texture The image is input to the preset tongue quality detection model to obtain the tongue quality state, determine tongue-related information according to the tongue quality state and tongue coating state, and determine several health programs to be selected, and obtain user mental state-related information and a number of Create a user behavior image, and determine the user's mental state, and determine the recommended health plan among the candidate health plans according to the user's mental state. Get the tongue quality and tongue coating status through the tongue image through the preset tongue coating detection model and the preset tongue quality detection model to determine the health plan to be selected, and determine the recommended health plan based on the user's mental state, combined with the user's physical health and mental state Recommended health solutions based on two factors, more comprehensive considerations, improved user satisfaction, simple, feasible, and highly targeted.

Description of drawings

FIG. 1 is a schematic flow diagram of a method for recommending a health plan based on deep learning provided in an embodiment of the present application;

FIG. 2 is another schematic flow diagram of a method for recommending a health plan based on deep learning provided in an embodiment of the present application;

FIG. 3 is another schematic flow diagram of a method for recommending a health plan based on deep learning provided in an embodiment of the present application;

FIG. 4 is a schematic structural diagram of a device for recommending a health plan based on deep learning provided in an embodiment of the present application;

Fig. 5 is a schematic structural diagram of a tongue coating extraction module provided in an embodiment of the present application;

Fig. 6 is a schematic structural diagram of a computer device provided in an embodiment of the invention.

The realization, functional features and advantages of the present application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Detailed ways

It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

In one embodiment, a method for recommending a health plan based on deep learning is provided, as shown in FIG. 1 , the method includes the following steps:

Step S101: Obtain the user's initial tongue image, and compare the initial tongue image with the standard tongue image to obtain the initial similarity.

In some embodiments, the initial tongue image can be collected by monitoring devices, mobile phones, wearable devices or professional health monitoring devices.

Optionally, before acquiring the initial tongue image, since the image acquisition effect of the acquisition device may be affected by the initial light, angle, etc., the acquisition device can first collect the image of the standard item, and then collect the initial tongue coating image after error calibration . The marked item may be a red color item selected by default and calibrated in the system. In this way, after the error of the standard item is calibrated, and then the initial tongue image is collected, the initial tongue image can be adjusted according to the above-mentioned calibrated error, so as to eliminate the error caused by factors such as light and angle as much as possible.

Optionally, the standard tongue image may be an image taken by the user in a healthy state. For users with chronic diseases, because the state of their chronic diseases will last for a long time, and they need to be supplemented by professional treatment methods, such as high blood pressure, diabetes, etc., for such users, the standard tongue image can also be the user's Images taken in a stable state of chronic disease.

An optional way to compare the initial tongue image to the standard tongue image includes:

The initial tongue image and the standard tongue image are respectively gray-scaled, and normalized to a preset pixel size to obtain a gray-scale initial tongue image and a gray-scale standard tongue image;

Determine the average gray value of each pixel in the gray-scale initial tongue image and the gray-scale standard tongue image respectively, and obtain the first average gray value of the gray-scale initial tongue image and the first average gray value of the gray-scale standard tongue image. Two average gray value;

Compare the grayscale value of each pixel in the grayscale initial tongue image with the first average grayscale value, if the grayscale value of a pixel is greater than the first average grayscale value, the converted value is recorded as N, otherwise converted The value is denoted as M, and the initial code is formed according to the arrangement order of the pixels in the grayscale initial tongue image;

Compare the grayscale value of each pixel in the grayscale standard tongue image with the second average grayscale value, if the grayscale value of a pixel is greater than the second average grayscale value, the converted value is recorded as N, otherwise converted The value is denoted as M, and a standard code is formed according to the arrangement order of the pixels in the grayscale standard tongue image;

Compare the initial code with the standard code to get the initial similarity.

Among them, before determining the first average gray value and the second gray value, the gray value of each pixel in the gray initial tongue image and the gray standard tongue image can be reduced by a certain factor in proportion to reduce the subsequent Calculations. If the grayscale initial tongue image and the grayscale standard tongue image are 8*8 size images, the initial code and the standard code can be 64-bit binary codes. N and M are different numbers, such as N=1, M=0 and so on. The initial encoding and standard encoding are arranged in the same order.

Comparing the initial code with the standard code, the method of obtaining the initial similarity includes comparing whether the two conversion values in the same arrangement position in the initial code and the standard code are the same, obtaining the number of the same conversion values, and using the same conversion value The ratio of the number of values to the number of all converted values in the initial encoding (or standard encoding) is taken as the initial similarity. Taking the default pixel size of 8*8 as an example, the two images have 8*8=64 pixels respectively, and the obtained initial code and standard code respectively have 64 conversion values, and after comparison, there are 60 positions The conversion values of are the same, then the initial similarity=60/64*100%=93.75%.

An optional way to compare the initial tongue image to the standard tongue image includes:

Carry out feature vector extraction to initial tongue image and standard tongue image respectively, obtain initial tongue image feature vector and standard tongue image feature vector;

Determine the cosine similarity between the initial tongue image feature vector and the standard tongue image feature vector as the initial similarity.

Wherein, the manner of comparing the initial tongue image with the standard tongue image may also adopt other manners known to those skilled in the art, which is not limited here.

By first comparing the initial tongue image with the standard tongue image to obtain the initial similarity, it is possible to have a certain understanding of whether the user's initial tongue condition is consistent with the historical tongue condition, and whether the user needs this embodiment. Offered health program recommendations for an initial screening. If the initial tongue image is similar to the standard tongue image (the initial similarity is higher than the preset initial similarity threshold), it means that the user's health status is relatively stable, and it is only necessary to maintain the original living and eating habits. Otherwise, if the original tongue image is not similar to the standard tongue image (the initial similarity is lower than the preset initial similarity threshold), it means that the user's health status has changed, and the health plan needs to be re-recommended.

In some embodiments, when collecting the initial tongue image, the user may have eaten foods that are easy to stain the tongue, such as grapes, dragon fruit, mulberries, and beverages containing pigments. If the judgment is inaccurate, preliminary screening can be performed on the initial tongue image to ensure that the initial tongue image is a qualified image. Obtain the user's initial tongue image, and compare the initial tongue image with the standard tongue image. Before obtaining the similarity, it also includes screening the initial tongue coating image. See Figure 2. The method also includes:

S001: Obtain the original tongue image of the user, and identify the original tongue coating image in the original tongue image;

S002: Acquire the hue of the original tongue coating image;

S003: If the hue belongs to the preset hue, the original tongue image is qualified, and the original tongue image is used as the initial tongue image;

S004: If the hue does not belong to the preset hue, prompt the user to clean the tongue.

Optionally, the original tongue image is the image directly collected by the collection device, or the image obtained by the collection device after the above error adjustment.

The acquisition method of the original tongue coating image can be extracted through a pre-trained target extraction model, or it can be similar to the extraction method of the initial tongue coating image described later, or it can be realized by means known to those skilled in the art.

The acquisition method of the hue of the original tongue coating image can be implemented in a manner known to those skilled in the art, including but not limited to visual recognition, machine recognition, and the like. The preset hue can be set by those skilled in the art as required, and the preset hue can be fixed or adjusted according to different seasons. For example, different colors can be set according to the ripening period of fruits rich in pigment Default hue.

If it is found that the color of the tongue coating is black, dark purple and other abnormal colors, the user can be reminded to clean the tongue in time to improve the accuracy of the program. You can wait for the user to finish cleaning the tongue before collecting a new initial tongue image.

Step S102: If the initial similarity is lower than the preset initial similarity threshold, extract an initial tongue coating image and an initial tongue quality image respectively from the initial tongue image.

According to the initial similarity, the initial tongue image can be screened to a certain extent. Only the initial tongue image whose initial similarity is lower than the preset initial similarity threshold will extract the tongue coating and tongue quality, which can reduce unnecessary Calculate and save computing power.

The preset initial similarity threshold may be a threshold set by those skilled in the art according to needs.

In some embodiments, the extraction of the initial tongue coating image and the initial tongue quality image from the initial tongue image can be realized through a pre-trained preset image extraction model, and the training of the model can be realized in a manner known to those skilled in the art . Among them, the preset image extraction model can manually mark several sample images in advance to obtain a number of tongue coating images and tongue texture images, as a training set, and train the preset neural network model to obtain the preset image extraction model. After the initial tongue image is input to the preset image extraction model, the initial tongue coating image and initial setting image are output.

In some embodiments, referring to FIG. 3 , the manner of extracting the initial tongue coating image from the initial tongue image includes:

S301: Divide the initial tongue image into several initial image blocks;

S302: Obtain the RGB values of the pixels in each initial image block respectively, and determine the average RGB value of the pixels in each initial image block;

S303: Determine the suspected tongue coating image block from each initial image block according to the pixel average RGB value and the preset tongue coating RGB value range;

S304: Obtain the image block position information of the suspected tongue coating image block, and determine the credible image block, the credible image block includes the suspected tongue coating image block whose distance from at least one other suspected tongue coating image block is less than a preset distance threshold;

S305: Concatenate the credible image blocks to obtain an initial tongue coating image.

The initial tongue image can be divided into several rectangular areas, and the size of each rectangular area can be the same or different. Or segment the initial tongue image into circular regions, rectangular regions, irregular regions, etc. The division method of the initial image block may be an image of a certain size square or rectangular grid, or it may be divided according to other methods set by those skilled in the art.

The RGB (R (red), G (green), B (blue)) values of each pixel in the initial image block can be acquired in a manner known to those skilled in the art.

Optionally, the average RGB value of pixels in the initial image block=the sum of the RGB values of each pixel in the initial image block/the number of pixels included in the initial image block.

Because there is a certain color difference between the tongue coating and the tongue body under normal circumstances, and the colors of the tongue coating itself are relatively close, and the colors of the tongue body are also relatively close, so it can be found from each initial image block by the average RGB value of the pixel. Image patches of suspected tongue coating that may include tongue coating.

In some embodiments, the preset tongue coating RGB value range can be set by those skilled in the art according to needs, because the tongue quality that usually may cause health problems is generally crimson tongue, dark red tongue, red tongue, light red tongue, pale tongue Purple and white tongue, and the color of the tongue coating that may cause health problems is generally purple-gray fur, yellow-white fur, yellow fur, and white fur. It can be seen that generally speaking, the color of the tongue coating is quite different from the color of the tongue texture, so the preset tongue coating RGB value range can be set according to the common colors of the tongue coating.

In some embodiments, the preset tongue coating RGB value range may also be determined by the average RGB value of pixels and its distribution. For example, the maximum and minimum values of the pixel average RGB values of the initial image blocks located in the middle of the initial tongue image (except for several initial image blocks distributed on the edge of the initial tongue image) are used as the preset values. Set the tongue coating RGB value range, etc.

Since the tongue coating is often attached to the tongue, and the position of the tongue coating is located inside the edge of the tongue, if a certain initial image block is discretely distributed outside other initial image blocks, a misjudgment may occur, and the initial image block is excluded. The remaining initial image blocks are used as trusted image blocks. Discrete cases can be judged according to the distance between the initial image blocks.

Optionally, the image block position information may be the position of one or more image points in the suspected tongue coating image block by setting an image coordinate system for the initial tongue image in advance with a certain point of the initial tongue image as the origin. The coordinates of are used as the image block position information. For example, the suspected tongue coating image block is a rectangle, and the coordinates of the image points corresponding to the four vertices of the rectangle are used as the position information of the image block.

Optionally, the image block position information can also use a single point in the display space where the initial tongue image is currently displayed as the origin to create a display coordinate system, and each image point in the initial tongue image corresponds to There is a coordinate, and the coordinates of the location of one or more image points in the suspected tongue coating image block are used as the image block position information.

Optionally, if the initial image block is a rectangular block, the position information of at least one image block of the suspected tongue coating image block is the position information of four vertices of the rectangular block, so that at least one vertex between the adjacent suspected tongue coating image blocks is coincident of. If the image block position information of a suspected tongue coating image block is different from the image block position information of all other image block position information image blocks, that is, it can be characterized that the suspected tongue coating image block is a discrete single image block, then the Image blocks are removed to ensure the accuracy of the initial tongue coating image.

It should be known that some of the initial image blocks include a large number or a small number of tongue pixels, and those skilled in the art can reduce the influence of initial image blocks that include a large number of tongue pixels by controlling the preset tongue coating RGB value range, or The above effects are avoided by controlling the number of pixels included in the initial image block.

The splicing of credible image blocks can be realized on the basis of retaining their original positions in the original tongue image.

Step S103: Input the initial tongue coating image into the preset tongue coating detection model to obtain the tongue coating state, and input the initial tongue texture image into the preset tongue texture detection model to obtain the tongue texture state.

Optionally, the preset tongue coating detection model and the preset tongue quality detection model can be obtained by those skilled in the art by training the basic model based on a number of marked tongue coating images and tongue coating status, tongue quality images and tongue quality status as training sets.

Optionally, the state of the tongue quality includes but is not limited to crimson tongue, dark red tongue, red tongue, light red tongue, lavender tongue, white tongue, etc. The state of the tongue coating includes, but is not limited to, purple-gray fur, yellow-white fur, yellow fur, and white fur.

Step S104: Determine the tongue related information according to the state of the tongue quality and the state of the tongue coating, and determine several health options to be selected.

Optionally, the health options to be selected include suggested foods and eating methods.

Optionally, the tongue-related information includes tongue-coat status-related information and tongue-quality-related information. Taking the tongue-coat status-related information as an example, if the tongue coating status is thick and white, it is necessary to know the specific physical conditions of the user, such as daily eating conditions, whether Fatigue, fatigue, sticky or shapeless stools, whether you have symptoms of nausea or vomiting, whether your living environment is damp or has been damp recently, whether you have symptoms of loss of appetite, fatigue, fatigue, and fever. The above information is the state of tongue coating Related Information.

Optionally, a preset related information question bank can be set in advance, and the preset related information question bank includes several preset related information questions, and each preset related information question is provided with a tongue quality label and a tongue coating label, and the tongue quality label Corresponds to the state of the tongue quality, and the tongue coating label corresponds to the state of the tongue coating. According to the state of the tongue quality and the state of the tongue coating, several corresponding preset related information questions can be found in the preset related information question database, and the tongue related information can be obtained according to the user's answer by asking the user the preset related information questions. For example, the tongue-related information can also determine the preset question list according to the tongue quality and tongue coating state, ask questions in the preset question list through text, voice, etc. and obtain user answers, and perform keyword extraction based on user answers , to get the answers to the above questions as tongue related information.

In some embodiments, the health plan to be selected is determined according to tongue association information, tongue quality, tongue coating state, and health-related information, wherein the health-related information includes gender, age, underlying disease, living environment climate, height, weight, body temperature , sleep pattern, blood pressure, blood sugar, heart rate, sedentary state, etc. Optionally, BMI (body mass index) can be determined according to height and weight, specifically, BMI=weight (kg)÷height^2(m). The health-related information may be collected through relevant health information collection equipment. For example, health-related information is collected through body fat scales, bracelets with health monitoring functions, and other equipment.

Optionally, each category of health-related information can be assigned a corresponding information impact factor. For numerical health-related information such as weight and height, the product of its value and the information impact factor can be directly taken as the intermediate value. For non-numeric type For health-related news, such as sleep type and sedentary state, first convert them into numerical representations. For example, sedentary state includes the daily average sitting time, divide different time lengths into corresponding levels, and then take the relationship between the level and the information impact factor The product is used as an intermediate value, and the way of numerical representation can also be realized by means known to those skilled in the art. The median value corresponding to each health-related information is summed and then averaged as the identification value of the health-related information, so as to realize the normalized representation of the health-related information, so as to subsequently determine the health plan to be selected.

According to at least one of the tongue-related information, tongue state, tongue coating state, and health-related information, it is compared with a preset health plan library including several preset health plans to obtain a suitable candidate health plan. It is assumed that the health plan is preset with at least one of tongue-related information tags, tongue quality status tags, tongue coating status tags, and health-related information tags. By comparing the tongue-related information label, tongue quality status label, tongue coating status label, and health-related information label with tongue-related information, tongue quality status, tongue coating status, health-related information, etc., determine the appropriate preset health plan as the Choose a health plan. Optionally, a preset health program may correspond to one or more tongue-related information labels and health-related information labels. Preset health programs can be constructed by professionals such as doctors and nutritionists.

Among them, the recommended food can be ordinary food, or food with both medicine and food. Help users solve or prevent some possible health problems through food supplements.

There may be many ways to eat the same food, such as eating raw, oiling, watering, carving, baking, grilling, making soup, etc. The eating method can be presented in the form of recipes or tips. The eating method can also be presented in the form of a video connection. If the user chooses or automatically recommends the health plan to the user, the corresponding eating method will be matched and the production process video will be played.

Optionally, the method also includes:

Determine the eating method and production process video according to the recommended health plan, and play the production process video.

Optionally, according to the user’s tongue quality, tongue coating and tongue-related information, it is possible to determine the user’s possible health problems, such as getting angry, etc., and recommend corresponding health solutions including suitable foods, so that the user can alleviate the problem through dietary therapy. Or address health issues.

Optionally, according to the user's health-related information, it can be known whether the user has problems such as obesity, high blood pressure, high blood fat, and high uric acid, and can be combined with health-related information, tongue coating status, tongue quality status, and tongue-related information to comprehensively determine the candidate health Programs to avoid food that may pose a threat to the user's health in the health program to be selected. For example, if a patient has high uric acid and suffers from ventilation, avoid determining a health plan that includes foods with high purine content such as shiitake mushrooms as a candidate health plan.

Step S105: Obtain information related to the user's mental state and several user behavior images within a preset time period, and determine the user's mental state.

Optionally, user behavior images can be collected by camera monitoring equipment. On the premise that the user knows and agrees to the image collection in advance, the specific time point and collection equipment of the collection behavior are preferably not known to the user, so as to improve the user experience. The credibility of the collected user behavior images.

The acquisition of user behavior images can also be realized by guiding the user to collect the mental state of the face. For example, first record the user's face images in three states (happy/sad/normal) as a training set to train the initial model of the mental state to obtain the user's mental state model. Follow-up guides the user to turn on the mobile phone camera for real-time video capture or collects the user's face image through other image capture devices, and inputs it into the user's mental state model of the training account to obtain the user's mental state.

Optionally, the associated information of the user's mental state includes but is not limited to at least one of the user's working (study) duration, whether there is any behavior such as dazed, crying, sighing, etc., the number and duration of non-work going out, and other behaviors that can reflect or reflect the user's mental state .

Optionally, the first user mood score can be obtained by analyzing the user behavior image, and then the second user mood score can be obtained according to the user's mental state related information, and the user mood total score can be obtained according to the first user mood score and the second user mood score To characterize the mental state of the user. For example, different values are assigned to happy/sad/normal user mental states as the first user mood score, and appropriate values are assigned to the associated information of different user mental states as associated values and their associated influence shadows. The factor determines the mood score of the second user (by taking the weighted average of the associated values of the associated information of each user's mental state).

The user behavior image may be a human body image, and the user's mental state may also be determined based on the human body image including the user's physical state. For example, the user's shrugging, hunchback, bowing, and raising the head can all reflect the user's mental state.

In some embodiments, the user behavior image includes the user's facial image, the user's mental state-related information includes information content and information content influencing factors, and the method for determining the user's mental state includes:

Obtain the key point position information of the target facial key point in the user's facial image, the target facial key point includes the key point of the nose tip, the key point of the corner of the mouth and the key point of the corner of the eye;

Obtain the first distance between the eye corner key point and the nose tip key point in each user's facial image, and the second distance between the mouth corner key point and the nose tip key point, and determine the first average value and first median of several first distances respectively , respectively determine the second average value and the second median of several second distances;

determining a first mental state score based on the first median, the first mean, the second median, and the second mean;

determining a second mental state score based on the information content and the information content image factor;

The mental state of the user is determined according to the first mental state score and the second mental state score.

Optionally, the information content includes but not limited to at least one of the number of times of laughter, times of dazed, times of crying, times of sighing, working hours, and non-working time out.

Wherein, target face key point can be determined by those skilled in the art, and above-mentioned is just an example, and other key points (such as the key point at the eyebrow etc.) adjacent to the currently selected target face key point can also be used as target Facial keypoints, whose user's mental state is determined in a similar manner to the above example.

Optionally, the manner of identifying the key points of the target face and the manner of acquiring the location information of the key points may be implemented in a manner known to those skilled in the art.

The first distance and the second distance can be based on the "virtual distance" obtained based on the image coordinate system, and then converted to the actual distance in the real world coordinate system according to the ratio conversion relationship between the image and the real world, or the key point position information is Relative to the coordinate data obtained in the coordinate system of the real world, the obtained first distance and the second distance are actual distances. The first average value and the second average value are the average number of several first distances and the average number of several second distances of the user captured in different facial images of the user. The first median and the second median are respectively obtained by sorting the first distance and the second distance in order of magnitude. Optionally, if the number of the first distance and/or the second distance is an even number, take the average of the two values in the middle as the first median and/or the second median.

Optionally, determining the first mental state score according to the first median, the first average, the second median and the second average includes:

If the first median is less than the first average, and the second median is less than the second average, then obtain the preset first facial score as the first mental state score, otherwise obtain the preset second facial score as the second A mental state score. The preset first facial score and the preset second facial score can be preset by those skilled in the art.

Optionally, when a person is happy, the corners of the mouth tend to rise, shortening the distance between the corners of the mouth and the tip of the nose. If the second median is smaller than the second average value, it means that the user may have taken more facial images when the user is happy. When a person is happy, the corners of the eyes will be close to the tip of the nose due to the stretching of the muscles, and the distance between the corners of the eyes and the tip of the nose will be shortened. If the first median is smaller than the first average value, it further indicates that the user’s face may be taken when the user is happy. There are many images, and at this time, the preset first face score is acquired as the first mental state score. In other cases, it means that when the user is normal or unhappy, there are more facial images of the user, and the preset second facial score is selected as the first mental state score.

Optionally, the second mental state score is determined by taking products of different information contents and corresponding information content impact factors and summing them up.

Take the sum of the first mental state score and the second mental state score as the total mental state score. The mental state of the user is determined through the mapping relationship between the total score of the mental state and the mental state corresponding to each preset score gradient. Alternatively, the weighted average of the first mental state score and the second mental state score is taken as the total mental state score.

S106: Determine the recommended health plan among the health plans to be selected according to the mental state of the user.

Optionally, a suitable mental state is marked for the preset health plan in advance, so that a suitable health plan can be conveniently selected according to the mental state.

In some embodiments, the health plan to be selected also includes implementation difficulty, duration, recommended eating season and recommended eating weather of the suggested food, and determining the recommended health plan in the health plan to be selected according to the mental state of the user includes:

Obtain the initial time, initial season, initial weather conditions, and the implementation difficulty and duration of each health plan to be selected;

According to the initial season, the initial weather conditions, the recommended eating weather and the recommended eating season, determine several first alternative health programs from each candidate health program;

Determine a number of second alternative health programs from each of the first alternative health programs according to the initial time and duration;

The recommended health plan is determined from the second alternative health plans according to the user's mental state and implementation difficulty.

Among them, the initial time, initial season, and initial weather state can be determined according to the current time, current season, and current weather state, or can be determined by relevant information of a period specified by those skilled in the art, such as the corresponding information for the next week or the next three days Sure. Optionally, the initial time, initial season, and initial weather state may also be determined according to relevant information corresponding to when the initial tongue image was collected.

Foods that are easier to obtain in the season can be determined according to the season to improve the implementability of the health plan. If it is winter, try not to use a health plan that includes foods that ripen more in summer as the first alternative health plan, such as mangoes. Suitable food can be selected according to the weather conditions. For example, when the temperature is low, a health plan including warming ingredients such as mutton can be determined as the first alternative health plan. For the health plan to be selected, the recommended eating weather and recommended eating season can be marked in advance according to the food and other information included in the plan, so as to facilitate the selection of the first alternative health plan.

The production process of some ingredients is cumbersome and takes a long time. If the time is too late at this time, it may take too long for the user to process the ingredients according to the recommended food and recommended eating methods of the health plan, causing the user to miss the meal. Not good, so you can select the second alternative health plan from the first alternative health plan based on the current time (initial time) to screen out those health plans that take too long and improve the user's feasibility for health plan execution and satisfaction.

The satisfaction through the processing and processing of food materials can improve the mood of some users to a certain extent and make them feel happy. However, for some users, they prefer simple food processing, and complex food processing will make them feel worse. Poor) and implementation difficulty to determine the recommended health program from each second alternative health program.

Optionally, the health plan includes but is not limited to implementation difficulty, duration, recommended food and eating methods, etc. After the recommended health plan is determined, the above information included in the health plan can be transmitted through voice, video, pictures, text, etc. displayed to the user.

In some embodiments, after determining the recommended health plan among the health plans to be selected according to the mental state of the user, the method further includes at least one of the following:

Obtain the recommended food and eating method in the recommended health plan, determine the required auxiliary ingredients according to the eating method, and generate a purchase list based on the recommended food and auxiliary ingredients. After obtaining the order confirmation information, send the purchase list to the preset shopping platform. Generate orders;

Send the tongue quality state, tongue coating state, user mental state associated information, tongue associated information and recommended health plan to the preset object;

Determine the user's health status according to the tongue quality status, tongue coating status, user's mental state related information and tongue related information, and prompt.

The eating method includes but not limited to the cooking method, according to the cooking method, various auxiliary ingredients can be obtained, which is convenient for users to place an order on the online shopping platform. The order confirmation information may be information directly determined by the user after the purchase list is presented to the user, or may be new confirmation information including the modified purchase list obtained after the user modifies the purchase list. Send the purchase list to the preset shopping platform to generate an order. When the user completes the payment, the ingredients and tools needed for the health plan can be delivered to the designated place, making it more convenient for the user to implement the health plan.

Sometimes, people who have fostered, fostered, or other relationships with the user need to know the current health condition of the user, so they can combine the tongue quality, tongue coating state, user's mental state related information, tongue related information and recommendation information with the user's informed consent. The health plan is notified to relevant personnel through text messages, WeChat, emails, voice calls, etc. For example, the tongue quality status, tongue coating status, user mental state related information, tongue related information and recommended health plan are sent to the designated email address, etc.

The current health status of the user can be obtained through different tongue quality status, tongue coating status and tongue related information. At the same time, the user's mental state is determined through the user's mental state-related information, and then the user's health status is obtained according to the health status and user's mental state, and the user is reminded of the current user's health status through different colors of logos or different words.

Optionally, determining the user's health status according to the tongue quality state, tongue coating state, user mental state association information, and tongue association information includes but is not limited to the following methods:

Grading and scoring various tongue quality states, tongue coating states, user mental state related information and tongue related information in advance to obtain a score comparison table;

In the score comparison table, look up the respective scores of the user's current tongue state, tongue coating state, user's mental state-related information and tongue-related information and add them up to obtain the user's total health score;

According to the corresponding relationship between the total health score and the preset user health status, the user health status is determined.

For example, determine the total mental state score corresponding to the user's mental state related information through the above-mentioned method of determining the total mental state score, and then find the corresponding mental state score in the score comparison table according to the total mental state score; According to the tongue association information, look up the scores corresponding to each tongue association sub-information in the score comparison table, and then add up to get the tongue association score; find the tongue quality score in the score comparison table according to the tongue quality state, according to Tongue coating status Find the tongue coating score in the score comparison table, and add up the mental state score, tongue association score, tongue quality score, and tongue coating score to obtain the total health score. Then find the preset user health status corresponding to the total health score as the user health status. For example, if the total health score is 80 points, the corresponding preset user health status is sub-health, the total health score is 60 points, the corresponding preset user health status is relatively unhealthy, the total health score is 30 points, and the corresponding preset user health status is 30 points. Let the user's health status be very unhealthy. The specific mapping relationship between the total health score and the preset user health status can be set by those skilled in the art as needed.

Optionally, another way to determine the user's health status based on the tongue quality state, tongue coating state, user mental state association information, and tongue association information includes but is not limited to the following methods:

Determining the user's health status according to the number of reaching preset conditions, where the preset conditions include at least one of the following:

The tongue state includes a preset tongue state;

The state of the tongue coating includes the preset state of the tongue coating;

Tongue associated information includes preset tongue associated information;

The mental state score determined by the associated information of the user's mental state reaches the preset mental state score.

Among them, the preset tongue quality state, the preset tongue coating state, the preset tongue related information, and the preset mental state score can be set by those skilled in the art.

Optionally, if one preset condition is met, the user health status includes sub-health, if two preset conditions are met, the user health status includes relatively unhealthy, and if three or more preset conditions are met, the user health status includes very unhealthy.

In some embodiments, after determining the recommended health plan among the health plans to be selected according to the mental state of the user, the method further includes:

Acquiring subsequent tongue images, the subsequent acquisition time of the subsequent tongue images is later than the initial acquisition time of the initial tongue images;

Comparing the follow-up tongue image with the standard tongue image to obtain the follow-up first similarity;

If the subsequent first similarity is lower than the preset initial similarity threshold, the subsequent tongue image is compared with the initial tongue image to obtain the subsequent second similarity;

If the subsequent second similarity is higher than the preset subsequent similarity threshold, obtain the elapsed time between the subsequent acquisition time and the initial acquisition time;

If the elapsed time exceeds the preset time threshold, recommend suitable doctors and/or medicines based on the tongue quality, tongue coating and tongue-related information.

If for a certain health problem, after a period of time (elapsed time longer than the preset time threshold) of dietary therapy (eating the ingredients in the recommended plan), there is no good improvement (the subsequent second similarity is higher than the preset subsequent similarity threshold ), at this time, the corresponding drug treatment or doctor can be recommended to help the user solve the health problem.

Optionally, if the subsequent tongue image is similar to the standard tongue image, it means that the user's physical state has returned to normal; otherwise, the user's physical state is still in an abnormal state.

Optionally, if the follow-up tongue image is not similar to the initial tongue image, that is, the follow-up second similarity is lower than the preset follow-up similarity threshold, at this time, a suitable doctor and/or medicine can be recommended, or it can be re-according to the new Subsequent tongue images of the new algorithm are used to determine the new recommended health regimen.

Through the above implementation manner, it is possible to further follow up the change of the user's health status, and adjust the health management plan in time if the health status changes. If the previous health management plan has not worked, timely adjuvant drug treatment is required.

The embodiment of the present application provides a health plan recommendation method based on deep learning. The method acquires the user's initial tongue image and compares the initial tongue image with the standard tongue image to obtain the initial similarity. If the initial The similarity is lower than the preset initial similarity threshold, and the initial tongue coating image and the initial tongue texture image are respectively extracted from the initial tongue image, and the initial tongue coating image is input into the preset tongue coating detection model to obtain the state of the tongue coating, and the initial tongue texture image is input to Go to the preset tongue quality detection model, get the tongue quality status, determine the tongue related information according to the tongue quality status and the tongue coating status, and determine several health plans to be selected, and obtain the user's mental state related information and several user pictures within the preset time period. Behavioral images, and determine the user's mental state, and determine the recommended health plan among the candidate health plans according to the user's mental state. Get the tongue quality and tongue coating status through the tongue image through the preset tongue coating detection model and the preset tongue quality detection model to determine the health plan to be selected, and determine the recommended health plan based on the user's mental state, combined with the user's physical health and mental state Two factors recommend the appropriate health plan, the consideration dimension is more comprehensive, and the user satisfaction is improved, which is simple and feasible.

In one embodiment, as shown in FIG. 4 , the embodiment of the present application also provides a device 400 for recommending a health plan based on deep learning, which includes:

An image acquisition module 401, configured to acquire an initial tongue image of the user, and compare the initial tongue image with a standard tongue image to obtain an initial similarity;

An image extraction module 402, configured to extract an initial tongue coating image and an initial tongue quality image from the initial tongue image if the initial similarity is lower than the preset initial similarity threshold;

The detection module 403 is configured to input the initial tongue coating image into the preset tongue coating detection model to obtain the tongue coating state, and input the initial tongue texture image to the preset tongue texture detection model to obtain the tongue texture state;

Alternative options determination module 404, used to determine tongue-related information according to the state of tongue quality and tongue coating, and determine several health options to be selected, including suggested foods and eating methods;

The mental state determination module 405 is used to obtain the relevant information of the user's mental state and several user behavior images within a preset time period, and determine the user's mental state;

The recommending module 406 is configured to determine a recommended health plan among candidate health plans according to the mental state of the user.

In one embodiment, the image extraction module includes a tongue coating extraction module and a tongue quality extraction module, wherein the tongue coating extraction module is used to extract an initial tongue coating image from an initial tongue image, and the tongue quality extraction module is used to extract an initial tongue coating image from an initial tongue image. Initial tongue image.

Referring to Fig. 5, the tongue coating extraction module 500 includes:

An image division module 501, configured to divide the initial tongue image into several initial image blocks;

The average value determination module 502 is used to obtain the RGB values of the pixels in each initial image block respectively, and determine the average RGB value of the pixels of each initial image block;

The suspected tongue coating image block determination module 503 is used to determine the suspected tongue coating image block from each initial image block according to the pixel average RGB value and the preset tongue coating RGB value range;

A credible image block determining module 504, configured to acquire image block position information of a suspected tongue coating image block, and determine a credible image block, where the credible image block includes a suspected tongue coating image block whose distance from at least one other suspected tongue coating image block is less than a preset distance threshold. Tongue coating image blocks;

The splicing module 505 is configured to splice the credible image blocks to obtain an initial tongue coating image.

In one embodiment, the user behavior image includes the user's facial image, the user's mental state related information includes information content and information content influencing factors, and the mental state determination module further includes:

The position information acquisition module is used to obtain key point position information of target facial key points in the user's facial image, and the target facial key points include nose tip key points, mouth corner key points and eye corner key points;

The distance determination module is used to respectively obtain the first distance between the eye corner key point and the nose tip key point in each user's facial image, and the second distance between the mouth corner key point and the nose tip key point, and respectively determine the first average value of several first distances and the first median, respectively determine the second average and the second median of several second distances;

The first mental state score determination module is used to determine the first mental state score according to the first median, the first average, the second median and the second average;

The second mental state score determination module is used to determine the second mental state score according to the information content and the information content image factor;

The user's mental state determination module is configured to determine the user's mental state according to the first mental state score and the second mental state score.

In one embodiment, the health plan to be selected also includes the difficulty of implementation, the duration, the recommended consumption season and the recommended consumption weather of the suggested food, and the recommendation module includes:

The information acquisition module is used to acquire the initial time, the initial season, the initial weather conditions, and the implementation difficulty and duration of each health plan to be selected;

The first alternative health program determination module is used to determine several first alternative health programs from each candidate health program according to the initial season, initial weather conditions, recommended eating weather and recommended eating season;

The second alternative health program determination module is used to determine several second alternative health programs from each of the first alternative health programs according to the initial time and the duration;

The recommended health plan determination module is used to determine the recommended health plan from the second alternative health plans according to the user's mental state and implementation difficulty.

In one embodiment, the device for recommending health solutions based on deep learning also includes:

The order generation module is used to obtain the recommended food and eating method in the recommended health plan, determine the required auxiliary ingredients according to the eating method, and generate a purchase list based on the recommended food and auxiliary ingredients. After obtaining the order confirmation information, send the purchase list Go to the default shopping platform and generate an order;

The sending module is used to send the tongue quality state, tongue coating state, user mental state related information, tongue related information and recommended health plan to preset objects;

The prompting module is used to determine the user's health status according to the state of the tongue quality, the state of the tongue coating, the associated information of the user's mental state, and the associated information of the tongue, and provide a prompt.

In one embodiment, the deep learning-based health plan recommendation device also includes a doctor and/or drug recommendation module, which is used for:

Acquiring subsequent tongue images, the subsequent acquisition time of the subsequent tongue images is later than the initial acquisition time of the initial tongue images;

Comparing the follow-up tongue image with the standard tongue image to obtain the follow-up first similarity;

If the subsequent first similarity is lower than the preset initial similarity threshold, the subsequent tongue image is compared with the initial tongue image to obtain the subsequent second similarity;

If the subsequent second similarity is higher than the preset subsequent similarity threshold, obtain the elapsed time between the subsequent acquisition time and the initial acquisition time;

If the elapsed time exceeds the preset time threshold, recommend suitable doctors and/or medicines based on the tongue quality, tongue coating and tongue-related information.

In one embodiment, the device for recommending health solutions based on deep learning also includes a pre-selection module, which is used to obtain the user's initial tongue image in the image acquisition module, and compare the initial tongue image with the standard tongue image Yes, before getting the similarity, perform the following steps:

Obtain the user's original tongue image, and identify the original tongue coating image in the original tongue image;

Obtain the hue of the original tongue coating image;

If the hue belongs to the preset hue, the original tongue image is qualified, and the original tongue image is used as the initial tongue image;

If the hue does not belong to the preset hue, the user is prompted to clean the tongue.

The embodiment of the present application provides a device for recommending a health plan based on deep learning, which obtains the state of the tongue quality and the state of the tongue coating through the tongue image through the preset tongue coating detection model and the preset tongue quality detection model, so as to determine the health plan to be selected. The user's mental state determines the recommended health plan, and recommends an appropriate health plan based on the user's physical health and mental state. The consideration dimension is more comprehensive and the user satisfaction is improved. It is simple and feasible.

In this embodiment, the device for recommending a health plan based on deep learning executes the method for recommending a health plan based on deep learning described in any of the above-mentioned embodiments. For specific functions and technical effects, please refer to the above-mentioned embodiment, and details will not be repeated here. .

In one embodiment, referring to FIG. 6, this embodiment also provides a computer device 600, including a memory 601, a processor 602, and a computer program stored in the memory and operable on the processor. The processor 602 The steps of the method described in any one of the above embodiments are realized when the computer program is executed.

In one embodiment, there is also provided a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described in any one of the above embodiments are implemented. The computer-readable storage medium may be non-volatile or volatile.

The embodiments of the present application may acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. .

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, apparatus, article or method comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional same elements in the process, apparatus, article or method comprising the element.

The serial numbers of the above embodiments of the present application are for description only, and do not represent the advantages and disadvantages of the embodiments. Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium as described above (such as ROM/RAM , magnetic disk, optical disk), including several instructions to enable a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structures or equivalent process transformations made by using the description of the application and the accompanying drawings are directly or indirectly used in other related technical fields. , are all included in the patent protection scope of the present application in the same way.

Claims (20)

1. A method for recommending a health plan based on deep learning, wherein the method includes:

   Obtaining an initial tongue image of the user, and comparing the initial tongue image with a standard tongue image to obtain an initial similarity;

   If the initial similarity is lower than the preset initial similarity threshold, extracting an initial tongue coating image and an initial tongue quality image respectively from the initial tongue image;

   Inputting the initial tongue coating image into a preset tongue coating detection model to obtain a tongue coating state, and inputting the initial tongue texture image into a preset tongue texture detection model to obtain a tongue texture state;

   Determine tongue-related information according to the state of the tongue quality and the state of the tongue coating, and determine several health options to be selected, the health options to be selected include suggested foods and eating methods;

   Obtain information related to the user's mental state and several user behavior images within a preset time period, and determine the user's mental state;

   A recommended health plan is determined among the candidate health plans according to the mental state of the user.
2. The health program recommendation method according to claim 1, wherein the method of extracting the initial tongue coating image from the initial tongue image comprises:

   Dividing the initial tongue image into several initial image blocks;

   Obtain the RGB values of the pixels in each of the initial image blocks respectively, and determine the average RGB value of the pixels in each of the initial image blocks;

   Determine the suspected tongue coating image block from each of the initial image blocks according to the pixel average RGB value and the preset tongue coating RGB value range;

   Acquiring image block position information of the suspected tongue coating image block, and determining a credible image block, the credible image block including the suspected tongue coating image whose distance from at least one other said suspected tongue coating image block is less than a preset distance threshold piece;

   The credible image blocks are spliced to obtain the initial tongue coating image.
3. The method for recommending a health plan according to claim 2, wherein the preset tongue coating RGB value range is determined according to the average RGB value of the pixel point and the distribution of the average RGB value of the pixel point.
4. The method for recommending a health plan according to claim 1, wherein the user behavior image includes a user's facial image, and the user's mental state-related information includes information content and information content influencing factors, and the method for determining the user's mental state includes:

   Obtain key point position information of target facial key points in the user's facial image, where the target facial key points include nose tip key points, mouth corner key points, and eye corner key points;

   Respectively acquire the first distance between the key point of the corner of the eye and the key point of the tip of the nose in each of the facial images of the user, and the second distance between the key point of the corner of the mouth and the key point of the tip of the nose, and respectively determine the first distance between the key points of the corner of the mouth and the key point of the tip of the nose. an average value and a first median, respectively determining a second average value and a second median of a plurality of said second distances;

   determining a first mental state score based on the first median, first average, second median, and second average;

   determining a second mental state score based on the information content and the information content imaging factor;

   The mental state of the user is determined according to the first mental state score and the second mental state score.
5. The method for recommending a health plan according to claim 4, wherein the information content includes at least one of the number of times of laughter, times of daze, times of crying, times of sighing, working hours, and non-working time of going out.
6. The health program recommendation method as claimed in claim 4, wherein, determining the first mental state score according to the first median, the first average value, the second median value and the second average value comprises:

   If the first median is smaller than the first average value, and the second median is smaller than the second average value, then obtain a preset first facial score as the first mental state score;

   Otherwise, acquire a preset second facial score as the first mental state score.
7. The method for recommending a health program according to claim 1, wherein the health program to be selected further includes implementation difficulty, time duration, recommended eating season and recommended eating weather of the suggested food, and the user’s mental state is based on the user’s mental state. Determining the recommended health program in the candidate health program includes:

   Obtain the initial time, initial season, initial weather conditions, and the implementation difficulty and time-consuming period of each health plan to be selected;

   According to the initial season, the initial weather conditions, the recommended eating weather and the recommended eating season, determine several first alternative health programs from each candidate health program;

   Determine a number of second health options from each of the first health options according to the initial time and duration;

   A recommended health plan is determined from each of the second alternative health plans according to the user's mental state and implementation difficulty.
8. The method for recommending a health plan according to claim 1, wherein, after determining the recommended health plan in the candidate health plan according to the mental state of the user, the method further includes at least one of the following:

   Obtain the recommended food and eating method in the recommended health plan, determine the required auxiliary ingredients according to the eating method, and generate a purchase list according to the recommended food and auxiliary ingredients, and after obtaining the order confirmation information, the purchased The list is sent to the preset shopping platform to generate an order;

   Send the tongue quality state, tongue coating state, user mental state associated information, tongue associated information and recommended health plan to the preset object;

   Determine the user's health status according to the tongue quality status, tongue coating status, user mental status related information and tongue related information, and prompt.
9. The method for recommending a health plan according to claim 8, wherein determining the user's health status according to the state of the tongue, the state of the tongue coating, the associated information of the user's mental state, and the associated information of the tongue includes:

   Grading and scoring each of the tongue quality state, tongue coating state, user mental state related information and tongue related information in advance to obtain a score comparison table;

   In the score comparison table, look up the respective scores of the user's current tongue state, tongue coating state, user's mental state-related information and tongue-related information and add them up to obtain the user's total health score;

   According to the corresponding relationship between the total health score and the preset user health status, the user health status is determined.
10. The health program recommendation method according to claim 8, wherein determining the user's health status according to the tongue state, tongue coating state, user's mental state related information and tongue related information includes determining the user's health status according to the number of preset conditions. Health status, the preset conditions include at least one of the following:

    The tongue state includes a preset tongue state;

    The state of the tongue coating includes the preset state of the tongue coating;

    Tongue associated information includes preset tongue associated information;

    The mental state score determined by the associated information of the user's mental state reaches the preset mental state score.
11. The method for recommending a health plan according to claim 8, wherein the eating method is presented by at least one of recipes, tips, and video links.
12. The health program recommendation method as claimed in claim 8, wherein said method further comprises:

    The eating method and the production process video are determined according to the recommended health plan, and the production process video is played.
13. The method for recommending a health plan according to claim 1, wherein the candidate health plan is determined according to the tongue-related information, the state of the tongue quality, the state of the tongue coating, and health-related information, and the health-related information includes At least one of gender, age, underlying disease, living environment climate, height, weight, body temperature, sleep pattern, blood pressure, blood sugar, heart rate, and sedentary state.
14. The method for recommending a health program according to any one of claims 1-13, wherein, after determining the recommended health program in the health program to be selected according to the mental state of the user, the method further comprises:

    acquiring a subsequent tongue image, the subsequent acquisition time of the subsequent tongue image is later than the initial acquisition time of the initial tongue image;

    Comparing the subsequent tongue image with the standard tongue image to obtain the subsequent first similarity;

    If the subsequent first similarity is lower than the preset initial similarity threshold, comparing the subsequent tongue image with the initial tongue image to obtain a subsequent second similarity;

    If the subsequent second similarity is higher than the preset subsequent similarity threshold, obtain the elapsed time between the subsequent acquisition time and the initial acquisition time;

    If the elapsed time exceeds the preset time threshold, a suitable doctor and/or medicine is recommended according to the tongue state, tongue coating state and tongue related information.
15. The health program recommendation method according to any one of claims 1-13, wherein said acquiring the user's initial tongue image and comparing said initial tongue image with a standard tongue image to obtain similarity , the method also includes:

    Obtaining the original tongue image of the user, and identifying the original tongue coating image in the original tongue image;

    Obtain the hue of the original tongue coating image;

    If the hue belongs to the preset hue, the original tongue image is qualified, and the original tongue image is used as the initial tongue image;

    If the hue does not belong to the preset hue, the user is prompted to clean the tongue.
16. The health program recommendation method according to any one of claims 1-13, wherein the way of comparing the initial tongue image with the standard tongue image comprises:

    Grayscale processing is performed on the initial tongue image and the standard tongue image respectively, and normalized to a preset pixel size to obtain a grayscale initial tongue image and a grayscale standard tongue image;

    Determine the average value of the grayscale values of each pixel in the grayscale initial tongue image and the grayscale standard tongue image respectively, to obtain the first average grayscale value of the grayscale initial tongue image and the grayscale standard tongue image. The second average gray value of the internal image;

    Comparing the grayscale value of each pixel in the grayscale initial tongue image with the first average grayscale value, if the grayscale value of a pixel is greater than the first average grayscale value, the conversion value is recorded as N, Otherwise, the conversion value is recorded as M, and the initial code is formed according to the arrangement order of the pixels in the gray-scale initial tongue image;

    The grayscale value of each pixel in the grayscale standard tongue image is compared with the second average grayscale value, if the grayscale value of a pixel is greater than the second average grayscale value, then the conversion value is recorded as N, Otherwise, the conversion value is recorded as M, and a standard code is formed according to the arrangement order of the pixels in the gray-scale standard tongue image;

    Comparing the initial code with the standard code to obtain an initial similarity.
17. The health program recommendation method according to any one of claims 1-13, wherein the way of comparing the initial tongue image with the standard tongue image comprises:

    Performing feature vector extraction on the initial tongue image and the standard tongue image respectively, to obtain an initial tongue image feature vector and a standard tongue image feature vector;

    Determine the cosine similarity between the initial tongue image feature vector and the standard tongue image feature vector as the initial similarity.
18. A device for recommending a health plan based on deep learning, wherein the device includes:

    An image acquisition module, configured to acquire an initial tongue image of the user, and compare the initial tongue image with a standard tongue image to obtain an initial similarity;

    An image extraction module, configured to extract an initial tongue coating image and an initial tongue quality image from the initial tongue image if the initial similarity is lower than a preset initial similarity threshold;

    A detection module, configured to input the initial tongue coating image into a preset tongue coating detection model to obtain a tongue coating state, and input the initial tongue texture image to a preset tongue texture detection model to obtain a tongue texture state;

    Alternative options determination module, used to determine tongue-associated information according to the state of tongue quality and tongue coating state, and determine several health options to be selected, the health options to be selected include suggested foods and eating methods;

    The mental state determination module is used to obtain the associated information of the user's mental state and several user behavior images within a preset time period, and determine the user's mental state;

    A recommendation module is configured to determine a recommended health plan among the candidate health plans according to the mental state of the user.
19. A computer device, comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein, when the processor executes the computer program, the method described in any one of claims 1 to 17 is realized method steps.
20. A computer-readable storage medium on which a computer program is stored, wherein the computer program implements the steps of the method according to any one of claims 1 to 17 when executed by a processor.

Images