JP2021010652A - Information processing device, evaluation method, and information processing program - Google Patents

Abstract

To calculate a fatigue degree or a skin age recognized from skin appearance.SOLUTION: An information processing device (1) comprises an evaluation value calculation unit (104) for calculating a fatigue degree of an object person from color information on a prescribed part reflecting in an image obtained by photographing the face of the object person by using an evaluation model generated on the basis of a fatigue degree of an evaluatee determined by an evaluator from an impression of the face of the evaluatee and color information on a prescribed part reflected in the image obtained by photographing the face of the evaluatee.SELECTED DRAWING: Figure 1

Description

The present invention relates to an evaluation system or the like that evaluates the impression of a human face using an information processing device.

Many people realize that the appearance of the face changes depending on their physical condition. For example, many women feel that their faces look tired in the evening than in the morning, and their skin age looks older. Against this background, there is an increasing demand for skin care cosmetics and the like that can suppress deterioration of the appearance of the face. In general, an index expressing the skin condition of the evaluated person by age by comparing the skin condition of the evaluated person with the average skin condition by age is called a skin age. .. On the other hand, the "skin age" of the above "skin age seems to be bulky" and the "skin age" in the following explanation are also indicators of the skin condition of the evaluated person, but this "skin age" is evaluated. It is the age of the evaluated person inferred based on the appearance impression (impression received from the skin) of the person's skin.

Japanese Unexamined Patent Publication No. 11-265443 (published on September 28, 1999) Japanese Unexamined Patent Publication No. 2016-193175 (published on November 17, 2016)

The impression that the skin looks tired or the skin age looks bulky is subjective and varies depending on the evaluator, so it is difficult to judge the effect of skin care cosmetics and the like. Therefore, there is a demand for a technique for quantifying the above impression by information processing using a computer or the like.

Here, in Patent Document 1, the feature amounts (size of each part of the face, etc.) of the facial images of a plurality of people and the impression element axes of "adult-like-childish" and "masculine-feminine" are described. A computer that calculates the evaluation value of the facial impression using the regression equation with the evaluation value of the impression evaluated in 1 is described. However, since this computer evaluates based on the size of each part of the face, it is considered that the evaluation results are basically the same when the same person is evaluated. Therefore, the technique of Patent Document 1 is not suitable for evaluating an impression that can fluctuate in a relatively short time, such as the degree of fatigue and skin age recognized from the appearance of the skin.

Further, in Patent Document 2, the images of the face taken from a plurality of angles are analyzed, the physical features of the face that change from different angles are measured, and the face is selected from the measured physical features. A computer device that extracts factors that are highly correlated with the appearance impression of is described. The technique of Patent Document 2 focuses on the amount of physical features of the face that change depending on the angle, and does not focus on the impression given by the appearance of the skin, but the degree of fatigue recognized from the appearance of the skin. Not suitable for evaluating skin age.

One aspect of the present invention is to realize an information processing apparatus or the like capable of calculating an evaluation value indicating the degree of fatigue or skin age recognized from the appearance of the skin, or the degree of change over time.

In order to solve the above problems, the information processing apparatus according to one aspect of the present invention has at least one of the brightness of the color and the degree of color unevenness of a predetermined part of the face appearing in the image of the subject of the impression evaluation. The color information acquisition unit that acquires the color information indicating the above, the evaluation result that the evaluator evaluated the fatigue degree or the skin age of the evaluated person, and the color information of the above-mentioned predetermined part that appears in the image of the evaluated person's face. Using the evaluation model generated based on the above, an evaluation value calculation unit that calculates an evaluation value indicating the degree of fatigue or skin age of the subject from the color information acquired by the color information acquisition unit. I have.

In order to solve the above-mentioned problems, the information processing apparatus according to another aspect of the present invention has a predetermined part of the face in each image taken after a predetermined period of time for the subject of the impression evaluation, and has a color of the predetermined part of the face in the predetermined period. A color information acquisition unit that acquires at least one of the amount of change in brightness and the amount of change in the degree of unevenness in color brightness during the predetermined period, and the evaluator is the degree of fatigue of the evaluated person or It is generated based on each evaluation result obtained by evaluating the skin age after the predetermined period and the color information generated from each image of the subject's face taken after the predetermined period. An evaluation value calculation unit that calculates an evaluation value indicating the degree of change in the degree of fatigue or skin age of the subject in the predetermined period from the color information acquired by the color information acquisition unit using the evaluation model. It has.

In order to solve the above-mentioned problems, the evaluation method according to one aspect of the present invention is an evaluation method of impression of a subject executed by one or a plurality of computers, and a face appearing in an image of the subject. The color information acquisition step for acquiring color information indicating at least one of the color brightness and the degree of color unevenness of a predetermined portion of the above, the evaluation result in which the evaluator evaluates the fatigue degree or the skin age of the evaluated person, and the corresponding Using the color information of the predetermined portion reflected in the photographed image of the face of the evaluated person and the evaluation model generated based on the color information, the fatigue of the subject is obtained from the color information acquired in the color information acquisition step. Includes an evaluation value calculation step of calculating an evaluation value indicating degree or skin age.

In order to solve the above problems, the evaluation method according to another aspect of the present invention is an evaluation method of impression of a subject executed by one or more computers, and the subject is photographed after a predetermined period of time. Acquire color information indicating at least one of the amount of change in color brightness during the above-mentioned predetermined period and the amount of change in the degree of color brightness unevenness in the above-mentioned predetermined period for a predetermined part of the face reflected in each of the images. The color information acquisition step to be performed, each evaluation result obtained by the evaluator evaluating the degree of fatigue or skin age of the evaluated person after the above-mentioned predetermined period, and the face of the evaluated person are photographed after the above-mentioned predetermined period. Using the color information generated from each of the images and the evaluation model generated based on the color information, the color information acquired in the color information acquisition step is used to determine the degree of fatigue or skin age of the subject. It includes an evaluation value calculation step of calculating an evaluation value indicating the degree of change in a period.

According to one aspect of the present invention, it is possible to calculate an evaluation value indicating the degree of fatigue or skin age recognized from the appearance of the skin, or the degree of change over time.

It is a block diagram which shows an example of the main part structure of the information processing apparatus which concerns on Embodiment 1 of this invention. It is a figure which shows the configuration example of the evaluation system including the said information processing apparatus. It is a figure which shows the evaluation result which the evaluator evaluated the impression of the face of the evaluated person, and the color information of the predetermined part of the evaluated person appearing in the photographed image of the face of the evaluated person. It is a figure which shows the comparison result between the predicted value calculated by the regression equation and the measured value about the evaluation value of the degree of fatigue and the skin age. It is a figure which shows an example of the process (evaluation method) executed by the said evaluation system. It is a figure which shows the structural example of the evaluation system which concerns on Embodiment 2 of this invention. It is a figure which shows the correlation coefficient calculated about the change amount of the evaluation result which evaluated the impression of the face of the evaluated person by the evaluator, and the change amount about the color of a predetermined part. It is a figure which shows the comparison result between the predicted value calculated by the regression equation and the actually measured value about the evaluation value which shows the change amount of the degree of fatigue and the evaluation value which shows the change amount of the skin age. It is a figure which shows an example of the process (evaluation method) executed by the said evaluation system. It is a figure which shows the structural example of the evaluation system which concerns on Embodiment 3 of this invention.

[Embodiment 1]
(System overview)
The outline of the facial impression evaluation system 100 according to the present embodiment will be described with reference to FIG. FIG. 2 is a diagram showing a configuration example of the evaluation system 100. The details will be described below, but the evaluation system 100 can analyze an image of the face of the subject A for face impression evaluation and calculate an evaluation value of the facial impression of the subject A. As shown in FIG. 2, the evaluation system 100 includes an information processing device 1 that calculates the evaluation value, a photographing device 2 that photographs the face of the subject A, and a display device 3 that is an output device of the evaluation result. include.

The information processing device 1 analyzes the image captured by the photographing device 2, calculates an evaluation value of the impression of the face of the subject A, and displays the calculated evaluation value on the display device 3. The information processing device 1 can be configured by, for example, a PC (Personal Computer) or the like. The details of the information processing apparatus 1 will be described later based on FIG.

The photographing device 2 is a device that photographs an image of the face of the subject A, and captures the photographing unit 2b in which five directions (upper, lower, left, right, and depth directions) other than the front are surrounded by the cover portion 2a. I have. The photographing device 2 photographs the face of the subject A by the photographing unit 2b with the subject A having the face inside the cover portion 2a from the front side of the photographing device 2 which is the opening of the cover portion 2a. To do. In order to prevent the cover portion 2a from hitting the face of the subject A with external light, the photographing device 2 can capture an image in which the influence of the external light is minimized. Further, although not shown, the photographing device 2 includes a lighting unit that illuminates the face of the subject A, a support portion for fixing the position and angle of the face of the subject A in the cover portion 2a, and the like. You may be. Of course, the photographing device 2 is not limited to the illustrated example as long as it can capture an image capable of evaluating an impression.

The display device 3 displays the evaluation value calculated by the information processing device 1. In the example in Figure 2, "Your fatigue is 42. It has increased by 12 compared to this morning. ] Is displayed on the display device 3. This degree of fatigue is an evaluation value calculated by the information processing apparatus 1, and indicates how tired the subject A looks. The method of outputting the evaluation result is arbitrary and is not limited to the example of FIG. For example, the evaluation result may be output by voice, or the evaluation result may be output by printing. In the former case, the evaluation system 100 may include an audio output device such as a speaker, and in the latter case, the evaluation system 100 may include a print output device such as a printer.

(Device configuration)
The configuration of the information processing device 1 will be described with reference to FIG. FIG. 1 is a block diagram showing an example of a main configuration of the information processing device 1. As shown in the figure, the information processing device 1 includes a control unit 10 that controls and controls each part of the information processing device 1, a storage unit 20 that stores various data used by the information processing device 1, and the information processing device 1. A communication unit 30 for communicating with the device of the above is provided. Further, the control unit 10 includes an image acquisition unit 101, a color information acquisition unit 102, an evaluation value calculation unit 104, a comparative evaluation unit 105, and an output control unit 106. Further, the storage unit 20 includes a color information storage unit 201 that stores the color information acquired by the color information acquisition unit 102 and an evaluation result storage unit 202 that stores the evaluation value calculated by the evaluation value calculation unit 104. There is.

The image acquisition unit 101 acquires an image of the face of the subject A. In the evaluation system 100, since the photographing device 2 photographs the face of the subject A, the image acquisition unit 101 acquires the image photographed by the photographing device 2. For example, the image acquisition unit 101 may acquire an image from the photographing device 2 by communication via the communication unit 30. The image acquisition method is not particularly limited, and for example, the operator of the evaluation system 100 may input the image data captured by the photographing device 2 into the information processing device 1. In this case, the image acquisition unit may be used. 101 acquires the input image.

The color information acquisition unit 102 acquires color information regarding the color of a predetermined portion of the image of the subject A's face. The color information acquisition unit 102 of the present embodiment includes a lightness information acquisition unit 1021 and a chromaticity information acquisition unit 1022 in order to acquire color information regarding lightness and color information regarding chromaticity. The predetermined portion may be a portion on the face where the color and brightness of the skin can be recognized (for example, around the cheeks, forehead, eyes, around the nose, etc.).

The brightness information acquisition unit 1021 acquires color information regarding the brightness of the predetermined portion. Specifically, the brightness information acquisition unit 1021 acquires the average value of the L * values of the L * a * b * color system in the predetermined portion as the color information indicating the brightness of the color in the predetermined portion. The brightness information acquisition unit 1021 may specify in which region of the image the predetermined portion is captured by image processing. Further, for example, an operator of the evaluation system 100 may visually determine an area in which a predetermined portion is captured and input the range to the information processing apparatus 1. In this case, the brightness information acquisition unit 1021 may input the above. The area where the predetermined part is reflected is specified from the contents of.

The chromaticity information acquisition unit 1022 acquires color information related to the chromaticity of the predetermined portion. Specifically, the chromaticity information acquisition unit 1022 acquires the standard deviation of the a * value and the standard deviation of the b * value in the predetermined portion as color information indicating the degree of color unevenness in the predetermined portion. The area in which the predetermined portion is captured may be specified in the same manner as in the brightness information acquisition unit 1021. The reason for using such color information will be described later based on FIG.

The evaluation value calculation unit 104 calculates the evaluation value of the facial impression of the subject A from the color information acquired by the color information acquisition unit 102 by using the evaluation model for evaluating the facial impression. Specifically, the evaluation value calculation unit 104 calculates an evaluation value indicating the degree of fatigue of the subject A. Although the details will be described later, in the evaluation model, the evaluation result in which the evaluator evaluates the impression of the face of the evaluated person, the color information of the predetermined portion appearing in the photographed image of the face of the evaluated person, and the color information of the predetermined portion. It is a model generated based on.

The comparative evaluation unit 105 uses a plurality of evaluation values calculated based on images taken at different timings to generate information indicating the degree of change in the degree of fatigue of the subject A. This makes it possible to grasp how much the fatigue level of the subject A has changed. For example, the comparative evaluation unit 105 may use the evaluation value calculated in the morning of a certain day as a comparison target for the evaluation value calculated in the afternoon of a certain day. Further, for example, the comparative evaluation unit 105 may use the average value of the evaluation values calculated in a predetermined period such as one month as a comparison target.

The output control unit 106 causes a predetermined output device to output the evaluation value calculated by the evaluation value calculation unit 104. Specifically, the output control unit 106 causes the display device 3 (FIG. 2) to display the evaluation value calculated by the evaluation value calculation unit 104. Further, the output control unit 106 also causes the display device 3 to display the result of the comparative evaluation by the comparative evaluation unit 105. Note that FIG. 1 shows an example in which the output control unit 106 controls the display device 3 by communication via the communication unit 30, but for example, the information processing device 1 and the display device 3 are connected by wire. In this case, the output control unit 106 may control the display device 3 by using the wired connection.

As described above, the information processing apparatus 1 includes a color information acquisition unit 102 that acquires color information indicating the brightness and the degree of color unevenness of a predetermined portion of the face appearing in the image of the subject A, and an evaluation model. It is provided with an evaluation value calculation unit 104 that calculates the degree of fatigue of the subject A from the color information acquired by the color information acquisition unit 102. Further, the evaluation model is a model generated based on the evaluation result in which the evaluator evaluates the degree of fatigue of the evaluated person and the color information of the predetermined portion reflected in the photographed image of the evaluated person's face. Is. According to this configuration, the degree of fatigue recognized from the appearance of the skin can be calculated. Although the details will be described later, the evaluation value calculation unit 104 includes the evaluation result in which the evaluator evaluates the skin age of the evaluated person, and the color information of the predetermined portion reflected in the photographed image of the evaluated person's face. It is also possible to calculate the skin age of the subject A from the color information acquired by the color information acquisition unit 102 using the evaluation model generated based on. The "skin age" is an index showing the skin condition of the evaluated person, and indicates the age of the evaluated person inferred from the impression received from the skin of the evaluated person.

(Experiment for evaluation model generation)
An experiment conducted by the inventors of the present invention to generate an evaluation model used for calculating the evaluation value will be described with reference to FIG. FIG. 3 is a diagram showing an evaluation result in which the evaluator evaluates the impression of the face of the evaluated person and color information of a predetermined portion of the evaluated person in an image of the face of the evaluated person. The predetermined portion in the example of FIG. 3 is the cheek portion.

FIG. 3 includes a record of "third-party impression evaluation" and a record of "color information". Each numerical value of the "third-party impression evaluation" and "color information" records was obtained as a result of an experiment in which a plurality of evaluators evaluate the impression of the face of the evaluated person. Specifically, in this experiment, a plurality of evaluators evaluated the impression of each of the n images obtained by photographing the faces of n evaluators consisting of adult men and women. In addition, in order to grasp the impression given by the bare skin, the evaluated person did not make makeup in any of the evaluations, and the face image of the evaluated person's bare skin was taken.

As shown in the figure, the evaluation items are skin age, looks tired, pores are conspicuous, complexion is good, skin is bright, skin is dull, skin is firm, and skin is moisturized. There are 9 items in total: visible and the bear under the eyes is conspicuous. Of these, the skin age is the age estimated by the evaluator based on the impression received from the skin of the evaluated person. The numerical values of the remaining eight items are the numerical values that the evaluator scored on a scale of 100 points to the extent that they apply to each item. Note that FIG. 3 shows the arithmetic mean value of the numerical values scored by each evaluator for each of the nine items.

In addition, color information was acquired by analyzing the image used for the evaluation. This result is shown in the "color information" record. The "color information" record includes L * mean, a * mean, b * mean, L * standard deviation, a * standard deviation, and b * standard deviation. The L * average is the arithmetic mean value of the L * value of the cheek portion in the image of the face of the evaluated person used for the evaluation by the evaluator described above. Further, the a * average is the arithmetic mean value of the a * value of the same portion, and the b * average is the arithmetic mean value of the b * value of the same portion.

The L * value is a value indicating the lightness of the color in the L * a * b * color system, and the a * value and the b * value are the chromaticity (hue and color) in the L * a * b * color system. It is a value indicating degree). Further, the L * standard deviation is the standard deviation of the L * value of the above portion, the a * standard deviation is the standard deviation of the a * value of the above portion, and the b * standard deviation is the standard deviation of the b * value of the above portion. It is a deviation. Each standard deviation represents the large number of variations (the width of the peak width when a histogram is generated) of each component (L * value / a * value / b * value) at a predetermined portion.

The experiment was performed four times without changing the n evaluated persons and changing the time zone and the state of the n evaluated persons. The results of the first experiment on the n evaluated subjects are shown in FIGS. 3A-1 to AN. Similarly, the results of the second experiment are shown in B-1 to Bn, the results of the third experiment are shown in C-1 to Cn, and the results of the fourth experiment are shown in D-1 to Dn. .. The first and second experiments were performed in the morning and afternoon of the same day, respectively. Similarly, the third and fourth experiments were performed in the morning and afternoon on different days, respectively. The first experiment and the second experiment were performed before the n evaluated persons continuously used the predetermined skin care product for a predetermined period, and the third experiment and the fourth experiment were performed in the above n. This was done after having a person evaluated continue to use the prescribed skin care product for a predetermined period of time.

The correlation coefficient calculated based on the above experimental results is shown in the last two lines of the "third-party impression evaluation" record. From this correlation coefficient, the skin age and the evaluation value of "looks tired" have a positive correlation with the evaluation value of the evaluation items related to negative impression (prominent pores, dull skin, etc.). You can see that. On the contrary, it can be seen that the skin age and the evaluation value of "looks tired" have a negative correlation with the evaluation value of the evaluation items (good complexion, bright skin, etc.) related to the positive impression. Although not shown, it was confirmed that there is a similar correlation in the results of each of the four experiments.

Further, the correlation coefficient calculated based on the color information obtained by analyzing the image used in the above experiment is shown in the last two lines of the "color information" record. From this correlation coefficient, it can be seen that the skin age and the degree of fatigue have a positive correlation with the standard deviation of the a * value and the standard deviation of the b * value, and have a negative correlation with the mean value of the L * value. .. In addition, the skin age and the evaluation value of "looks tired" have a positive correlation with the average value of the a * values, and the skin age has the standard deviation of the L * value and "looks tired". It can be seen that the evaluation value of "" has a positive correlation with the b * mean. Although not shown, it was confirmed that there is a similar correlation in the results of each of the four experiments.

Since the L * value is a value indicating the brightness of the color, that is, the brightness of the skin, it is considered that the brightness of the skin color affected the skin age and the evaluation value of "looks tired". Further, since the standard deviation of the a * value and the standard deviation of the b * value are values indicating the degree to which the a * value and the b * value are dispersed, the uneven skin color and the like are the skin age and "tired". It is considered that it affected the evaluation value of "looks like".

(About the evaluation model)
From the above experimental results, an evaluation model generated based on the evaluation result in which the evaluator evaluates the impression of the face of the evaluated person and the color information of a predetermined portion appearing in the photographed image of the face of the evaluated person is used. Therefore, it can be seen that the evaluation value of the degree of fatigue or the skin age can be calculated. As the color information, at least one of a value indicating the brightness of the color of the cheek portion reflected in the image and a value indicating the degree of color unevenness in the cheek portion may be used.

In the present embodiment, as an evaluation model for calculating the degree of fatigue, each value of L * average, a * standard deviation, and b * standard deviation obtained by the above experiment and the evaluation value of "looks tired" An example of using a regression equation calculated by performing multiple regression analysis of the correlation with and will be described. In addition, as an evaluation model for calculating skin age, multiple regression analysis was performed on the correlation between each value of L * mean, a * standard deviation, and b * standard deviation obtained by the above experiment and the evaluation value of skin age. An example of using the regression equation calculated in the above will be described. The independent variables of these regression equations are L * mean value, a * standard deviation, and b * standard deviation.

Therefore, the color information acquisition unit 102 uses the average value of the L * value in the predetermined part of the face, the standard deviation of the a * value in the predetermined part, and the standard deviation of the b * value in the predetermined part as color information. get. Then, the evaluation value calculation unit 104 calculates the evaluation value using the above regression equation as the evaluation model. This makes it possible to calculate the degree of fatigue or skin age recognized from the appearance of the skin. When generating a regression equation by multiple regression analysis, at least two of L * mean value, a * standard deviation, and b * standard deviation may be used, and it is not always necessary to use all three.

FIG. 4 shows a comparison result between the evaluation value calculated using the regression equation generated in this way and the actually measured value. FIG. 4 is a diagram showing a comparison result between the predicted value calculated by the regression equation and the measured value for the evaluation values of the degree of fatigue and the skin age. In FIG. 4, each point with the measured value as the value on the horizontal axis and the predicted value as the value on the vertical axis is plotted on the graph plane. The evaluated person and the evaluator are the same as in the above experiment. The actually measured value is an evaluation value evaluated by the evaluator. From this comparison result, it can be seen that the predicted value and the measured value correspond well.

(Processing flow)
The processing flow in the evaluation system 100 will be described with reference to FIG. FIG. 5 is a diagram showing an example of processing (evaluation method) executed by the evaluation system 100. As in the example of FIG. 2, the subject of evaluation is subject A. Further, an example of calculating the degree of fatigue will be described below. However, if the evaluation model used in S14 of FIG. 5 is an evaluation model for calculating skin age, FIG. 5 shows a flow of calculating the evaluation value of skin age. ..

In S11, the photographing device 2 captures an image of the face of the subject A. It is sufficient that the L * value, a * value, and b * value of the predetermined part of the face can be obtained in S13 later. Therefore, in S11, at least an image showing the predetermined part may be taken, and the entire face is not necessarily taken. No need. For example, shooting may be performed with a shooting device different from the shooting device 2.

In S12, the image acquisition unit 101 of the information processing device 1 acquires the image captured in S11. For example, the image acquisition unit 101 may acquire an image from the photographing device 2 by communication via the communication unit 30.

In S13 (color information acquisition step), the color information acquisition unit 102 acquires the color information of a predetermined portion in the image acquired in S12. Specifically, the brightness information acquisition unit 1021 included in the color information acquisition unit 102 calculates the arithmetic mean value of the L * values of each pixel included in the area in which the predetermined portion of the image is captured. Further, the chromaticity information acquisition unit 1022 included in the color information acquisition unit 102 calculates the standard deviation of the a * value and the standard deviation of the b * value of each pixel included in the area in which the predetermined portion of the image is captured. Then, the brightness information acquisition unit 1021 and the chromaticity information acquisition unit 1022 store the calculated values as color information in the color information storage unit 201. At this time, it is preferable that the brightness information acquisition unit 1021 and the chromaticity information acquisition unit 1022 store the identification information of the target person A and the shooting date and time of the image in association with the color information.

In S14 (evaluation value calculation step), the evaluation value calculation unit 104 calculates the fatigue degree from the color information stored in the color information storage unit 201 by using the evaluation model for calculating the fatigue degree. Specifically, the evaluation value calculation unit 104 uses a regression equation having the average of L * values, the standard deviation of a * values, and the standard deviation of b * values as independent variables as the evaluation model. Then, the evaluation value calculation unit 104 substitutes the average of the L * values stored in the color information storage unit 201, the standard deviation of the a * value, and the standard deviation of the b * value into this regression equation to obtain the degree of fatigue. Is calculated, and the calculated value is stored in the evaluation result storage unit 202. At this time, it is preferable that the evaluation value calculation unit 104 stores the identification information of the target person A in association with the calculated value of the shooting date and time of the image.

In S15, the comparative evaluation unit 105 determines whether or not there is an evaluation result to be compared with the index calculated in S14. For example, the comparative evaluation unit 105 stores a plurality of evaluation results calculated for the subject A, including the one calculated in the latest S14, in the evaluation result storage unit 202, and the evaluation result becomes a comparison target. It may be determined that there is (YES in S15). If YES is determined in S15, the process proceeds to S16, and if NO is determined in S15, the process proceeds to S17.

In S16, the comparative evaluation unit 105 calculates the difference between the fatigue degree calculated in the latest S14 and the fatigue degree to be compared. For example, if the fatigue level calculated in the latest S14 is calculated based on an image taken in the evening of one day, the comparative evaluation unit 105 uses the image taken in the morning of that day. The difference from the calculated evaluation value may be calculated. The comparative evaluation unit 105 may calculate the ratio instead of calculating the difference.

In S17, the output control unit 106 outputs the evaluation result. More specifically, when the output control unit 106 determines YES in S15, the output control unit 106 causes the display device 3 to display each value calculated in S14 and S16. As a result, an image like the example of FIG. 2 is displayed on the display device 3. On the other hand, when the output control unit 106 determines NO in S15, the output control unit 106 causes the display device 3 to display the value calculated in S14. In this case, only the evaluation result of this time may be displayed, or the fact that there is no comparison result because there is no comparison target may be displayed.

With the above, the process of FIG. 5 is completed. In S17, it is not always necessary to display a numerical value. For example, if the degree of fatigue is larger than a predetermined reference value, a message such as "looks tired" or if it is small, "looks fine" is displayed to notify the evaluation result. You may.

[Embodiment 2]
Other embodiments of the present invention will be described below. For convenience of explanation, the same reference numerals will be added to the members having the same functions as the members described in the above embodiment, and the description will not be repeated. This also applies to the third and subsequent embodiments.

(System overview)
FIG. 6 is a diagram showing a configuration example of the evaluation system 200 according to the present embodiment. Similar to the evaluation system 100 shown in FIG. 2, the evaluation system 200 includes an information processing device 1 for calculating an evaluation value, a photographing device 2 for photographing the face of a target person, and a display device for outputting an evaluation result. 3 and are included. In the present embodiment, the target person for calculating the evaluation value is the target person B.

In the example of FIG. 6, the display device 3 displays information indicating that the skin age of the subject B has increased by 1 year as compared with this morning. In this way, the evaluation system 200 calculates an evaluation value indicating how much the impression of the subject's face has changed in a predetermined period. Therefore, although the color information and the evaluation model used for calculating the evaluation value are different from the above-described embodiment, the configuration of each device constituting the evaluation system 200 is substantially the same as that of the first embodiment. Therefore, the process executed by the information processing apparatus 1 of the present embodiment will be described with reference to the block diagram of FIG. However, in the information processing apparatus 1 of the present embodiment, the chromaticity information acquisition unit 1022 is unnecessary.

The color information acquisition unit 102 of the present embodiment has the change amount of the color brightness in the predetermined period and the color change in the predetermined period for the predetermined part of the face reflected in each image taken by the subject B after a predetermined period. Acquires color information indicating the amount of change in the degree of unevenness of brightness. More specifically, the brightness information acquisition unit 1021 included in the color information acquisition unit 102 indicates the amount of change in the L * value indicating the color brightness in the predetermined portion and the unevenness of the color brightness in the predetermined portion. The amount of change in the standard deviation of the L * value is acquired as the above color information. The predetermined period is, for example, several hours, and the predetermined portion is, for example, the cheek portion as in the first embodiment.

In addition, the evaluation value calculation unit 104 of the present embodiment uses the evaluation model to determine the degree of change in the fatigue degree or skin age of the subject B in the predetermined period from the color information acquired by the color information acquisition unit 102. Calculate the indicated evaluation value. Then, in the evaluation model, each evaluation result obtained by the evaluator evaluating the fatigue degree or skin age of the evaluated person after the predetermined period and the face of the evaluated person are photographed after the predetermined period. It is a model generated based on the above-mentioned color information generated from each image. According to these configurations, it is possible to calculate an evaluation value indicating the degree of fatigue or the degree of change in skin age recognized from the appearance of the skin.

(About the evaluation model)
As a result of analyzing the experimental results shown in FIG. 3 by the inventors of the present invention, a correlation was established between the amount of change in skin age and the amount of change in the color of a predetermined portion, and similarly, the amount of change in the degree of fatigue. It was found that a correlation was established between and the amount of change in the color of the predetermined part. Therefore, as in the first embodiment, it is possible to calculate the amount of change in skin age and the amount of change in fatigue using the regression equation as an evaluation model. This will be described with reference to FIG. FIG. 7 is a diagram showing a correlation coefficient calculated for the amount of change in the evaluation result in which the evaluator evaluated the impression of the face of the evaluated person and the amount of change in the color of a predetermined portion.

From the calculation results of the gray part in FIG. 7, the amount of change in skin age and the amount of change in the evaluation value of "looks tired" both have a negative correlation with the amount of change in the arithmetic mean value of the L * value. It can be seen that there is a positive correlation with the amount of change in the standard deviation of the L * value.

Since the L * value is a value indicating the brightness of the color, it is considered that the evaluation of the skin age and the degree of fatigue changed due to the change in the brightness of the skin color. In addition, since the standard deviation of the L * value can be said to indicate the degree of unevenness in color brightness, it is considered that the evaluation of skin age and the degree of fatigue changed due to the change in the degree of unevenness in skin color brightness. Be done.

From the above analysis results, each evaluation result obtained by the evaluator evaluating the degree of fatigue of the evaluated person after a predetermined period of time and each image of the evaluated person's face taken after a predetermined period of time are generated. It can be seen that the evaluation value indicating the degree of change in the degree of fatigue can be calculated by using the obtained color information and the evaluation model generated based on the above. Similarly, the color generated from each evaluation result obtained by the evaluator evaluating the skin age of the evaluated person after a predetermined period of time and each image of the evaluated person's face taken after a predetermined period of time. It can be seen that the evaluation value indicating the degree of change in skin age can be calculated by using the information and the evaluation model generated based on the information. As the color information, at least one of the amount of change in the color brightness of the predetermined portion and the amount of change in the color brightness of the predetermined portion in the image may be used.

In the present embodiment, as an evaluation model for calculating the evaluation value indicating the degree of change in the degree of fatigue, the amount of change in the arithmetic mean value of L * and the amount of change in the standard deviation of the L * value obtained by the above experiment. An example of using a regression equation calculated by performing multiple regression analysis on the correlation with the amount of change in the degree of fatigue will be described. In addition, as an evaluation model for calculating the evaluation value indicating the degree of change in skin age, the amount of change in the arithmetic mean value of L * obtained by the above experiment, the amount of change in the standard deviation of the L * value, and the skin age. An example of using a regression equation calculated by multiple regression analysis of the correlation with the amount of change in is described.

FIG. 8 shows a comparison result between the evaluation value calculated by using the regression equation generated in this way and the actually measured value. FIG. 8 is a diagram showing a comparison result between a predicted value calculated by a regression equation and an actually measured value for an evaluation value indicating a change in fatigue and an evaluation value indicating a change in skin age. In FIG. 8, each point with the measured value as the value on the horizontal axis and the predicted value as the value on the vertical axis is plotted on the graph plane. The evaluated person and the evaluator are the same as in the above experiment. The actually measured value is an evaluation value evaluated by the evaluator. From this comparison result, it can be seen that the predicted value and the measured value correspond well.

(Processing flow)
The processing flow in the evaluation system 200 will be described with reference to FIG. FIG. 9 is a diagram showing an example of processing (evaluation method) executed by the evaluation system 200. Similar to the example of FIG. 6, the subject of evaluation is subject B. Further, an example of calculating the amount of change in skin age will be described below. However, if the evaluation model used in S24 is an evaluation model for calculating the amount of change in fatigue, FIG. 9 shows the amount of change in fatigue. It becomes the calculation flow of. It is assumed that the image of the face of the subject B has been taken before the predetermined period for performing the process of FIG.

In S21, the photographing device 2 captures an image of the face of the subject B. Then, in S22, the image acquisition unit 101 of the information processing apparatus 1 acquires the image taken in S21 and the image of the face of the subject B taken before a predetermined period of the image. The image taken before the predetermined period may be stored in the storage unit 20 or the like in advance.

In S23 (color information acquisition step), the color information acquisition unit 102 acquires the color information of a predetermined portion. This color information is the amount of change in color brightness during the above-mentioned predetermined period and the amount of change in the degree of color unevenness in the above-mentioned predetermined period for a predetermined part of the face reflected in each image taken with the subject B after a predetermined period. This is the information to be shown.

Specifically, the brightness information acquisition unit 1021 included in the color information acquisition unit 102 calculates the arithmetic mean value of the L * value of each pixel included in the predetermined portion and the standard deviation of the L * value. This process is performed on each of the image taken in S21 and the image taken before the predetermined period. Subsequently, the brightness information acquisition unit 1021 has an arithmetic mean value of the L * value calculated from the image taken in S21 and an arithmetic mean value of the L * value calculated from the image taken before a predetermined period of the processing of S21. The difference from, that is, the amount of change in the arithmetic mean value of the L * value is calculated. Further, the brightness information acquisition unit 1021 has a difference between the standard deviation of the L * value calculated from the image taken in S21 and the standard deviation of the L * value calculated from the image taken before a predetermined period of the processing of S21. That is, the amount of change in the standard deviation of the L * value is calculated. Then, the brightness information acquisition unit 1021 stores the calculated change amount of the arithmetic mean value of the L * value and the change amount of the standard deviation of the L * value in the color information storage unit 201 as color information.

In S24 (evaluation value calculation step), the evaluation value calculation unit 104 uses an evaluation model for evaluating the amount of change in skin age, and the amount of change in skin age from the color information stored in the color information storage unit 201. Is calculated. Specifically, the evaluation value calculation unit 104 uses a regression equation in which the amount of change in the arithmetic mean value of the L * value and the amount of change in the standard deviation of the L * value are independent variables as the evaluation model. Then, the evaluation value calculation unit 104 substitutes the change amount of the arithmetic mean value of the L * value stored in the color information storage unit 201 and the change amount of the standard deviation of the L * value into this regression equation, and the skin The amount of change in age is calculated, and the calculated amount of change is stored in the evaluation result storage unit 202.

In S25, the output control unit 106 causes the display device 3 to display and output the evaluation result. For example, an image like the example of FIG. 6 is displayed on the display device 3. As a result, the process of FIG. 9 is completed. When calculating the amount of change in skin age and the amount of change in fatigue, the comparative evaluation unit 105 may perform comparative evaluation in the same manner as in the example of FIG. In this case, the comparative evaluation unit 105 determines the difference or ratio between the amount of change in skin age and the amount of change in fatigue calculated before the previous day and the amount of change in skin age and degree of fatigue calculated on the day. It may be calculated.

[Embodiment 3]
FIG. 10 is a diagram showing a configuration example of the evaluation system 300 according to the present embodiment. The evaluation system 300 includes an information processing device 1 and a mobile terminal 5. In the present embodiment, the target person for calculating the evaluation value is the target person C. The evaluation system 300 is substantially the same as the evaluation system 100 and the evaluation system 200, except that the mobile terminal 5 takes a picture of the subject C and outputs the evaluation value.

The mobile terminal 5 is a portable type having a function of capturing an image of the face of the subject C, a function of transmitting the captured image to the information processing device 1, and a function of outputting an evaluation value calculated by the information processing device 1. It is an information processing device. For example, a smartphone or the like can be made to function as a mobile terminal 5.

In the evaluation system 300, the subject C captures an image of his / her own face using the mobile terminal 5, and the mobile terminal 5 transmits the captured image to the information processing device 1 via a network such as the Internet. The information processing device 1 that has received the image calculates the fatigue degree or the skin age of the subject C by the same processing as in the first embodiment, and notifies the mobile terminal 5 of the calculation result. In addition, if the information processing device 1 previously stores the result of calculating the fatigue degree or the skin age of the subject C, the information processing device 1 also notifies the mobile terminal 5 of the comparison evaluation result between the result and the current result. You may.

As a result, as shown in FIG. 10, the mobile terminal 5 can output information indicating the evaluation result of the facial impression of the subject C and the comparative evaluation result. In the example of FIG. 10, the mobile terminal 5 displays the skin age of the subject C and also displays the difference between the skin age and the average value of the skin age of last month. The output mode is arbitrary, and may be display output, audio output, or a combination thereof.

Further, in the evaluation system 300, the amount of change in skin age and the amount of change in fatigue can be calculated as in the second embodiment. In this case, the subject C captures a plurality of images (those having different shooting time points) captured by the mobile terminal 5, and the mobile terminal 5 transmits the images to the information processing device 1. If there are at least two images at different shooting times, the information processing apparatus 1 can calculate the amount of change in skin age and the amount of change in fatigue by the same processing as in the second embodiment.

Each process of the mobile terminal 5 can also be executed by installing predetermined application software (hereinafter, simply referred to as an application) on the mobile terminal 5. For example, the application may cause the mobile terminal 5 to perform processing such as image shooting control, shooting image transmission processing, and output control of the evaluation result notified from the information processing device 1.

Further, in order for the information processing device 1 to calculate an appropriate evaluation value, the shooting environment (for example, ambient brightness, how the light hits the face) and shooting conditions (for example, shutter speed, shooting distance, flash) It is desirable to match the presence / absence, etc.) with the time when the image used to generate the evaluation model was taken.

Therefore, in the above-mentioned shooting control, the shooting conditions of the shooting device included in the mobile terminal 5 may be set according to the shooting conditions of the image used to generate the evaluation model. For example, in the mobile terminal 5, the shutter speed and the presence / absence of the flash may be the same as when the image used for generating the evaluation model is captured. Further, for example, the mobile terminal 5 may display or output a shooting distance at the time of shooting the image used for generating the evaluation model, and prompt the subject C to shoot at the shooting distance.

Further, in the above-mentioned shooting control, the subject C may be guided to adjust the shooting environment so as to approach the shooting environment at the time of shooting the image used for generating the evaluation model. For example, when the surroundings of the subject C are darker than when the image used for generating the evaluation model is taken, the mobile terminal 5 may display or output a voice recommending taking a picture in a brighter place.

[Example of realization by software]
The control block (particularly, each unit included in the control unit 10) of the information processing device 1 described in each of the above embodiments may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like. , May be realized by software.

In the latter case, the information processing apparatus 1 includes a computer that executes instructions of an information processing program, which is software that realizes each function. The computer includes, for example, one or more processors and a computer-readable recording medium that stores the information processing program. Then, in the computer, the processor reads the information processing program from the recording medium and executes it, thereby achieving the object of the present invention. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, a "non-temporary tangible medium", for example, a ROM (Read Only Memory) or the like, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. Further, a RAM (Random Access Memory) or the like for developing the information processing program may be further provided. Further, the information processing program may be supplied to the computer via an arbitrary transmission medium (communication network, broadcast wave, etc.) capable of transmitting the information processing program. One aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the information processing program is embodied by electronic transmission.

[Modification example]
The execution subject of each process described in the above embodiment can be changed as appropriate. For example, at least one of the blocks of the control unit 10 shown in FIG. 1 may be omitted, and the omitted processing unit may be provided in another device (computer). In this case, a part of the processing executed by the information processing apparatus 1 in the above embodiment is executed by another one or a plurality of apparatus. The other one or more devices may include a mobile terminal 5. For example, the mobile terminal 5 may have the function of the evaluation value calculation unit 104. In this case, the mobile terminal 5 transmits the captured image to the information processing device 1 to acquire the color information, acquires the color information from the information processing device 1, and calculates the evaluation value by the evaluation value calculation unit 104. .. Further, the mobile terminal 5 may also have the function of the color information acquisition unit 102. In this case, the mobile terminal 5 alone can perform from image capture to output of the evaluation value.

Further, in each of the above embodiments, an example of calculating the evaluation value using the regression equation obtained by the multiple regression analysis has been described, but one color information correlating with the degree of fatigue, the skin age, or the amount of change thereof is one. The evaluation value may be calculated using the regression equation obtained by the simple regression analysis used.

In addition, the evaluation model is a model generated based on the evaluation result in which the evaluator evaluates the impression of the face of the evaluated person and the color information of a predetermined portion appearing in the photographed image of the face of the evaluated person. It is sufficient, and it is not limited to the regression equation. For example, an evaluation model is used in which the relationship between the evaluation result in which the evaluator evaluates the impression of the evaluated person's face and the color information of a predetermined portion appearing in the photographed image of the evaluated person's face is modeled by machine learning. You can also do it. In this case, as the color information, the average value or standard deviation of the L * value, a * value, and b * value may be used as in each of the above embodiments, or an image of a predetermined portion (for example, a face is photographed). A region in which a predetermined part is captured in the image) may be used as color information. This is because such an image shows the brightness of the color of a predetermined portion, the degree of color unevenness, the degree of color brightness unevenness, and the like.

Further, in the above embodiment, in the color information, the arithmetic mean value of L * is used as the value indicating the brightness of the color, and the standard deviation of a * and the standard deviation of b * are used as the values indicating the degree of color unevenness. An example of using the standard deviation of L * as a value indicating unevenness of color brightness has been described. However, the color information is not limited to these examples as long as it indicates the brightness of the color, the degree of the color unevenness, the unevenness of the color brightness, and the like. For example, the variance value may be used instead of the standard deviation. Further, for example, instead of the above L * value, a * value, and b * value, the L value, a value, and b value of the Hunter Lab color system may be used. Further, since both the Hunter Lab color space and the L * a * b * color space are based on the coordinates of the CIE XYZ color space, the value of the CIE XYZ may be used as the color information. In addition to this, for example, color information may include information indicating color brightness, degree of color unevenness, color unevenness, etc. by RGB values, CMKY values, sRGB values, or the like.

The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims, and the embodiments obtained by appropriately combining the technical means disclosed in the different embodiments. Is also included in the technical scope of the present invention.

1 Information processing device 102 Color information acquisition unit 104 Evaluation value calculation unit 105 Comparison evaluation unit

Claims (7)

A color information acquisition unit that acquires color information indicating at least one of the brightness and the degree of color unevenness of a predetermined part of the face appearing in an image of the subject of impression evaluation.
Using the evaluation model generated based on the evaluation result in which the evaluator evaluated the fatigue level or the skin age of the evaluated person and the color information of the predetermined part shown in the photographed image of the evaluated person's face. An information processing device including an evaluation value calculation unit that calculates an evaluation value indicating the degree of fatigue or skin age of the subject from the color information acquired by the color information acquisition unit. It is provided with a comparative evaluation unit that generates information indicating the degree of fatigue or the degree of change in skin age of the subject by using a plurality of the above evaluation values calculated based on images taken at different timings. The information processing apparatus according to claim 1. The color information includes the average value of the L * value of the L * a * b * color system in the predetermined part, the standard deviation of the a * value in the predetermined part, and the standard deviation of the b * value in the predetermined part. And at least two are included,
The evaluation value calculation unit according to claim 1 or 2, wherein the evaluation value calculation unit calculates the evaluation value by using a regression equation having at least two values included in the color information as independent variables as the evaluation model. Information processing equipment. The amount of change in color brightness in a predetermined period, the amount of change in the degree of color brightness unevenness in the predetermined period, and the amount of change in the degree of color brightness unevenness in a predetermined part of the face appearing in each image of the subject of impression evaluation after a predetermined period. A color information acquisition unit that acquires color information indicating at least one of
It was generated from each evaluation result obtained by the evaluator evaluating the degree of fatigue or skin age of the evaluated person after the above-mentioned predetermined period, and each image of the evaluated person's face taken after the above-mentioned predetermined period. Using the color information and the evaluation model generated based on the color information, the evaluation indicating the degree of change in the fatigue degree or the skin age of the subject in the predetermined period from the color information acquired by the color information acquisition unit. An information processing device including an evaluation value calculation unit for calculating a value. A method of evaluating a subject's impression performed by one or more computers.
A color information acquisition step for acquiring color information indicating at least one of the brightness and the degree of color unevenness of a predetermined part of the face appearing in the photographed image of the subject, and
Using the evaluation model generated based on the evaluation result in which the evaluator evaluated the fatigue degree or the skin age of the evaluator and the color information of the predetermined part shown in the photographed image of the evaluator's face. , An evaluation method including an evaluation value calculation step of calculating an evaluation value indicating the degree of fatigue or skin age of the subject from the color information acquired in the color information acquisition step. A method of evaluating a subject's impression performed by one or more computers.
The amount of change in color brightness during the above-mentioned predetermined period and the amount of change in the degree of color brightness unevenness in the above-mentioned predetermined period for a predetermined part of the face appearing in each image of the subject taken after a predetermined period of time. A color information acquisition step for acquiring at least one of the color information,
It was generated from each evaluation result obtained by the evaluator evaluating the degree of fatigue or skin age of the evaluated person after the above-mentioned predetermined period, and each image of the evaluated person's face taken after the above-mentioned predetermined period. Using the color information and the evaluation model generated based on the color information, the degree of change in the fatigue degree or the skin age of the subject in the predetermined period is shown from the color information acquired in the color information acquisition step. An evaluation method including an evaluation value calculation step for calculating an evaluation value. An information processing program for operating a computer as the information processing device according to claim 1 or 4, wherein the computer functions as the color information acquisition unit and the evaluation value calculation unit.

Images