JP7828381B2 - Estimation method, estimation device, estimation system, and program - Google Patents

Description

The present invention relates to a method for estimating the state of a coating of a composition when the composition is applied to a user's skin.

There is a system that diagnoses the skin or scalp based on medical interview data and image data of keratin and/or sebum (Patent Document 1).

JP 2018-97899 A

When a user applies a composition, such as a cosmetic, to their skin, if the cosmetic film is uniform, the ingredients contained in the cosmetic will spread evenly over the skin, making it easier for the cosmetic's effects to be fully realized. Therefore, whether the cosmetic film is sufficiently formed is an item of great interest to users. However, while Patent Document 1 was able to diagnose the skin or scalp from interview data and image data, it did not diagnose the condition of the cosmetic film when the cosmetic is applied.

The present invention was made in consideration of the above-mentioned problems, and relates to a method for estimating the state of a coating of a composition that would be formed on the skin if a specific composition were applied, without the need to apply the specific composition to the user's skin.

The present invention relates to an estimation method for acquiring a skin condition index that indicates a user's skin characteristics, and estimating the state of a coating film of a composition when a predetermined composition is applied to the user's skin using a mathematical model that indicates the relationship between the acquired skin condition index and the state of a coating film of the composition when the predetermined composition is applied to the user's skin.

The present invention also relates to an estimation device that includes a skin condition index that indicates a user's skin characteristics, a storage means that stores information indicating the correlation between the skin condition index and the state of a coating of a predetermined composition when the composition is applied to the user's skin, a skin condition acquisition means that acquires the skin condition index, and a coating film estimation means that references the storage means and uses the acquired skin condition index to estimate the state of a coating of a composition when the composition is applied to the user's skin.

The present invention also relates to an application program for operating an estimation device that estimates the state of a coating of a composition when a predetermined composition is applied to a user's skin, the application program including a skin condition acquisition process that acquires a skin condition index that indicates the user's skin characteristics, and a coating estimation process that uses the acquired skin condition index to estimate the state of a coating of the composition on the user's skin.

The present invention also relates to an estimation system that includes a skin condition index that indicates a user's skin characteristics, a storage means that stores information indicating the correlation between the skin condition index and the state of a coating of a predetermined composition when the composition is applied to the user's skin, a skin condition acquisition means that acquires the skin condition index, and a coating film estimation means that references the storage means and uses the acquired skin condition index to estimate the state of a coating of the composition when the composition is applied to the user's skin.

The method provided by the present invention makes it possible to estimate the state of a coating of a composition when a specific composition is applied to a user's skin.

FIG. 10 is an image diagram illustrating the process of acquiring a skin condition index according to the first embodiment. 10A and 10B are conceptual diagrams illustrating the process of acquiring the reflectance of skin to which a predetermined cosmetic material has been applied in the first and second embodiments. (a) is a face image of bare skin taken in FIG. 2-1, and (b) is a face image of a face with cosmetics applied taken in FIG. 2-1. (a) is an image of bare skin irradiated with 320 nm wavelength in the UVB region, and (b) is an image of skin to which a cosmetic has been applied irradiated with 320 nm wavelength in the UVB region. 1A shows the results of predicting whether the quality of the coating state of a cosmetic material will fall into the low or high group using the coating film estimation mathematical model 1, and FIG. 1B shows the accuracy rate of the predicted group and the actual group using the coating film estimation mathematical model 1. 10A and 10B are conceptual diagrams illustrating the acquisition of a skin image (bare skin image) in the second and third embodiments. 10A shows the results of predicting whether the quality of the coating state of a cosmetic material will fall into the low or high group using the coating film estimation mathematical model 2, and FIG. 10B shows the accuracy rate of the predicted group and the actual group using the coating film estimation mathematical model 2. 11 is an image diagram illustrating the process of obtaining an evaluation value for a comprehensive evaluation of skin to which a predetermined cosmetic material has been applied in embodiment 3. FIG. 10A shows the results of predicting whether the quality of the coating state of a cosmetic material will fall into the low or high group using the coating film estimation mathematical model 3, and FIG. 10B shows the accuracy rate of the predicted group and the actual group using the coating film estimation mathematical model 3. This is a diagram for estimating the effects of cosmetics 1 to 3 when they are applied. FIG. 2 is a block diagram of an estimation device. FIG. 2 is a block diagram of an estimation device. FIG. 1 is a block diagram of an estimation system. FIG. 1 is a block diagram of an estimation system. (a-1) to (a-3) are examples of display modes of skin information classification images, (b-1) to (b-3) are examples of display modes of composition classification images, (c-1) is an example of display modes of skin information classification images and composition classification images, and (d-1) and (d-2) are examples of display modes of skin information classification images and composition classification images. FIG. 1 is a graph showing the results of a comparative test of the UV protection effects of cosmetics C to E. FIG. 10 is a graph showing the results of another comparative test of the UV protection effects of cosmetics C to E. 10 is a flowchart showing the flow of a process for presenting skin type characteristic information and cosmetic score information based on a skin image performed by an estimation device or estimation system. FIG. 10 is a diagram showing an example of skin type characteristic information generated and stored by the estimation device or the estimation system. FIG. 10 is a diagram showing an example of score information regarding the UV protection effect of cosmetics for each skin type, which is generated and stored by the estimation device or estimation system. 1A is a diagram showing an example of skin type characteristic information that an estimation device or estimation system displays on a user's information processing terminal, and FIG. 1B is a diagram showing an example of cosmetic score information that an estimation device or estimation system displays on a user's information processing terminal.

Below, preferred embodiments of the present invention will be described with reference to the drawings. Note that the drawings of the present embodiments are intended to explain the technical concept, configuration, and operation of the present invention, and do not specifically limit the configuration. Furthermore, in all drawings, similar components are designated by similar reference numerals, and duplicate explanations will be omitted where appropriate.

An outline of the estimation method in this embodiment (hereinafter sometimes referred to as the present method) will be described.
The estimation method of this embodiment is a method of acquiring a skin condition index that indicates a user's skin characteristics, and estimating the state of a composition coating film formed when a specified composition is applied to the user's skin using a mathematical model that indicates the relationship between the acquired skin condition index and the state of a composition coating film formed when a specified composition is applied to the user's skin.
The "skin condition index indicating the user's skin characteristics" refers to an index indicating the skin characteristics of bare skin, i.e., skin to which a predetermined composition or predetermined cosmetic product has not been applied, or skin after face washing. Here, "skin characteristics" refer to the nature of skin, such as skin types such as dry skin, normal skin, oily skin, and combination skin. The "skin condition index" is an index indicating these skin characteristics. Examples of such indices include stratum corneum moisture content, moisture evaporation rate, skin viscoelasticity, softness (e.g., skin flexibility), contact angle, color, melanin content, melanin density, skin temperature, moisture content, moisture/oil balance, surface shape, texture, wrinkles, acne, and moles. However, the index also includes skin impressions indicating the skin condition, such as skin age, makeup feel, masculinity/femininity, makeup smudge rate, and powdery feel. In addition to the commonly known skin types described above, skin types can also be classified as appropriate based on the above skin condition indexes. Furthermore, skin types are classified based on predetermined elements included in the skin condition index (e.g., three elements: "texture,""moisture/oilbalance," and "softness"). Any method for classifying skin types may be used. For example, conditions for classification into each skin type may be predetermined, such as when the value of a predetermined element among the three elements is equal to or greater than a predetermined value, when all elements are equal to or greater than a predetermined value, or when the value of a predetermined element is equal to or greater than a predetermined value and the values of the other elements are equal to or less than a predetermined value. Alternatively, for example, five clusters may be created by cluster analysis based on the values of the three elements: "texture,""moisture/oilbalance," and "softness." Each cluster may be associated with a corresponding skin type, and the values of the three elements included in the user's skin condition index may be analyzed to determine which cluster the user belongs to, and the skin may be classified into the corresponding cluster. Note that the above three elements are merely examples, and the types and number of elements are not limited to these.
"User's skin" refers to the soft tissue on the surface of the living body, regardless of the part of the body, such as the face, arms, or hands.
The "predetermined composition" refers to a composition intended to be applied to the skin, and broadly encompasses, for example, cosmetics known as lotions, toners, creams, emulsions, etc., as well as quasi-drugs and pharmaceuticals. Specific examples include, but are not limited to, skin care cosmetics such as lotions, toners, emulsions, creams, beauty serums, massage packs, and lip balms; makeup cosmetics such as foundations, makeup bases, liquid foundations, oil-based foundations, powder foundations, concealers, color control, eye shadows, blushers, lipsticks, lip glosses, lip liners, and body and décolleté products; UV protection cosmetics such as sunscreen emulsions, sunscreen gels, and sunscreen creams; bath cosmetics such as bath oils and bath additives; and ointments containing antibacterial ingredients, anti-itching ingredients, and anti-inflammatory ingredients.
A "film of a composition" is a film formed on the skin when a specific composition is applied to the skin. Details will be described later, but even when the same amount of the same composition is applied, the state of the film formed by the composition will differ depending on the condition of the skin to which it is applied, and even when the same amount of the composition is applied to the same skin, the state of the film formed by the composition will differ depending on the type (ingredients) of the composition and the amount applied.
The "state of formation of a coating film" refers to how a coating film of a composition is formed on the skin, and is evaluated by the thickness of the coating film formed, the unevenness of the thickness of the coating film formed, and the uniformity of the coating film formed. For example, when the composition to be applied contains a color, it refers to a state in which a coating film of the composition is formed without unevenness in color; when the composition to be applied contains particles, it refers to a state in which a coating film of the composition is formed with uniform particles; when the composition to be applied contains an ultraviolet absorber, it refers to a state in which ultraviolet rays are absorbed regardless of the position of the coating film formed; when the composition to be applied contains an ultraviolet scattering agent, it refers to a state in which ultraviolet rays are reflected regardless of the position of the coating film formed; etc.
"Estimating" means estimating the state of a coating film of a composition that would be formed if a specified composition were applied without applying the specified composition to the user's skin, and the estimated results can be output in any form, such as numerical output, graphical output, or simulation image output.
The "skin information classification image" is any image showing skin types classified based on the skin condition index.
A "composition classification image" is any image showing a composition classification based on the characteristics of the composition. The "characteristics of the composition" are not particularly limited as long as they are factors related to the state of formation of a coating film of the composition, and examples thereof include the formulation type of the composition, physical properties such as wettability, viscosity, and color.
The skin information classification images and composition classification images are each created based on shape, pattern, color, or shade, or a combination of these, and the estimation results can be output by combining these images, such as by overlaying them.

When creating the coating film mathematical model prepared in this embodiment, the specific composition applied to the subject's skin was a cosmetic containing at least one of a component that absorbs wavelengths in the ultraviolet range (hereinafter sometimes referred to as an "UV absorber") and a component that scatters wavelengths in the ultraviolet range (hereinafter sometimes referred to as an "UV scattering agent"). Here, "added cosmetic" includes not only a cosmetic containing at least one of an UV absorber or an UV scattering agent already added (contains these components), but also a cosmetic containing no such component but to which a predetermined proportion of the component has been added. This makes it possible to estimate the state of a cosmetic coating film formed when a cosmetic containing no components that absorb and/or scatter wavelengths in the ultraviolet range is applied to the user's skin.
In this embodiment, for example, a cosmetic product was used in which an ultraviolet absorber or an ultraviolet scattering agent was added to a W/O cosmetic product (water-in-oil cosmetic product) or an O/W cosmetic product (oil-in-water cosmetic product). Therefore, by using the estimation results of the state of a coating film formed on the skin of a user when a cosmetic product containing an ultraviolet absorber or an ultraviolet scattering agent is applied to the user's skin, it is possible to estimate the ultraviolet protection properties of the cosmetic product. Note that when creating a coating film mathematical model, the cosmetic product may contain only an ultraviolet absorber, only an ultraviolet scattering agent, or both an ultraviolet scattering agent and an ultraviolet absorber. In all these cases, the ultraviolet protection properties of the cosmetic product can be estimated based on the estimation results.
Some cosmetics containing added ultraviolet absorbers or ultraviolet scattering agents have a clearly indicated index indicating their ultraviolet protection properties (previously published published ultraviolet protection values). The most widely known index indicating ultraviolet protection properties is the Sun Protection Factor (SPF) for ultraviolet rays in the UVB region, and the ultraviolet protection effect is expressed as an SPF value (e.g., "SPF 30"). For ultraviolet rays in the UVA region, PFA (Protection Factor of UVA) or UVAPF (UVA Protection factor of product) is used. When evaluating the ultraviolet protection properties of cosmetics, evaluation may be performed using clearly indicated indices indicating ultraviolet protection properties.
In this embodiment, the predetermined composition for which the coating film formation state is estimated is a cosmetic containing at least one of a component that absorbs wavelengths in the ultraviolet region and a component that scatters wavelengths in the ultraviolet region, and the estimated coating film formation state of the composition is described as the coating film formation state of the cosmetic. Furthermore, this description explains how to estimate whether the cosmetic can exhibit its intended functions (for example, whether the desired UV protection function can be expected) based on the estimated coating film formation state of the cosmetic.

<Embodiment 1>
1 , the first embodiment is a method for measuring (obtaining) a skin condition index on a user's skin (bare skin) using a predetermined measuring device 10, and estimating the state of a coating film that would be formed on the skin by a predetermined cosmetic if the cosmetic were applied, based on the measured skin condition index. The state of a coating film of the cosmetic is estimated based on the measured skin condition index and a mathematical model created in advance (the mathematical model used for coating film estimation in the first embodiment is referred to as "coating film estimation mathematical model 1"). In the first embodiment, the state of a coating film that would be formed on the skin if cosmetic A (SPF 50) and cosmetic B (SPF 50+) were applied as the predetermined cosmetic is estimated.
The skin condition index acquired in the first embodiment is a physical property value indicating the skin condition measured on the user's skin using a predetermined measuring device 10. The physical property value is, for example, the stratum corneum moisture content, the amount of water evaporation, skin viscoelasticity, skin flexibility, contact angle, etc.

Figure 1 shows how the amount of water evaporation is measured by placing a contactor on the skin. The skin condition indicators obtained are as follows:
- Stratum corneum moisture content: Values measured using a stratum corneum moisture meter (Corneometer, manufactured by Courage+Khazaka) are used. - Water evaporation rate: Values measured using a water evaporation meter (Tewameter, manufactured by Courage+Khazaka) are used. - Skin viscoelasticity: Values measured using a skin viscoelasticity measuring device (Cutometer, manufactured by Courage+Khazaka) are used. - Skin flexibility: Values measured using a skin viscoelasticity measuring device (Cutometer, manufactured by Courage+Khazaka) are used. - Contact angle: Values measured using a contact angle meter (Contact angle meter PCA-11, manufactured by Kyowa Interface Science Co., Ltd.) are used. Note that the above-mentioned measuring devices are merely examples, and values measured using other measuring devices or values calculated by photographing the skin and processing the images may also be used.
Thus, in embodiment 1, the skin condition index includes at least one evaluation item of stratum corneum moisture content, moisture evaporation rate, skin viscoelasticity, skin flexibility, and contact angle measured by a specified measuring device.

Next, the paint film estimation mathematical model 1 will be described.
The training data and conditions used to create the coating film estimation mathematical model 1 are as follows:
Prescribed composition 20: Cosmetic A (SPF 50, formulation W/O)
Cosmetic B (SPF50+, formulation W/O)
・Application area: 4cm x 4cm
Amount applied: 16 mg per 4 cm x 4 cm
・Target subjects: Cosmetic A was applied to the forehead or cheeks of 19 men and women.
Cosmetic B was applied to the forehead or cheek of 20 men and women. Camera used: UV camera 30
Application method: Apply evenly by a trained tester. Light source: MAX-303, manufactured by Asahi Spectroscopy. Bandpass filter (320 nm ± 5 nm)
As shown in Figure 2-1, a predetermined composition 20 is applied to the skin of a subject, and image data (ultraviolet skin image) of the subject's skin taken under ultraviolet irradiation with an ultraviolet camera 30 is used. A coating film estimation mathematical model 1 is created from the taken image data using the following procedure.

1) Calculating the UV reflectance of the coating film of each cosmetic formed on the skin When a predetermined composition 20 (cosmetic A) is applied to the skin (skin 15) of a subject, the UV reflectance of the coating film formed on the skin by cosmetic A (referred to as "UV reflectance A") is calculated, and when a predetermined composition 20 (cosmetic B) is applied to the skin of a subject, the UV reflectance of the coating film formed on the skin by cosmetic B (referred to as "UV reflectance B") is calculated. The calculation method is shown below.
Figure 3(a) shows the case where ultraviolet light with a wavelength of 320 nm (hereinafter referred to as "ultraviolet light") included in the UVB region is irradiated onto the skin (bare skin) of a subject. In the figure, "incident light" is the irradiated ultraviolet light, "specularly reflected light" is the ultraviolet light reflected from the surface of the skin, and "internal scattered light" is the ultraviolet light that is not reflected from the surface of the skin, penetrates the interior of the skin, spreads within the skin, and emerges from the skin. The intensity of the internal scattered light from which the specular reflected light has been removed is obtained. This is the internal scattered light intensity of bare skin.
3(b) shows the case where a predetermined composition 20 is applied to the skin of a subject and ultraviolet light is irradiated. In the figure, "incident light" refers to the irradiated ultraviolet light, "specularly reflected light" refers to the ultraviolet light reflected from the surface of the predetermined composition 20 and the ultraviolet light reflected from the surface of the skin, and "internal scattered light" refers to the ultraviolet light that is not reflected from the surface of the predetermined composition 20 and the surface of the skin, penetrates the interior of the skin, spreads within the skin, and emerges from the skin. The intensities of internal scattered light from which specular reflected light is removed when cosmetic A is applied as the predetermined composition 20, and internal scattered light from which specular reflected light is removed when cosmetic B is applied as the predetermined composition 20, are obtained. The internal scattered light intensity of the applied skin when cosmetic A is applied is designated A, and the internal scattered light intensity of the applied skin when cosmetic B is applied is designated B.
Then, the internal scattered light intensity of the applied skin when the specified composition 20 is applied is divided by the internal scattered light intensity of the bare skin to calculate the ultraviolet reflectance of the composition present at the site where the specified composition 20 has been applied (internal scattered light intensity A of the applied skin when cosmetic A is applied/internal scattered light intensity of the bare skin = ultraviolet reflectance of the coating film formed by cosmetic A present at the site where cosmetic A has been applied (ultraviolet reflectance A); internal scattered light intensity B of the applied skin when cosmetic B is applied/internal scattered light intensity of the bare skin = ultraviolet reflectance of the coating film formed by cosmetic B present at the site where cosmetic B has been applied (ultraviolet reflectance B)).
In this way, by removing specularly reflected light and calculating UV reflectance using the intensity of internally scattered light, it is possible to accurately estimate the application state of the coating film formed on the skin (the formation state of the coating film).
Here, polarizing plates are attached to the light source and the ultraviolet camera, ultraviolet light is irradiated onto the skin to which cosmetic A (or cosmetic B) has been applied, the specularly reflected light is removed, and the reflected internally scattered light is photographed with the ultraviolet camera 30, thereby making it possible to measure the internal scattering intensity.
In this embodiment, the reflectance of ultraviolet light at a wavelength of 320 nm is calculated taking into consideration that the wavelength range of the UVB region is 320 to 280 nm. However, the wavelength range is not limited to this. For example, when estimating the state of a coating film formed on the skin by a cosmetic product and estimating the ultraviolet protection effect in the UVA region from the estimation result, the reflectance of ultraviolet light may be calculated using ultraviolet light with a wavelength of 380 to 320 nm, for example, ultraviolet light with a wavelength of 350 nm.
2) Dividing into two groups according to UV reflectance The calculated UV reflectance of the coating film formed on the skin by each cosmetic was sorted in order of UV reflectance, and each subject was divided into two groups: those with high UV reflectance (50%) and those with low UV reflectance (50%). The group with high UV reflectance was designated the "group with low quality of cosmetic coating film formation," and the group with low UV reflectance was designated the "group with high quality of cosmetic coating film formation." Generally, when UV reflectance is high, the cosmetic coating film is thin, and when UV reflectance is low, the cosmetic coating film is thick. This difference occurs when the area to which the cosmetic is applied and the amount of cosmetic applied are constant, as in this embodiment. For example, if there are depressions on the skin surface to which the cosmetic is applied, such as wrinkles or acne scars, the cosmetic will be applied in a way that fills the depressions, resulting in a thin cosmetic coating film formed on the skin. This is because the state of the cosmetic coating film formation is affected by the condition of the skin to which it is applied. Therefore, the UV reflectance of the coating film formed on the skin by each cosmetic calculated for each subject will differ (they will be ranked). Note that although the shape of the skin surface has been described as a factor that affects the state of formation of a cosmetic coating, this is merely one example, and other skin conditions and differences in skin physical properties are also factors that affect the state of formation of a cosmetic coating.
3) Measuring Skin Condition Indicators of the Subject The skin condition indices described above (stratum corneum moisture content, water evaporation rate, skin viscoelasticity, skin flexibility, and contact angle) are measured for the subject using a predetermined measuring device.
4) Create training data to be used for machine learning. Using the results of dividing into two groups in 2) and the measurement results in 3), create training data to create a paint film estimation mathematical model 1.
5) Creating a coating film estimation mathematical model 1 A standardization process is performed on each evaluation value, and the most suitable model is selected from multiple classification models (e.g., naive Bayes (NB), logistic regression (LR), linear discriminant analysis (LDA), random forest (RF), gradient boosting (GB), support vector machine (SVM), Gaussian process (GP)) to create a prediction model. The prediction model is used to predict whether the quality of the coating film formation state of the cosmetic falls into a low or high group. The degree of agreement between the predicted group and the actual group is calculated as the accuracy rate. The generalization performance of the classification models was evaluated using leave-one-out cross-validation (LOOCV), and the calculation model with the highest accuracy rate was designated as the optimal model: coating film estimation mathematical model 1. In this embodiment, classification model GB was used as coating film estimation mathematical model 1 for cosmetic A, and classification model LDA was used as coating film estimation mathematical model 1 for cosmetic B.

FIG. 4( a ) shows the results of predictions made using the Coating Film Estimation Mathematical Model 1 to determine whether the quality of the coating film formed by a cosmetic on the skin falls into the low or high group, as well as the actual results. FIG. 4( b ) shows the results of calculations of the accuracy rate, which is the degree of agreement between the group predicted using the Coating Film Estimation Mathematical Model 1 and the actual group.
4(a) and 4(b) show that when the film formation state of a cosmetic material formed on the skin is predicted using the film formation estimation mathematical model 1, a high accuracy rate can be obtained.
Therefore, by using the user's skin condition index obtained using a specified measuring device and the coating film estimation mathematical model 1, it is possible to estimate the coating film formation state of a specified cosmetic without applying the cosmetic to the skin.As a result, even without applying the specified cosmetic (cosmetics containing an ultraviolet absorber or an ultraviolet scattering agent) to the skin, it is possible to estimate the coating film formation state of the cosmetic that will be formed on the skin when the specified cosmetic is applied to the user's skin (the quality of the coating film formed by the cosmetic on the skin), and this makes it possible to evaluate the extent to which an effect based on the function possessed by the specified cosmetic (for example, the ultraviolet protection function in the case of cosmetics containing an ultraviolet absorber or an ultraviolet scattering agent) can be expected.

As described above, in embodiment 1, a mathematical model (film estimation mathematical model 1) is created that shows the relationship between the skin condition indexes and the UV reflectance of the cosmetic film, using skin condition indexes (e.g., stratum corneum moisture content, water evaporation rate, skin viscoelasticity, skin flexibility, and contact angle measured by measuring device 10) for each of multiple subjects (e.g., 19 to 20 men and women) and the UV reflectance of the cosmetic film formed on the skin calculated from UV skin images of the multiple subjects (e.g., images shown in FIG. 2-2 taken with the imaging device shown in FIG. 2-1). Using the created film estimation mathematical model 1 and the user's skin condition indexes (e.g., stratum corneum moisture content, water evaporation rate, skin viscoelasticity, skin flexibility, and contact angle) measured by measuring device 10, it is possible to estimate the state of the cosmetic film formed on the user's skin when the cosmetic is applied, thereby enabling an estimated evaluation of the functions of the cosmetic.

Furthermore, because the state of the coating film formed by a cosmetic on the skin was estimated using the reflectance of ultraviolet light with a wavelength of 320 nm in the UVB wavelength range, it is possible to directly estimate and evaluate the functionality of the cosmetic (for cosmetics with UVB protection functionality, the UVB protection functionality) based on the state of the coating film formed by the cosmetic on the skin. Specifically, for example, in the case of cosmetic A (SPF 50), the UV reflectance of each of multiple subjects (19 people) was calculated, and a standard value for the calculated UV reflectance was determined. In this embodiment, the "standard value" refers to the UV reflectance that is located in the middle when the UV reflectances are sorted in order of UV reflectance, as described in "2) Dividing into two groups according to UV reflectance." If the state of the film formed by the cosmetic on the subject's skin (ultraviolet reflectance) estimated using the above method is a standard value, then the claimed effect of cosmetic A (equivalent to SPF 50) can be expected; if it is higher than the standard value, then the claimed effect of cosmetic A (equivalent to SPF 50) can be expected to be sufficient; and if it is lower than the standard value, then the claimed effect of cosmetic A (equivalent to SPF 50) cannot be expected to be very good. In this way, the user can determine whether or not a cosmetic is suitable for their skin based on the estimated state of the film formed by the cosmetic on the skin.

Generally, over time after a given cosmetic is applied to the skin, the state of the coating film that the cosmetic forms on the skin changes. This is due to various factors, such as external factors such as humidity, temperature, and wind, factors related to the skin characteristics of the subject (user), and the subject (user)'s constitution (whether or not they sweat easily, whether or not they move their skin surface a lot (in the case of the face, whether or not they change their facial expressions a lot)).
Therefore, the coating film estimation mathematical model 1 may be configured to estimate the formation state of a coating film formed on the skin by the cosmetic in a first time period after applying a specified cosmetic to the user's skin (hereinafter referred to as the "first formation state"), and the formation state of a coating film formed on the skin by the cosmetic in a second time period after the first time period (hereinafter referred to as the "second formation state").

A method for estimating the first formation state and the second formation state will be described.
Here, the first time period is, for example, one minute after application, and the second time period is, for example, eight hours after application. Then, the coating film estimation mathematical model 1 can be created for each time period. That is, the above-described step "1) Calculating the UV reflectance of the coating film of each cosmetic formed on the skin" can be performed by calculating the UV reflectance of each cosmetic one minute after application of a predetermined cosmetic to the skin of multiple subjects and eight hours after application of a predetermined cosmetic to the skin of multiple subjects, and then performing the above-described steps "2) Dividing into two groups according to UV reflectance" to "5) Creating the coating film estimation mathematical model 1." This allows for the creation of a coating film estimation mathematical model 1 that estimates the formation state of a coating film formed on the skin by the cosmetic one minute after application (first formation state), and a coating film estimation mathematical model 1 that estimates the formation state of a coating film formed on the skin by the cosmetic eight hours after application (second formation state).
In this way, a coating film estimation mathematical model 1 is created that estimates the formation state of a coating film formed on the skin by a cosmetic one minute after application (first formation state), and a coating film estimation mathematical model 1 is created that estimates the formation state of a coating film formed on the skin by a cosmetic eight hours after application (second formation state).By using the two created coating film estimation mathematical models 1 and a user's skin condition index obtained using a specified measuring device, it is possible to estimate the formation state of a coating film formed on the skin by a cosmetic a specified time after application, without applying the specified cosmetic to the user's skin and even before a specified time has passed since application (in this embodiment, first time period: 1 minute or second time period: 8 hours).
In this way, it is possible to estimate the state of a coating film formed on the skin by a given composition (e.g., cosmetic A, cosmetic B) in a first time period (e.g., 1 minute after application) (first formation state) and the state of a coating film formed on the skin by the composition in a second time period (e.g., 8 hours after application) after the first time period (second formation state). It is not necessary to estimate both the first formation state and the second formation state; only one of them may be estimated depending on the purpose. It is also possible to add a third time period, a fourth time period, etc.

Furthermore, by using the first formation state and the second formation state, it is possible to estimate the effects that can be expected in the first time period and the second time period, respectively, in relation to the effects that the applied composition is seeking.
In this way, the specified cosmetic is a cosmetic (e.g., cosmetic A, cosmetic B) to which an ultraviolet absorber or an ultraviolet scattering agent has been added, and it is possible to estimate the first and second formation states and, based on the estimated first and second formation states, estimate and evaluate the functions exerted by the cosmetic (evaluate the extent to which the effect can be expected from the functions possessed by the cosmetic).
Therefore, for example, if one wishes to estimate and evaluate the UV protection effect when a cosmetic is applied before going out and then the person is active in a situation where they are likely to be exposed to UV rays eight hours later, it is sufficient to estimate the state of the film that the cosmetic forms on the skin in the second time period. On the other hand, if one wishes to evaluate the UV protection effect when a cosmetic is applied before going out and then the person is active in a situation where they are likely to be continuously exposed to UV rays for eight hours thereafter, it is sufficient to estimate the state of the film that the cosmetic forms on the skin in both the first time period and the second time period, and use both to estimate and evaluate. In this way, it is possible to use the coating film estimation mathematical model 1 in a way that suits the purpose.

<Embodiment 2>
The second embodiment is a method for estimating the state of a coating film formed on the skin when a specified composition is applied, based on a skin condition index generated using a mathematical model from an image of the user's skin (bare skin) and a mathematical model showing the relationship between this skin condition index and the ultraviolet reflectance when a specified composition is applied to the skin.
In this embodiment, the mathematical model used to generate the skin condition index is referred to as the "skin condition index mathematical model." Furthermore, the mathematical model showing the relationship between the skin condition index obtained in embodiment 2 and the UV reflectance of a coating film of a predetermined composition when the composition is applied to the skin is referred to as the "coating film estimation mathematical model 2." In embodiment 2, the "skin condition index mathematical model" and the "coating film estimation mathematical model 2" are used to estimate the state of a coating film formed on the skin by a composition without applying the composition to the user's skin. In embodiment 2, the state of a coating film formed on the skin when cosmetic C (SPF50+, formulation O/W) is applied as the predetermined composition is estimated using two mathematical models, the "skin condition index mathematical model" and the "coating film estimation mathematical model 2."
The skin condition index mathematical model and paint film estimation mathematical model 2 will be described.

The "skin condition index mathematical model" is a trained discrimination model that has been trained based on multiple training data, and is a discrimination model that has been trained to obtain at least the following seven skin condition indexes:
・Overall evaluation (overall skin condition)
Skin color (whiteness of skin)
・Makeup feel (does it look like makeup on the skin?)
・Skin age (an index showing the degree of skin aging)
-Masculinity/femininity (whether the skin texture is closer to male or female)
・Makeup smudge level (an index showing the degree of makeup smudge)
・Powderiness (an index indicating the smoothness of the skin)
These seven skin condition indices are acquired as the skin condition indices in embodiment 2. As shown in Fig. 5 , these skin condition indices can be acquired by photographing the user's skin (bare skin) with an imaging device 40 and based on the captured skin image (bare skin image) and a skin condition index mathematical model. The imaging device 40 used in this embodiment may be any type, and may be an independent digital camera, a video camera, or an imaging device included in a mobile phone, tablet terminal, or personal computer, which are generally referred to as a mobile terminal. However, in order to determine the appearance or attributes of the user's skin image (bare skin image), it is preferable that the imaging device 40 be a camera capable of capturing color images using visible light.
In this way, the skin condition index is an evaluation item (the above seven items) obtained using a skin image (bare skin image) captured of the user's skin and a skin condition index mathematical model that shows the relationship between the skin condition index.

Here, a method for creating a mathematical model for skin condition indicators will be described.
The training data used to create the skin condition index mathematical model is a combination of facial images of multiple people with bare skin and facial images with a specified cosmetic applied, along with correct answer information indicating the skin condition of the human faces reflected in each facial image.
The facial image may include the subject's face to an extent that allows for analysis of the skin condition of the estimation item, and may include all or part of the subject's face. The acquired facial image may include other parts of the subject's face, such as hair or neck, or the background. Furthermore, the facial image may be a color image or a grayscale image, as long as it is possible to estimate the skin condition of the human face included in the facial image. Furthermore, the facial image may be normalized to a predetermined rectangular shape, or may include a facial image of an image size or shape different from the predetermined image size of the predetermined rectangular shape.
The predetermined cosmetic material may be any material, and may or may not contain an ultraviolet absorber or an ultraviolet scattering agent. In this embodiment, a foundation is used.
The correct answer information is information indicating the skin condition of a human face obtained by a professional evaluator visually evaluating the corresponding face image or the subject of the face image itself using a measuring instrument, etc. In this embodiment, information on the above skin condition indexes (seven items) obtained by visually evaluating face images by five professional evaluators was used.
- Overall evaluation (overall skin condition) Overall skin condition was rated on a 7-point scale. - Skin color (skin whiteness) Skin whiteness was rated on a 7-point scale. - Make-up appearance (whether or not the skin looks like it has been made up). Evaluation of whether or not the skin looks like it has been made up. - Skin age (index showing the degree of skin aging) The degree of skin aging was rated on a 5-point scale. - Masculinity/femininity (whether the skin texture is closer to that of a man or woman). Evaluation of whether the skin texture is closer to that of a man or woman. - Degree of makeup smudged (index showing the degree of makeup smudged). Evaluation of whether or not the makeup has smudged. - Powdery feel (index showing the smoothness of the skin). Evaluation of whether or not the skin feels smooth. In this embodiment, the above 7 items were evaluated for both facial images of bare skin and facial images of faces with a specified cosmetic applied.
Next, for each acquired face image, the position coordinates of a plurality of predetermined locations (predetermined position coordinates) in the face region of the human face captured in the target face image are identified. The predetermined position coordinates include at least the position coordinates of the shape feature points of the face region.
Next, the position of each pixel of the target facial image is converted so that the facial area has a predetermined rectangular shape based on the identified multiple predetermined position coordinates, and the position of each pixel of the original facial image corresponds to the input rules of the skin condition index mathematical model.
Then, a mathematical model for skin condition indicators is created using training data including the converted facial images. Although the learning algorithm is not limited, in this embodiment, deep learning is used.
As described above, the "skin condition index mathematical model" used in this embodiment was created using both a face image of bare skin and a face image with a specified cosmetic applied, and the evaluation results obtained by evaluating the above seven items for each of the images.
In this embodiment, the above-mentioned skin condition indices (seven items) are indices suitable for estimating the state of a coating film formed on the skin by a cosmetic product when it is assumed that the cosmetic product contains an ultraviolet absorber or an ultraviolet scattering agent has been applied, and therefore a skin condition index mathematical model for calculating these indices was used. However, when estimating the state of a coating film formed on the skin by a cosmetic product when it is assumed that the cosmetic product has been applied other than a cosmetic product containing an ultraviolet absorber or an ultraviolet scattering agent, it is sufficient to use a skin condition index mathematical model that can calculate a skin condition indices suitable for estimating the state of a coating film formed on the skin by the cosmetic product.
Furthermore, the "skin condition index mathematical model" may be either or both of a skin condition index mathematical model (bare skin) created using a bare skin face image and evaluation results obtained by evaluating the bare skin face image for the seven items, or a skin condition index mathematical model (application) created using a face image with a predetermined cosmetic applied and evaluation results obtained by evaluating the bare skin face image for the seven items, and the skin condition index (seven items) may be obtained using either or both. However, in the second embodiment, because the user's skin image (bare skin image) is used when obtaining the user's skin condition index, it is preferable to include at least a skin condition index mathematical model (bare skin) including a bare skin face image and evaluation results obtained by evaluating the bare skin face image for the seven items.

Next, the paint film estimation mathematical model 2 will be described.
The training data and conditions used to create the coating film estimation mathematical model 2 are as follows:
Prescribed composition 20: Cosmetic C (SPF 50+, formulation O/W)
・Application area: 4cm x 4cm
Amount applied: 16 mg per 4 cm x 4 cm
・Target subjects: forehead or cheek of 20 women ・Camera used: UV camera 30
Application method: Apply evenly by a trained tester. Light source: MAX-303, manufactured by Asahi Spectroscopy. Bandpass filter (320 nm ± 5 nm)
Polarizing Plate As shown in FIG. 2-1 , a predetermined composition 20 is applied to the skin of a subject, and image data (ultraviolet skin image) of the subject's skin taken with an ultraviolet camera 30 under ultraviolet irradiation is used. From the taken image data, a coating film estimation mathematical model 2 is created using the following procedure. The method for obtaining the ultraviolet reflectance of the predetermined composition 20 (cosmetic C) is the same as in embodiment 1.

1) Calculating the UV reflectance of a coating film of a cosmetic formed on the skin When a predetermined composition 20 (cosmetic C) is applied to the skin (skin 15) of a subject, the UV reflectance of the coating film formed by cosmetic C on the skin (referred to as "UV reflectance C") is calculated. The calculation method is shown below.
Figure 3(a) shows the case where ultraviolet light with a wavelength of 320 nm (hereinafter referred to as "ultraviolet light") included in the UVB region is irradiated onto the skin (bare skin) of a subject. In the figure, "incident light" is the irradiated ultraviolet light, "specularly reflected light" is the ultraviolet light reflected from the surface of the skin, and "internal scattered light" is the ultraviolet light that is not reflected from the surface of the skin, penetrates the interior of the skin, spreads within the skin, and emerges from the skin. The intensity of the internal scattered light from which the specular reflected light has been removed is obtained. This is the internal scattered light intensity of bare skin.
3(b) shows the case where a predetermined composition 20 was applied to the skin of a subject and ultraviolet light was irradiated. In the figure, "incident light" is the irradiated ultraviolet light, "specularly reflected light" is the ultraviolet light reflected from the surface of the predetermined composition 20 and the surface of the skin, and "internal scattered light" is the ultraviolet light that is not reflected from the surface of the predetermined composition 20 and the surface of the skin, penetrates the inside of the skin, spreads inside the skin, and emerges from the skin. The intensity of internal scattered light from which specular reflected light has been removed is obtained when cosmetic C is applied as the predetermined composition 20. This is called the internal scattered light intensity C of the applied skin.
Then, the internal scattered light intensity C of the applied skin when the specified composition 20 is applied is divided by the internal scattered light intensity of bare skin to calculate the ultraviolet reflectance of the composition present at the site where the specified composition 20 has been applied (internal scattered light intensity C of the applied skin when cosmetic C is applied / internal scattered light intensity of bare skin = ultraviolet reflectance of the coating film formed by cosmetic C present at the site where cosmetic C has been applied (ultraviolet reflectance C)).
In this way, by removing specularly reflected light and calculating UV reflectance using the intensity of internally scattered light, it is possible to accurately estimate the application state of the coating film formed on the skin (the formation state of the coating film).
Here, by attaching polarizing plates to the light source and the ultraviolet camera, ultraviolet light is irradiated onto the skin to which cosmetic C has been applied, the specularly reflected light is removed, and the reflected internally scattered light is photographed with the ultraviolet camera 30, thereby making it possible to measure the internal scattering intensity.
In this embodiment, the reflectance of ultraviolet light at a wavelength of 320 nm is calculated taking into consideration that the wavelength range of the UVB region is 320 to 280 nm. However, the wavelength range is not limited to this. For example, when estimating the state of a coating film formed on the skin by a cosmetic product and estimating and evaluating the ultraviolet protection effect in the UVA region from the estimation result, the reflectance of ultraviolet light may be calculated using ultraviolet light with a wavelength of 380 to 320 nm, for example, ultraviolet light with a wavelength of 350 nm.
2) Divide into two groups according to UV reflectance The calculated UV reflectance of the coating film formed on the skin by each cosmetic was ranked in order of UV reflectance, and each subject was divided into two groups: those with high UV reflectance (50%) and those with low UV reflectance (50%). The group with high UV reflectance was designated the "group with low quality of cosmetic coating film formation," and the group with low UV reflectance was designated the "group with high quality of cosmetic coating film formation." Note that the UV reflectance of the coating film formed on the skin by each cosmetic calculated for each subject was different (ranked) because the coating film of each cosmetic differs depending on the condition of the skin to which the cosmetic is applied. Note that the condition of the skin to which the cosmetic is applied affects the condition of the cosmetic coating, as explained in embodiment 1.
3) Measuring the subject's skin condition index Skin condition indexes (7 items) are obtained using the above-described mathematical model of skin condition indexes and the subject's captured skin image (bare skin image).
4) Creating training data to be used for machine learning Using the results of dividing into two groups in 2) and the results obtained in 3), training data for creating a paint film estimation mathematical model 2 is created.
5) Creating a coating film estimation mathematical model 2 A standardization process is performed on each evaluation value, and the most appropriate model is selected from multiple classification models (e.g., naive Bayes (NB), logistic regression (LR), linear discriminant analysis (LDA), random forest (RF), gradient boosting (GB), support vector machine (SVM), Gaussian process (GP)) to create a prediction model. The prediction model is used to predict whether the quality of the coating film formation state of the cosmetic falls into a low or high group. The degree of agreement between the predicted group and the actual group is calculated as the accuracy rate. The generalization performance of the classification model was evaluated using leave-one-out cross-validation (LOOCV), and the calculation model with the highest accuracy rate was designated as the optimal model: coating film estimation mathematical model 2. In this embodiment, classification model RF was used as the coating film estimation mathematical model 2 for cosmetic C.

FIG. 6( a ) shows the results of predictions made using the Coating Film Estimation Mathematical Model 2 to determine whether the quality of the coating film formed by a cosmetic on the skin falls into the low or high group, as well as the actual results. FIG. 6( b ) shows the results of calculations of the accuracy rate, which is the degree of agreement between the group predicted using the Coating Film Estimation Mathematical Model 2 and the actual group.
6(a) and 6(b) show that when the film formation state of a cosmetic material formed on the skin is predicted using the film formation estimation mathematical model 2, a high accuracy rate can be obtained.
Therefore, by using the user's skin condition index estimated using the skin condition index mathematical model and the coating film estimation mathematical model 2, it is possible to estimate the state of formation of a coating film that a specific composition will form on the skin without applying the composition to the skin.Therefore, even without applying the specific composition (a cosmetic product to which an ultraviolet absorber or an ultraviolet scattering agent has been added) to the skin, it is possible to estimate the quality of the state of formation of a coating film that a specific composition will form on the skin when the specific composition is applied to the user's skin.This makes it possible to evaluate the extent to which an effect based on the function of the specific composition (for example, the ultraviolet protection function in the case of a cosmetic product to which an ultraviolet absorber or an ultraviolet scattering agent has been added) can be expected.

As described above, in embodiment 2, a mathematical model (film estimation mathematical model 2) is created that shows the relationship between the skin condition index and the UV reflectance of the cosmetic film, based on the skin condition index (e.g., seven items calculated using the subject's skin image and the skin condition index mathematical model) of each of multiple subjects (e.g., 20 women) and the UV reflectance of the cosmetic film formed on the skin calculated from the UV skin images of the multiple subjects (e.g., the images shown in Figure 2-2 taken with the imaging device shown in Figure 2-1). Using the created film estimation mathematical model 2, the user's skin image (bare skin image), and the user's skin condition index (e.g., the above seven items) obtained from the skin condition mathematical model, it is possible to estimate the state of the film formed on the skin by the cosmetic when the cosmetic is applied to the user's skin, thereby making it possible to estimate and evaluate the functions of the cosmetic.

Furthermore, as in embodiment 1, the state of the coating film formed by the cosmetic on the skin was estimated using the reflectance of ultraviolet light with a wavelength of 320 nm in the UVB wavelength range. Therefore, the state of the coating film formed by the cosmetic on the skin can be used to directly estimate and evaluate the functionality of the cosmetic (for cosmetics with UVB protection functionality, the UVB protection functionality). Specifically, for example, in the case of cosmetic C (SPF 50+), the UV reflectance of each of multiple subjects (20 people) was calculated, and a standard value for the calculated UV reflectance was determined. In this embodiment, the "standard value" refers to the UV reflectance that is located in the middle when the UV reflectances are sorted in order of UV reflectance, as described in "2) Divide into two groups according to UV reflectance." If the state of the film formed by the cosmetic on the subject's skin (ultraviolet reflectance) estimated using the above method is a standard value, then the claimed effect of cosmetic C (equivalent to SPF 50+) can be expected; if it is higher than the standard value, then the claimed effect of cosmetic C (equivalent to SPF 50+) can be expected to be sufficient; and if it is lower than the standard value, then the claimed effect of cosmetic C (equivalent to SPF 50+) cannot be expected to be very good. Therefore, the user can determine whether or not the cosmetic is suitable for their skin based on the estimated state of the film formed by the cosmetic on the skin.

Furthermore, similarly to embodiment 1, it is possible to estimate the first formation state and the second formation state by creating the coating film estimation mathematical model 2, where the first time period is, for example, one minute after immediately after application, and the second time period is, for example, eight hours after immediately after application. Note that the creation method, similarly to embodiment 1, can be achieved by calculating the UV reflectance of the cosmetic coating film formed on the skin one minute after application of the specified cosmetic to the subject's skin and eight hours after application of the specified cosmetic to the subject's skin, in addition to the step of “1) calculating the UV reflectance of the cosmetic coating film formed on the skin,” as described in embodiment 2, and then performing the steps of “2) dividing into two groups according to the UV reflectance” to “5) creating the coating film estimation mathematical model 2,” thereby creating a coating film estimation mathematical model 2 that estimates the formation state of the coating film formed on the skin by the cosmetic one minute after application (first formation state), and a coating film estimation mathematical model 2 that estimates the formation state of the coating film formed on the skin by the cosmetic eight hours after application (first formation state).
In this way, a coating film estimation mathematical model 2 is created that estimates the formation state of a coating film formed on the skin by a cosmetic one minute after application (first formation state), and a coating film estimation mathematical model 2 that estimates the formation state of a coating film formed on the skin by a cosmetic eight hours after application (second formation state).By using the two created coating film estimation mathematical models 2 and the user's skin condition index obtained using the skin condition index mathematical model, it is possible to estimate the formation state of a coating film formed on the skin by a cosmetic a predetermined time after application, without applying the specified cosmetic to the user's skin and even before a predetermined time has passed since application (in this embodiment, first time period: 1 minute or second time period: 8 hours).
In this way, it is possible to estimate the state of a coating film formed on the skin by a given composition (e.g., cosmetic C) in a first time period (e.g., 1 minute after application) after application (first formation state), and the state of a coating film formed on the skin by the composition in a second time period (e.g., 8 hours after application) after the first time period (second formation state). It is not necessary to estimate both the first formation state and the second formation state; only one of them may be estimated depending on the purpose. It is also possible to add a third time period, a fourth time period, etc.

Furthermore, as in embodiment 1, by using the first formation state and the second formation state, it is possible to estimate the effects that can be expected in the first time period and the second time period in relation to the effects that the applied composition is seeking.
In this way, the specified cosmetic is a cosmetic (e.g., cosmetic C) to which an ultraviolet absorber or an ultraviolet scattering agent has been added, and it is possible to estimate the first and second formation states and, based on the estimated first and second formation states, estimate and evaluate the functions exerted by the cosmetic (evaluate the extent to which the effect can be expected from the functions possessed by the cosmetic).
Therefore, for example, if one wishes to estimate and evaluate the UV protection effect when a cosmetic is applied before going out and then 8 hours later is active in a situation where exposure to UV rays is high, one can simply estimate the state of the film that the cosmetic forms on the skin in the second time period. Alternatively, if one wishes to evaluate the UV protection effect when a cosmetic is applied before going out and then is active for 8 hours continuously in a situation where exposure to UV rays is high, one can estimate both the state of the film that the cosmetic forms on the skin in the first time period and the second time period and use both to estimate and evaluate. In this way, it is possible to use different coating film estimation mathematical models 2 depending on the purpose.

<Embodiment 3>
Embodiment 3 is a method for estimating the state of a coating film formed on the skin by a specified composition when the composition is applied, based on a skin condition index generated using a mathematical model from an image of the user's skin (bare skin) and a mathematical model showing the relationship between this skin condition index and the state of a coating film formed by the specified composition when the composition is applied to the skin.
In this embodiment, the mathematical model used to generate the skin condition index is referred to as a "skin condition index mathematical model," and is the same as the "skin condition index mathematical model" described in embodiment 2. A mathematical model showing the relationship between the acquired skin condition index and the state of a coating film formed by a composition when a predetermined composition is applied to the skin is referred to as a "coating film estimation mathematical model 3." In embodiment 3, the "skin condition index mathematical model" and the "coating film estimation mathematical model 3" are used to estimate the state of a coating film formed by a composition on the user's skin without applying the composition to the skin.
The skin condition index mathematical model and paint film estimation model 3 will be described.

The "skin condition index mathematical model" is a trained discriminant model that has been trained based on multiple training data and is trained to be able to obtain at least the following seven skin condition indices, and is similar to the "skin condition index mathematical model" described in embodiment 2.
・Overall evaluation (overall skin condition)
Skin color (whiteness of skin)
・Makeup feel (does it look like makeup on the skin?)
・Skin age (an index showing the degree of skin aging)
-Masculinity/femininity (whether the skin texture is closer to male or female)
・Makeup smudge level (an index showing the degree of makeup smudge)
・Powderiness (an index showing how smooth the skin feels)
The "skin condition index mathematical model" is the same as the "skin condition index mathematical model" described in embodiment 2, and is therefore created using both a face image of bare skin and a face image with a predetermined cosmetic applied, as well as the evaluation results obtained by evaluating the above seven items for each of the images. A description of the method for creating the "skin condition index mathematical model" will be omitted.

Next, the paint film estimation mathematical model 3 will be described.
The training data and conditions used to create the coating film estimation mathematical model 3 are as follows:
Prescribed composition 20: Cosmetic D (SPF50+, formulation O/W)
Cosmetic E (SPF50+ formulation W/O)
・Application area: Whole face ・Application amount: 300 mg for the whole face
Subjects: 20 female full faces for both Cosmetic D and Cosmetic E; Camera used: Visible light camera; Application method: Subjects applied the cosmetic to the entire face evenly. As shown in FIG. 7 , a predetermined composition 20 (Cosmetic D, Cosmetic E) was applied to the skin of the subject, and image data (applied skin image) captured with a photographing device 40 (smartphone 40) was used. Note that the photographing device 40 (smartphone 40) may be any type, including a digital camera, a video camera, or a photographing device included in a mobile phone, tablet terminal, or personal computer, which are commonly referred to as mobile terminals. A camera capable of capturing color images using visible light is preferred. In this embodiment, photographs are taken using a camera installed on a smartphone. A coating film estimation mathematical model 3 is created from the captured image data (applied skin image) using the following procedure.

1) Obtaining an evaluation value for the overall evaluation of applied skin As described above, images of the entire face of multiple subjects (20 women in this embodiment) who applied each of cosmetic D and cosmetic E are obtained. An evaluation value for the overall evaluation of applied skin is obtained using the obtained images of the entire face of applied skin and the "mathematical model for skin condition indexes."
The "evaluation value of the overall applied skin evaluation" is the evaluation value of the "overall evaluation" among the skin condition indices (the above seven items (overall evaluation, skin color, makeup feel, skin age, masculinity/femininity, makeup smudge level, and powdery feel)) calculated using the acquired applied skin image of the entire face and the "skin condition index mathematical model." In this embodiment, to distinguish it from the "overall evaluation," which is one item among the "skin condition indices" calculated using the user's skin image (bare skin image) and the "skin condition index mathematical model," the "overall evaluation" calculated using the applied skin image of the entire face to which cosmetic D (or cosmetic E) has been applied and the "skin condition index mathematical model" is referred to as the "applied skin overall evaluation." Furthermore, the evaluation value of the applied skin overall evaluation is referred to as the "applied skin overall evaluation value."
2) Dividing into two groups according to the overall skin application evaluation score The overall skin application evaluation scores for each cosmetic calculated for each subject were sorted in order of overall skin application evaluation score, and the subjects were divided into two groups: those with high overall skin application evaluation scores (50%) and those with low overall skin application evaluation scores (50%). The group with high overall skin application evaluation scores was designated the "group with high quality of the cosmetic film formation state," and the group with low overall skin application evaluation scores was designated the "group with low quality of the cosmetic film formation state."
3) Measuring the subject's skin condition index Skin condition indexes (7 items) are obtained for the subject using the skin condition index mathematical model described in embodiment 2 and an image of the subject's skin (bare skin) taken with a visible light camera.
4) Create training data to be used for machine learning. Create training data using the results of dividing the data into two groups in 2) and the results obtained in 3).
5) Creating a coating film estimation mathematical model 3 A standardization process is performed on each evaluation value, and the most appropriate classification model is selected from multiple classification models (e.g., naive Bayes (NB), logistic regression (LR), linear discriminant analysis (LDA), random forest (RF), gradient boosting (GB), support vector machine (SVM), Gaussian process (GP)) to create a prediction model. This prediction model is used to predict whether the quality of the coating film formation state of the cosmetic falls into a low or high group. The degree of agreement between the predicted group and the actual group is calculated as the accuracy rate. The generalization performance of the classification model is evaluated using leave-one-out cross validation (LOOCV), and the calculation model with the highest accuracy rate is designated as the optimal model: coating film estimation mathematical model 3. In this embodiment, the coating film estimation mathematical model 3 for cosmetic D uses the classification model NB, and the coating film estimation mathematical model 3 for cosmetic E uses the classification model LDA.

In this way, the coating film estimation mathematical model 3 is created by learning using "seven skin condition indices" generated using the skin condition index mathematical model and skin images (bare skin images) of multiple subjects, and "overall applied skin evaluation value" generated using the skin condition index inference model and full-face applied skin images of multiple subjects with cosmetic D (or cosmetic E) applied to their skin. The seven skin condition indices include items that change depending on skin physical properties (e.g., moisture content of each layer, moisture evaporation value, etc.) that affect the formation of a cosmetic film on the skin. Therefore, research has shown that using all of the calculated seven skin condition indices can create a coating film estimation mathematical model 3 with a high accuracy rate. Furthermore, while items similar to the seven skin condition indices can be calculated using the skin condition index inference model and full-face applied skin images of subjects with cosmetic D (or cosmetic E) applied to their skin, this embodiment uses only the "overall applied skin evaluation value," which is the evaluation value of the overall applied skin evaluation, which corresponds to the overall evaluation. This is because this method estimates the state of a coating film formed on the skin when cosmetics are applied to the skin, and as a result of further research, it was found that the evaluation results of the appearance and finish of the skin to which cosmetics have been applied are the most important when estimating the state of a coating film formed on the skin. As a result, it was found that a mathematical model for estimating the state of a coating film with a high accuracy rate can be created by using the "overall evaluation value of applied skin," which is an evaluation value for an overall evaluation that indicates the overall quality of the skin condition.

FIG. 8( a ) shows the results of predictions using the coating film estimation mathematical model 3 to determine whether the quality of the coating film formation state of the cosmetic material falls into the low or high group, as well as the actual results, and FIG. 8( b ) shows the results of calculations of the accuracy rate, which is the degree of agreement between the group predicted using the coating film estimation mathematical model 3 and the actual group.
8(a) and 8(b) show that when the coating film formation state of a cosmetic material is predicted using the coating film estimation mathematical model 3, a high accuracy rate can be obtained.
Therefore, by using the user's skin condition index (7 items) obtained using the skin condition index mathematical model and the coating film estimation mathematical model 3, it is possible to estimate the state of coating film formation of a composition by a specified composition.As a result, even without applying the specified composition (a cosmetic product to which an ultraviolet absorber or an ultraviolet scattering agent has been added) to the skin, it is possible to estimate the quality of the coating film formation state assuming that the specified composition is applied to the user's skin.This makes it possible to estimate the degree of effect that can be expected based on the function possessed by the specified composition (for example, the ultraviolet protection function in the case of a cosmetic product to which an ultraviolet absorber or an ultraviolet scattering agent has been added).
As described above, in the third embodiment, skin condition indices (e.g., seven items calculated using skin images (bare skin images) of the subjects and a skin condition index mathematical model) are acquired for each of a plurality of subjects (e.g., 20 women). A comprehensive skin-applied evaluation value is calculated using skin images (e.g., skin images (applied skin images) captured with the imaging device shown in FIG. 7 ) of the skin (applied skin) of the plurality of subjects to which cosmetics (e.g., cosmetics D and E) have been applied, and the skin condition index mathematical model. A mathematical model (coating film estimation mathematical model 3) showing the relationship between the skin condition indices and the state of formation of a cosmetic coating film is created from the acquired skin condition indices (seven items) and the comprehensive skin-applied evaluation value. Using a user's skin image (bare skin image), the user's skin condition indicators (e.g., the seven items mentioned above) obtained from the skin condition mathematical model, and the coating film estimation mathematical model 3, it is possible to estimate the state of the coating film that a cosmetic (e.g., cosmetic D, cosmetic E) will form on the skin when it is applied to the user's skin, thereby making it possible to estimate and evaluate the functions of the cosmetic.

Furthermore, similarly to Embodiments 1 and 2, it is possible to estimate the first and second formation states by creating the coating film estimation mathematical model 3, where the first time period is, for example, one minute after immediately after application, and the second time period is, for example, eight hours after immediately after application. The creation method, similarly to Embodiments 1 and 2, involves performing the step of “1) obtaining an evaluation value for each overall skin application evaluation” described in Embodiment 3, obtaining evaluation values for each overall skin application evaluation one minute after application of a predetermined cosmetic to the skin of the subject, and eight hours after application of the predetermined cosmetic to the skin of the subject, and then performing the steps of “2) dividing into two groups according to the evaluation value for the overall skin application evaluation” to “5) creating a coating film estimation mathematical model 3” described in Embodiment 3, thereby constructing a coating film estimation mathematical model 3 that estimates the formation state of a cosmetic coating film one minute after application (first formation state) and a coating film estimation mathematical model 3 that estimates the formation state of a cosmetic coating film eight hours after application (second formation state), respectively.
In this way, a coating film estimation mathematical model 3 is created that estimates the formation state of a coating film formed on the skin by a cosmetic one minute after application (first formation state), and a coating film estimation mathematical model 3 that estimates the formation state of a coating film formed on the skin by a cosmetic eight hours after application (second formation state).By using the two created coating film estimation mathematical models 3 and the user's skin condition index obtained using the skin condition index mathematical model, it is possible to estimate the formation state of a coating film formed on the skin by a cosmetic a predetermined time after application, without applying the specified cosmetic to the user's skin and even before a predetermined time has passed since application (in this embodiment, first time period: 1 minute or second time period: 8 hours).
In this way, it is possible to estimate the state of a coating film formed on the skin by a given composition (e.g., cosmetic D or cosmetic E) in a first time period (e.g., 1 minute after application) after application (first formation state), and the state of a coating film formed on the skin by the composition in a second time period (e.g., 8 hours after application) after the first time period (second formation state). It is not necessary to estimate both the first formation state and the second formation state; only one of them may be estimated depending on the purpose. It is also possible to add a third time period, a fourth time period, etc.

Furthermore, as in embodiments 1 and 2, by using the first formation state and the second formation state, it is possible to estimate the effects that can be expected in the first time period and the second time period in relation to the effects that the applied cosmetic product is claiming.
In this way, the specified cosmetic is a cosmetic to which an ultraviolet absorber or an ultraviolet scattering agent has been added (e.g., cosmetic D, cosmetic E), and it is possible to estimate the first formation state and the second formation state, and based on the estimated first formation state and the estimated second formation state, to estimate and evaluate the function exerted by the cosmetic (evaluate the degree to which the effect can be expected from the function possessed by the cosmetic).

Next, we will explain a method for evaluating the compatibility of a cosmetic having a specified function with a user's skin using the state of the coating film formed on the skin by the cosmetic, estimated by the methods of embodiments 1 to 3.
As described above, in all of the embodiments, the state of a coating film that would be formed on the skin by a cosmetic containing an ultraviolet absorber or an ultraviolet scattering agent is estimated without actually applying the cosmetic to the user's skin. The specified cosmetic is a cosmetic containing an ultraviolet absorber or an ultraviolet scattering agent, and in Embodiments 1 and 2, the state of a coating film that would be formed on the skin by the cosmetic is estimated based on the ultraviolet reflectance at a wavelength of 320 nm, which is included in the UVB region, making it possible to estimate whether the specified cosmetic can be expected to have the effect of protecting against ultraviolet rays in the UVB wavelength region.
Furthermore, as described above, it is also possible to estimate a first formation state, which is the state of the coating film formed on the skin by the cosmetic in a first time period (e.g., one minute after application), and a second formation state, which is the state of the coating film formed on the skin by the cosmetic in a second time period (e.g., eight hours after application), and therefore, by using the first formation state and the second formation state, it is possible to estimate the effects that can be expected in the first time period and the second time period, respectively, in relation to the effects claimed by the applied cosmetic.
Then, a method for suggesting cosmetics suitable for the user using these estimation results will be described.

FIG. 9 shows an example in which the estimated first formation state in the first time period (after 1 minute) and the estimated second formation state in the second time period (after 8 hours) when three types of cosmetics, Cosmetic 1 to Cosmetic 3, were applied were higher or lower than the standard values.
Here, cosmetics 1 to 3 all have the same published UV protection value (in this embodiment, all are SPF 50). This method makes it possible to estimate whether the cosmetic can be expected to achieve its claimed effect by estimating the state of the coating film that the cosmetic will form on the skin when it is hypothesized to be applied, and therefore it is effective to compare cosmetics with the same published UV protection value. However, because standard values vary depending on the cosmetic, it cannot be said that similar UV protection values can be expected even if, for example, cosmetics 1 and 2 are both around the standard value. However, this information is useful when suggesting cosmetics suitable for the user.
9, for example, since both the first and second formation states of cosmetic 2 are lower than the standard values, it can be inferred that it will be difficult for the user to apply it evenly, and therefore it can be inferred that it will be difficult to expect the effects that cosmetic 2 claims. Furthermore, for cosmetic 1, the first formation state is higher than the standard value and the second formation state is about the standard value, while for cosmetic 3, the first formation state is about the standard value but the second formation state is higher than the standard value. Therefore, if the user expects UV protection effects immediately after applying the cosmetic, cosmetic 1 should be recommended, and if the user applies the cosmetic before going out and will be in an environment where they will be exposed to UV rays after a predetermined time, cosmetic 3 should be recommended. In this way, it is possible to use this information as reference when recommending cosmetics.
In this way, the cosmetics (e.g., cosmetics 1 to 3) have publicly announced UV protection values (e.g., SPF values) that have been published in advance, and it is possible to estimate and evaluate the UV protection function as a predetermined function based on the estimated state of the coating film that the cosmetics will form on the skin and the published UV protection values.

<Estimation device>
The estimation device 200 will be described with reference to Fig. 10-1 and Fig. 10-2. Fig. 10-1 shows the case of the first embodiment, and Fig. 10-2 shows the cases of the second and third embodiments.
The estimation device 200 in this embodiment is composed of a skin condition acquisition unit 130 and a coating film estimation unit 140. The estimation device 200 is a general-purpose personal computer and includes input devices such as a keyboard and a pointing device, a processing unit including an MPU (a CPU implemented as an integrated circuit), an input/output interface, a storage unit, and the like. The skin condition acquisition unit 130 is realized by the processing unit executing predetermined processing of a program via the input/output interface, and the coating film estimation unit 140 is realized by the processing unit executing predetermined processing of a program. The estimation device 200 preferably includes an evaluation unit 150 and a display unit 160 (display device), but the display unit 160 may be provided external to the estimation device 200 and connected via a network or the like. Furthermore, in the case of embodiment 1, a measuring device 10 that measures skin condition indices is required as shown in FIG. 10-1 , and in the cases of embodiments 2 and 3, an imaging device 40 that captures skin images (bare skin images) for calculating skin condition indices is required as shown in FIG. 10-2 . When recommending cosmetics by estimating the effectiveness of a specific cosmetic (cosmetics containing UV absorbers or UV scattering agents) by simulating its application without actually applying it to the user's skin in a store or other location, it is possible to sequentially suggest to the user the expected UV protection effect of the specific cosmetic and the recommended cosmetic by providing a measuring device 10 and/or a photographing device 40 in addition to the estimation device 200.

The skin condition acquisition unit 130 is a means (skin condition acquisition means) for acquiring skin condition indices (seven items) from measurements taken by the measuring instrument 10 (Embodiment 1) or using a skin image (bare skin image) captured by the imaging device 40 and a skin condition index mathematical model (Embodiment 2 or 3). Note that in Embodiments 2 and 3, an information processing terminal (not shown) different from the estimation device 200 may calculate the skin condition indices using a skin image (bare skin image) captured of the user and the skin condition index mathematical model, and the calculated skin condition indices may be acquired by the estimation device 200 via a network line, medium, or the like. Alternatively, the skin condition index mathematical model may be stored in the estimation device 200, a skin image (bare skin image) captured by the imaging device 40 may be acquired, and the skin condition indices may be calculated and acquired by the estimation device 200.
The coating film estimation unit 140 is a means (coating film state estimation means) that estimates the state of a coating film that a predetermined cosmetic material will form on the skin when it is assumed that the cosmetic material is applied to the skin of a user, using the acquired skin condition index and any one of the coating film estimation mathematical models 1 to 3. The estimation method is the same as that described above.
The evaluation unit 150 is a means for comparatively evaluating cosmetics containing UV absorbers or UV scattering agents that have published UV protection values, or for suggesting cosmetics containing UV absorbers or UV scattering agents that are suitable for the user's skin, using the state of the coating film formed on the skin by the cosmetic estimated by the coating film estimation unit 140. The evaluation results by the evaluation unit 150 are preferably displayed on the display unit 160 in a manner that is easy for the user to understand.
By displaying the evaluation results by the evaluation unit 150, for example, the diagram shown in FIG. 9, on the display unit 160, the user can easily understand the estimation results and evaluation results.
The storage unit of the estimation device 200 stores the above-mentioned mathematical models and a program for executing the above-mentioned estimation method, and the program causes the coating film estimation unit 140 to estimate the formation state of a coating film formed by the composition on the skin using the skin condition indices acquired by the skin condition acquisition unit 130 and the mathematical models, causes the evaluation unit 150 to make an evaluation, and causes the evaluation results to be displayed on the display unit 160. Note that some functions may be shared with another information processing device, such as by storing the mathematical models in another device, for example, a predetermined server (not shown), accessing the server from the estimation device 200 via the Internet, and having the other device estimate the formation state of a cosmetic coating film using the skin condition indices acquired by the estimation device 200.

<Estimation system>
The estimation system 400 will be described with reference to Fig. 11-1 and Fig. 11-2. Fig. 11-1 shows the case of the first embodiment, and Fig. 11-2 shows the cases of the second and third embodiments.
In this embodiment, the estimation system 400 is composed of a skin condition acquisition unit 330 and a coating film estimation unit 340. The estimation system 400 includes an information processing terminal 300 capable of executing various processes, and the information processing terminal 300 is equipped with the skin condition acquisition unit 330 and the coating film estimation unit 340. The information processing terminal 300 is a general-purpose personal computer and includes input devices such as a keyboard and a pointing device, an arithmetic processing unit constituted by an MPU (a CPU implemented as an integrated circuit), an input/output interface, a memory unit, etc. The skin condition acquisition unit 330 is realized by the arithmetic processing unit executing predetermined processing of a program via the input/output interface, and the coating film estimation unit 340 is realized by the arithmetic processing unit executing predetermined processing of a program. Furthermore, the information processing terminal 300 preferably includes an evaluation unit 350 and a display unit 360 (display device), but the display unit 360 may be provided external to the information processing terminal 300 and connected via a network or the like. Furthermore, in the case of embodiment 1, a measuring device 10 that measures the skin condition index is required as shown in Fig. 11-1, and in the cases of embodiments 2 and 3, a photographing device 40 that captures an image for calculating the skin condition index is required as shown in Fig. 11-2. When recommending cosmetics by estimating the degree of effect of a predetermined cosmetic (a cosmetic containing an ultraviolet absorber or an ultraviolet scattering agent) on the user's skin in a store or the like without actually applying it, and assuming that the cosmetic would be applied, by providing the measuring device 10 and/or the photographing device 40 in addition to the estimation system 400, it is possible to sequentially suggest to the user the expected degree of ultraviolet protection effect of the predetermined cosmetic and to recommend a cosmetic.

Skin condition acquisition unit 330 is a means (skin condition acquisition means) for acquiring a skin condition index from a measurement value measured by measuring instrument 10 (Embodiment 1) or using a skin image (bare skin image) captured by imaging device 40 and a skin condition index mathematical model (Embodiment 2 or 3). Note that in Embodiments 2 and 3, an information processing terminal (not shown) different from estimation system 400 may calculate a skin condition index using a skin image (bare skin image) captured of a user and the skin condition index mathematical model, and the calculated skin condition index may be acquired by information processing terminal 300 via a network line, medium, or the like. Alternatively, a skin condition index mathematical model may be stored in estimation system 400, a skin image (bare skin image) captured by imaging device 40 may be acquired, and the skin condition index may be calculated and acquired by estimation system 400.
The coating film estimation unit 340 is a means (coating film estimation means) that estimates the state of a coating film that a cosmetic will form on the skin when a predetermined cosmetic is applied to the user's skin, using the acquired skin condition index and any one of the coating film estimation mathematical models 1 to 3. The estimation method is the same as that described above.
The evaluation unit 350 is a means for comparatively evaluating cosmetics containing UV absorbers or UV scattering agents that have published UV protection values, or for suggesting cosmetics containing UV absorbers or UV scattering agents that are suitable for the user's skin, using the state of the coating film formed on the skin by the cosmetic estimated by the coating film estimation unit 340. The evaluation results by the evaluation unit 350 are preferably displayed on the display unit 360 in a manner that is easy for the user to understand.
By displaying the evaluation results by the evaluation unit 350, for example, the diagram shown in FIG. 9, on the display unit 360, the user can easily understand the estimation results and evaluation results.
The storage unit of the information processing terminal 300 stores the above-mentioned mathematical models and a program for executing the above-mentioned estimation method, and the program causes the coating film estimation unit 340 to estimate the state of a coating film formed by the composition on the skin using the skin condition indices acquired by the skin condition acquisition unit 330 and the mathematical models, causes the evaluation unit 350 to make an evaluation, and displays the evaluation results on the display unit 360. Note that some functions may be shared with another information processing device, such as by storing the mathematical models in another device, for example, a predetermined server (not shown), accessing the server from the estimation system 400 via the Internet, and having the other device estimate the state of a coating film of the cosmetic using the skin condition indices acquired by the estimation system 400.

<Application Program>
An application program (hereinafter also referred to as the program) for causing the estimation device 200 to execute the above-described estimation method is installed in the estimation device 200.
Similarly, an application program (hereinafter also referred to as the present program) for causing the information processing terminal 300 to execute the above-described estimation method is installed in the information processing terminal 300. In this embodiment, the application program is application software that estimates the formation state of a coating film of a composition when the composition is applied to the user's skin, without applying the composition to the user's skin.
This program includes skin condition acquisition processing and paint film estimation processing.
The skin condition acquisition process is a process of acquiring a skin condition index calculated using a skin condition index measured by the measuring instrument 10, a skin image (bare skin image) of the user captured by the imaging device 40, and a skin condition index mathematical model. Note that in the second and third embodiments, the skin condition index may be calculated by an information processing terminal (not shown) different from the estimation device 200 or the information processing terminal 300 using the captured skin image (bare skin image) of the user and the skin condition index mathematical model, and the calculated skin condition index may be acquired by the estimation device 200 or the information processing terminal 300 via a network line, medium, or the like. Alternatively, the skin condition index mathematical model may be stored in the estimation device 200 or the information processing terminal 300, a skin image (bare skin image) captured by the imaging device 40 may be acquired, and the skin condition index may be calculated and acquired by the estimation device 200 or the information processing terminal 300.
The coating film estimation process is a process that uses the acquired skin condition index and one of the coating film estimation mathematical models 1 to 3 to estimate the state of formation of a cosmetic coating when a specified cosmetic is applied to the user's skin.
Preferably, the program further includes an evaluation process and a display process.
The evaluation process uses the state of the coating film formed on the skin by the cosmetic product estimated in the estimation process to perform a comparative evaluation with cosmetics containing specified UV absorbers or UV scattering agents that have published UV protection values, and to suggest cosmetics containing UV absorbers or UV scattering agents that are suitable for the user.
The display process displays the evaluation results of the evaluation process, for example, the diagram shown in FIG. 9, making it possible for the user to easily understand the estimation results and evaluation results.

<Information processing terminal>
An information processing terminal (not shown) may be connected via a network to the estimation system 400. The information processing terminal may be, for example, a tablet terminal installed in a store, a smartphone, or a tablet terminal or smartphone owned by a user.
The information processing terminal is equipped with a transmitting means (e.g., an input/output interface) for transmitting data (e.g., a skin image (bare skin image)) for determining a skin condition index indicating the user's skin characteristics, and a receiving means (e.g., an input/output interface) for receiving the estimation results sent from the coating film estimation means (coating film estimation unit 340).
In this way, if there is an information processing terminal equipped with a transmission means for transmitting data (e.g., a skin image (bare skin image)), for example, if the information processing terminal is a tablet terminal or smartphone owned by the user, a user who is reluctant to have their bare skin image taken in a store can take a facial image of the user's bare skin using the tablet terminal or smartphone owned by the user at home, etc., and transmit the facial image of the user's bare skin as a skin image (bare skin image) to the information processing terminal 300 equipped in the estimation system 400 located in the store, thereby making it possible to estimate the coating film formation state when a predetermined cosmetic is applied. Furthermore, if the information processing terminal has a receiving means for receiving the estimation results, the estimation results can also be confirmed on the tablet terminal or smartphone owned by the user. Note that, as described above, it is preferable that the estimation system 400 includes an evaluation unit 350, and therefore, if the evaluation unit 350 is included, it is preferable that the receiving means be able to receive the evaluation results.

<How to display the estimation results>
Based on the skin condition index obtained using this estimation method, an image classified into skin types can be displayed as a skin classification image. Also, a predetermined composition can be displayed as a composition classification image, and an evaluation based on the estimated result of the state of the composition coating formed when the predetermined composition is applied to the skin of the user can be displayed by combining the skin classification image and the composition classification image.
As described above, the results estimated by this estimation method or the results evaluated using the estimated results are displayed on the display unit 160, the display unit 360, or the information processing terminal. The display forms will be explained using Figures 12(a-1) to 12(d-2).
The skin information classification images 50 in Figures 12(a-1) to (a-3) are examples of skin information classification images 50 when a user's skin type is classified into three skin types (skin type A, skin type B, and skin type C). Figures 12(a-1) to (a-3) show skin information classification images 50, including a skin information classification image 50 for skin type A, a skin information classification image 50 for skin type B, and a skin information classification image 50 for skin type C. Figures 12(b-1) to (b-3) are display examples of composition classification images 60 for three compositions (composition 1, composition 2, and composition 3), including a composition classification image 60 for composition 1, a composition classification image 60 for composition 2, and a composition classification image 60 for composition 3. As such, the skin information classification images 50 and the composition classification images 60 are all different images. Furthermore, the skin information classification images 50 may be set to images that evoke skin types, and the composition classification images 60 may be set to images that evoke compositions. For example, if skin type A is a moisturized skin type, the image may be a design or color that evokes moisture, making it easier for the user to visualize the skin type into which they have been classified.

For example, the compatibility between a user's skin type and a composition can also be displayed by combining the skin information classification image 50 and the composition classification image 60. For example, when the user's skin type is skin type A and composition 1 is selected, the skin information classification image 50 in Fig. 12(a-1) and the composition classification image 60 in Fig. 12(b-1) are displayed as a moving image that gradually overlaps, making it possible to display the compatibility results between skin type A and composition 1 in an easy-to-understand manner to the user.
FIG. 12(c-1) is an image showing the state in the middle of superimposing the skin information classification image 50 and the composition classification image 60.
Figures 12(d-1) and (d-2) show compatibility images 70 that show the estimated results by combining the skin information classification image 50 and the composition classification image 60. Figure 12(d-1) shows an image in which the skin information classification image 50 and the composition classification image 60 are perfectly overlapped, and is used when the compatibility between the user's skin type and the composition is high. On the other hand, Figure 12(d-2) shows an image in which the skin information classification image 50 and the composition classification image 60 are misaligned, and is used when the compatibility between the user's skin type and the composition is low. In this way, the user may understand the compatibility between their skin type and the composition based on the degree of overlap of the compatibility image 70. Furthermore, the estimated results may be displayed by combining the skin information classification image 50 and the composition classification image 60, and any method of combination (overlapping) or creation of the combined image may be used. For example, the skin information classification image 50 and the composition classification image 60 may be displayed side by side. For example, transparent images of the skin information classification image 50 and the composition classification image 60 may be created and displayed superimposed. For example, a portion of each of the skin information classification image 50 and the composition classification image 60 may be made transparent, and the transparent image portions (transparent pixel portions) of both images may be superimposed and displayed. For example, an image in which the skin information classification image 50 and the composition classification image 60 are superimposed may be created in advance, and the superimposed image created in advance may be displayed at the timing when the skin information classification image 50 and the composition classification image 60 are superimposed, thereby displaying the skin information classification image 50 and the composition classification image 60 in combination (superimposed). For example, any method may be used, such as thinning the skin information classification image 50 and the composition classification image 60 (shifting the thinning locations (thinning lines) between the skin information classification image 50 and the composition classification image 60) and displaying them superimposed. Furthermore, for example, when the compatibility between the user's skin type and the composition is high, the transmittance of the skin information classification image 50 may be increased and the transmittance of the composition classification image 60 may be decreased, whereas when the compatibility between the user's skin type and the composition is low, the transmittance of the skin information classification image 50 may be decreased and the transmittance of the composition classification image 60 may be increased. By displaying the skin information classification image 50 and the composition classification image 60 in combination, the compatibility of the composition with the user's skin type can be understood from the display mode of the compatibility image 70. Furthermore, for example, when the compatibility of the composition with the user's skin type is low, the composition classification image 60 may be displayed smaller than when the compatibility is high. In this way, when the compatibility is high, the skin information classification image 50 and the composition classification image 60 overlap at the same size, whereas when the compatibility is low, the composition classification image 60 can be displayed so that it overlaps a portion (e.g., the center) of the skin information classification image 50. This makes it possible to understand the compatibility of the composition with the user's skin type from the display mode of the compatibility image 70. Furthermore, for example, the brightness or saturation of the composition classification image 60 may be changed depending on the compatibility of the composition with the user's skin type, and the changed composition classification image 60 may be displayed in combination with the skin information classification image 50. This also makes it possible to understand the compatibility of the composition with the user's skin type from the compatibility image 70. The above content is an example, and displaying the skin information classification image 50 and the composition classification image 60 in combination means any display method or display mode as long as it is clear that the skin information classification image 50 and the composition classification image 60 are displayed in combination.
In addition to displaying a combination of the skin information classification image 50 and the composition classification image 60, a numerical value of the compatibility between the user's skin type and the composition can also be displayed.

As described above, the present invention has been described by showing specific embodiments, but the present invention is not limited to the above-described embodiments and also includes various modifications, improvements, and other aspects as long as the object of the present invention is achieved.
<Modification>
In the present embodiment, when estimating the first and second formation states, a coating film estimation mathematical model was created when a first time period had elapsed and when a second time period had elapsed after a predetermined cosmetic was applied to the skin of multiple subjects. However, this is not limiting, and a coating film estimation mathematical model may be created depending on the environment in which the first and second time periods elapsed. For example, by creating different coating film estimation mathematical models for when the first and second time periods elapsed under the blazing sun and when the first and second time periods elapsed after swimming in the sea, it becomes possible to suggest the optimal cosmetic for the environment in which the user will use the predetermined cosmetic (cosmetics containing ultraviolet absorbers or ultraviolet scattering agents).

In this embodiment, a cosmetic containing an ultraviolet absorber or an ultraviolet scattering agent was applied as the specified cosmetic, but this is not limited to this. Sunscreen cosmetics containing only an ultraviolet absorber, only an ultraviolet scattering agent, or both are preferred.

In this embodiment, cosmetic A was a W/O formulation, cosmetic B was a W/O formulation, cosmetic C was an O/W formulation, cosmetic D was an O/W formulation, and cosmetic E was a W/O formulation, but this is not limited to this. Furthermore, cosmetics A to E used in this embodiment contain 0-15% UV absorbers and 5-30% UV scattering agents, but the proportions of UV absorbers and/or UV scattering agents added to the cosmetics are not limited to this. Cosmetics containing 0-15% UV absorbers and 5-30% UV scattering agents are preferred.

In this embodiment, a cosmetic to which an ultraviolet absorber and/or an ultraviolet scattering agent was added was applied as the specified cosmetic. However, this is not limited to this. By adding a specified proportion of an ultraviolet absorber or an ultraviolet scattering agent (for example, 1% by mass or more but less than 30% by mass, preferably 5% by mass or more but less than 20% by mass, relative to the cosmetic) to a cosmetic to which no ultraviolet absorber or ultraviolet scattering agent has been added, it is possible to estimate the state of a coating film that would be formed if the cosmetic to which no ultraviolet absorber and/or ultraviolet scattering agent has been added were actually applied to the skin, without actually applying the cosmetic to the skin. In this case, although either an ultraviolet absorber or an ultraviolet scattering agent may be added, adding an ultraviolet absorber is preferable. This is because adding an ultraviolet absorber is less likely to affect the formation of a coating film of the cosmetic to which it is added.

As described above, this embodiment describes three types of estimation methods, from embodiments 1 to 3. Any of the estimation methods may be selected depending on the location and purpose of estimation. Furthermore, in embodiment 3, when creating the coating film estimation mathematical model 3, cosmetics containing ultraviolet absorbers and ultraviolet scattering agents are applied to the skin of multiple subjects, but the subjects' skin is photographed without being exposed to ultraviolet light (photographed with imaging device 40 (smartphone 40) rather than with imaging device 30 (ultraviolet camera 30) as in embodiments 1 and 2), thereby making it possible to reduce the burden on the subjects' skin when creating the coating film estimation mathematical model 3.

As described above, in this embodiment, the coating film formation state was estimated assuming that cosmetics A and B were applied in embodiment 1, cosmetics C in embodiment 2, and cosmetics D and E in embodiment 3. However, it is also possible to estimate the coating film formation state of cosmetic C, for example, using the method of embodiment 1. All that is required is to create coating film estimation mathematical models 1 to 3 corresponding to the cosmetic to be estimated.

As described above, in this embodiment, the state of the coating film formed when a cosmetic containing an ultraviolet absorber or an ultraviolet scattering agent is applied is estimated, and compatibility with the user's skin is evaluated based on the estimated results. However, this is not limited to this. The extent to which a specific composition's functions (e.g., UV protection function, anti-itch function, makeup function, whitening function, etc.) can also be evaluated from the estimated state of the coating film. This is because the better the state of the coating film formed, the more likely it is that the specific composition will be able to fully exhibit its functions, and this can be used, for example, as an indicator during development.

As described above, in this embodiment, the state of a coating film formed when a cosmetic containing an ultraviolet absorber or an ultraviolet scattering agent is applied is estimated, and the compatibility of the cosmetic with the user's skin is evaluated based on the estimated results. However, this is not limited to this. It is also possible to evaluate the compatibility of a composition with the user's skin based on the estimated results of the state of a coating film formed by the composition.

As described above, in this embodiment, the compatibility with the user's skin is evaluated by estimating the state of the coating film formed when a cosmetic containing an ultraviolet absorber or an ultraviolet scattering agent is applied, and evaluating whether the UV protection function can be expected to be high or low based on the estimation results. However, this is not limited to this. For example, if a cosmetic containing a whitening ingredient is applied, if the state of the coating film formed by application of the cosmetic is high, it can be inferred that the whitening ingredient will be evenly applied to the skin. Therefore, it is also possible to evaluate the compatibility of a cosmetic having a specified function (whitening function) with the skin.

As described above, in this embodiment, the "standard value" is the UV reflectance that is located in the middle when the UV reflectances are sorted in order of UV reflectance, as described in "2) Dividing into two groups according to UV reflectance," but this is not limited to this. For example, a value that falls within a predetermined top percentage when sorted in order of UV reflectance (e.g., a value within the top 25%) may be used as the standard value, and if the value is higher than the standard value, it may be evaluated that the cosmetic product is expected to have the intended effect. Furthermore, if the overall skin evaluation values are sorted in order of the overall skin evaluation values when applied, it may be used as the standard value, and if the value is higher than the standard value, it may be evaluated that the cosmetic product is expected to have the intended effect. Furthermore, multiple standard values may be prepared and divided into three or more groups.

In this embodiment, the first and second formation states are used individually to estimate the UV protection effect for each time period, but this is not limited to this. For example, the average value of the first and second formation states may be calculated, and the expected UV protection effect for the second time period from the first time period may be estimated from the average value. Alternatively, a predetermined weighting may be assigned to either the first or second formation state (e.g., prioritizing the value of the second formation state, which has been applied longer since application, or prioritizing the value of the first or second formation state, whichever has a higher coating film formation state), and then the average value may be calculated to estimate the expected UV protection effect for the second time period from the first time period. In this way, by using the first and second formation states to estimate the expected UV protection effect for the second time period from the first time period, it is possible to estimate the UV protection effect that changes over time since application, making it possible to provide users with more detailed information they need.

In this embodiment, the user is assumed to be, for example, a customer who comes to a store looking for cosmetics, but this is not limited to this. For example, the user could also be a monitor during evaluations conducted in a laboratory during the development of cosmetics. In this embodiment, the "user" is a person with skin for which the state of a coating film formed on the skin when a specified composition is applied is estimated using coating film estimation mathematical models 1 to 3 without applying the specified composition to the skin, and the "subject" is a person with skin whose skin is used as training data when creating each mathematical model.

(Additional Test Example 1)
13 is a diagram showing the results of a comparative test of the UV protection effects of the above-mentioned cosmetics C to E. As shown in the figure, the "overall evaluation" and "skin color" determined by the above-mentioned skin condition index mathematical model were each divided into two groups using the median values as thresholds, creating four groups, and the UV protection effects of cosmetics C (Embodiment 2), cosmetic D (Embodiment 3), and cosmetic E (Embodiment 3) were compared for each group.

The results showed that, in terms of UV protection effect, cosmetic product E is recommended for Group 1, which has a high overall rating and a high skin color value; cosmetic product C is recommended for Group 2, which has a high overall rating and a low skin color value; cosmetic products D and E are recommended for Group 3, which has a low overall rating and a high skin color value; and cosmetic products C and E are recommended for Group 4, which has a low overall rating and a high skin color value.

In this way, not only when using the seven skin condition indices in the above-mentioned embodiment as explanatory variables, but also when classifying skin types into four groups using the two items "overall evaluation" and "skin color" as explanatory variables, it is possible to recommend to the user cosmetics with high UV protection effects for that skin type.

(Additional Test Example 2)
14 is a diagram showing the results of another comparative test of the UV protection effects of the above-mentioned cosmetics C to E. The test participants were classified into five groups by a non-hierarchical clustering method (k-means method) using seven items determined by the above-mentioned skin condition index mathematical model: "overall evaluation,""skincolor,""makeupfeel,""skinage,""masculinity/femininity,""makeup smudge level," and "powdery feel," and the UV protection effects of cosmetics C (embodiment 2), cosmetics D (embodiment 3), and cosmetics E (embodiment 3) were compared for each group.

As a result, differences were found in the UV protection effects of cosmetics C, E, and E among groups 1 to 5. In terms of UV protection effect, cosmetic D was recommended for group 1, cosmetic C for group 2, cosmetics C and D for group 3, cosmetic E for group 4, and cosmetic D for group 5.

In this way, even if skin types are classified into five groups using the seven skin condition indices in the above-mentioned embodiment as explanatory variables, cosmetics with high UV protection effects for that skin type can be recommended to the user.

(Processing for suggesting cosmetics to users according to their skin type)
In the above-described embodiment, an example was shown in which an image classified into skin types based on estimated skin condition indexes is displayed as a skin classification image on the user's information processing terminal, and a specified composition is displayed as a composition classification image on the user's information processing terminal, and an evaluation based on the estimated results of the formation state of a coating film of the composition when the specified composition is applied to the user's skin is displayed by combining the skin classification image and the composition classification image.

In this case, the estimation device 200 or estimation system 400 may display information indicating the characteristics of the classified skin types on the user's information processing terminal, and may also display a score indicating the compatibility (UV protection effect) of each cosmetic product corresponding to that skin type with the user's skin type on the user's information processing terminal.

Figure 15 is a flowchart showing the flow of the process of presenting skin type characteristic information and cosmetic score information based on a skin image by the estimation device 200 or the estimation system 400. For convenience, in the following description, the estimation device 200 will be the subject of operations, but the operations are actually performed by the hardware of the estimation device 200 or the estimation system, such as the CPU, communication unit, and memory unit, in cooperation with the software (program) stored in the memory unit.

As shown in the figure, the estimation device 200 receives from the user's information processing terminal a skin image of the user's natural face captured by the information processing terminal (step 11).

The estimation device 200 then obtains a skin condition index value based on the received skin image and the skin condition index mathematical model (step 12).

The estimation device 200 then classifies the user into one of the skin types based on the acquired skin condition index values (step 13).

When generating the classification information for the skin types, the estimation device 200 or estimation system 400 generates and stores feature information to explain the characteristics of each skin type, as shown in FIG. 16, for example. The feature information is generated by evaluating each skin type based on the skin condition index values and the subject's awareness of skin quality.

In addition, the estimation device 200 or estimation system 400 generates and stores score information indicating scores related to the UV protection effects of multiple types of cosmetics based on the coating film formation state for each of the above skin types.

Score information is generated by creating a matrix of skin type and cosmetic product, as shown in Figure 17, for example. Score information is calculated numerically using, for example, the UV reflectance of the coating film and the overall evaluation value immediately after application and after a certain period of time has passed since application. When comparing products of the same skin type, a higher score indicates a higher UV protection effect.

The score information is not limited to being expressed as a value between 0 and 100 as described above, but may be expressed as a value between 0 and 10, or may be any evaluation information, such as a five-level rating of A, B, C, D, E (or the number of stars, etc.).

The estimation device 200 then displays, on the user's information processing terminal, the stored characteristic information corresponding to the user's skin type, along with the classified user's skin type information, as shown in FIG. 18(A) (step 14). In the example shown in the figure, the user is classified as skin type A, and the corresponding characteristic information (moist skin type) is displayed along with the information on skin type A.

Then, as shown in FIG. 18(B), the estimation device 200 displays information about cosmetics with high UV protection effects for the skin type and their score information on the user's information processing terminal, for example, when the user scrolls/tap on the displayed characteristic information or when a predetermined time has passed since the start of display (step 15).

The cosmetics and scores displayed here will be those with at least the highest score for the classified skin type, but information on the top several cosmetics (1st to 3rd place, etc.) and their scores may also be displayed.

In this way, based on the user's skin image and the skin condition index obtained from the skin condition index mathematical model, the user can be classified into one of several skin types, and the user can be presented with and understood information about the characteristics of that skin type and information about cosmetics with high UV protection effects for that skin type.

10 Measuring instrument 15 Skin 20 Composition 30 Photography device (ultraviolet camera)
40. Camera (smartphone)
50 Skin information classification image 60 Composition classification image 70 Compatibility image 130 Skin condition acquisition unit 140 Coating film estimation unit 150 Evaluation unit 160 Display unit 200 Estimation device 330 Skin condition acquisition unit 340 Coating film estimation unit 350 Evaluation unit 360 Display unit 400 Estimation system

Claims (17)

Acquire a skin condition index indicating a skin characteristic of the user;
The method uses a mathematical model showing the relationship between the acquired skin condition index and the state of formation of a coating film of a composition when the specified composition is applied to the user's skin to estimate the state of formation of a coating film of the composition when the specified composition is applied to the user's skin, and estimates the UV protection function based on the results. The estimation method described in claim 1, characterized in that the skin condition index includes evaluation items based on physical property values indicating skin condition measured by a specified measuring device. The estimation method described in claim 2, characterized in that the evaluation items based on the physical property values include at least one of the following: stratum corneum moisture content, water evaporation rate, skin viscoelasticity, skin flexibility, and contact angle. The estimation method described in claim 1, characterized in that the skin condition index includes evaluation items obtained using a skin condition index mathematical model that shows the relationship between a skin image captured of the user's skin and the skin condition index. the predetermined composition is a cosmetic containing at least one of an ultraviolet absorbing component and an ultraviolet scattering component added thereto,
a method for estimating a coating state of the cosmetic based on the skin condition index acquired from the user and the mathematical model, and estimating an ultraviolet protection function based on the result of the estimation,
Acquire skin condition indexes for each of the subjects,
acquiring ultraviolet reflectances of the coating film of the cosmetic from ultraviolet skin images of the skin of a plurality of subjects to which the cosmetic has been applied, under ultraviolet irradiation;
The estimation method according to any one of claims 1 to 4, characterized in that the mathematical model is created by performing machine learning based on the acquired skin condition indices of the plurality of subjects and the acquired ultraviolet reflectance of the cosmetic coating film of the plurality of subjects. the predetermined composition is a cosmetic containing at least one of an ultraviolet absorbing component and an ultraviolet scattering component added thereto,
a method for estimating a state of formation of a coating film of the cosmetic based on the skin condition index acquired from the user and the mathematical model, and estimating an ultraviolet protection function based on the result of the estimation,
Acquire skin condition indexes for each of the subjects,
obtaining a comprehensive skin evaluation indicating the condition of the skin to which the cosmetic has been applied for each of the plurality of subjects;
the overall evaluation of the applied skin is an evaluation item obtained using an applied skin image obtained by capturing an image of the skin of the subject to which the cosmetic has been applied and a mathematical model of skin condition indicators showing a relationship between the applied skin overall evaluation,
The estimation method according to any one of claims 1 to 4, characterized in that the mathematical model is created by performing machine learning based on the acquired skin condition indexes of the plurality of subjects and the acquired overall skin evaluations of the plurality of subjects. The estimation method described in any one of claims 1 to 6, characterized in that the function of the composition is evaluated based on the estimated state of formation of the coating film of the composition. the predetermined composition is a cosmetic containing at least one of an ultraviolet absorbing component and an ultraviolet scattering component added thereto,
The cosmetic has a previously published UV protection value,
The estimation method according to claim 1 , wherein the UV protection function is estimated and evaluated as a predetermined function based on the estimated state of formation of the cosmetic film and the published UV protection value. the predetermined composition is a cosmetic containing at least one of an ultraviolet absorbing component and an ultraviolet scattering component added thereto,
The cosmetic has a predetermined function,
The estimation method according to claim 1 , wherein the estimated state of formation of a coating film of the cosmetic is used to evaluate compatibility between the cosmetic and the user's skin. The estimation method described in any one of claims 1 to 9, characterized in that at least one of a first formation state, which is the formation state of the coating film of the composition in a first time period after application of the specified composition, and a second formation state, which is the formation state of the coating film of the composition in a second time period after the first time period, is estimated. classifying the user's skin into one of a plurality of skin types based on the skin condition index;
a skin information classification image indicating each of the plurality of skin types is determined in advance,
a composition classification image indicating the predetermined composition is determined in advance;
The estimation method according to claim 1 , wherein the estimation result is displayed using the skin information classification image and the composition classification image. The estimation method described in claim 11, characterized in that the estimation result is displayed by combining the skin information classification image and the composition classification image. storing characteristic information indicating characteristics of a plurality of types of skin types and evaluation information indicating an evaluation of the ultraviolet protection function of at least one or more cosmetics based on the state of formation of the coating film for each of the plurality of types of skin types;
classifying the user's skin into one of the plurality of skin types based on the skin image and the skin condition index acquired based on the skin condition index mathematical model;
outputting the characteristic information corresponding to the classified skin type;
The estimation method according to claim 4 , further comprising outputting the evaluation information for each of the cosmetics corresponding to the classified skin types. a storage means for storing a skin condition index indicating a user's skin characteristics and information indicating a correlation between the skin condition index and a coating state of a composition when a predetermined composition is applied to the user's skin;
a skin condition acquisition means for acquiring the skin condition index;
a coating film estimation means for estimating the state of a coating film of a composition when a predetermined composition is applied to the skin of the user by referring to the storage means and using the acquired skin condition index, and estimating the UV protection function based on the result of the estimation;
An estimation device comprising: An application program for operating an estimation device that estimates the state of a coating film formed by a predetermined composition when the composition is applied to a user's skin and estimates the UV protection function based on the result of the estimation,
a skin condition acquisition process for acquiring a skin condition index indicating a skin characteristic of the user;
and a coating film estimation process that estimates the state of a coating film of the composition on the user's skin using the acquired skin condition index, and estimates the UV protection function based on the result. a storage means for storing a skin condition index indicating a user's skin characteristics and information indicating a correlation between the skin condition index and a coating state of a composition when the composition is applied to the user's skin;
a skin condition acquisition means for acquiring the skin condition index;
a coating film estimation means for estimating the state of a coating film of a composition when a predetermined composition is applied to the skin of the user by referring to the storage means and using the acquired skin condition index, and estimating the UV protection function based on the result of the estimation;
An estimation system comprising: An information processing terminal used in an estimation system connected to the estimation device according to claim 14 via a network,
a transmitting means for transmitting data for determining a skin condition index indicating a user's skin characteristics; and a receiving means for receiving an estimation result transmitted from the coating film estimation means.
An information processing terminal used in an estimation system comprising: