JP2023121082A - Information providing device, information providing method, and information providing program - Google Patents

Abstract

To provide an information providing device capable of providing a user with useful information, an information providing method, and an information providing program.SOLUTION: In an information processing system in which an information providing device 10, a user terminal, and a makeup information database are communicably connected, the information providing device comprises: an acquisition unit 13a which acquires mode information on a hair mode capable of being provided by a user; a calculation unit 13b which calculates a congeniality degree indicating a congeniality degree of combinations of hair modes and makeup on the basis of the acquired mode information; a generation unit 13c which generates provision information provided to the user who has selected a hair mode on the basis of the calculated congeniality degree; and a learning unit 13e which learns a machine learning model so as to output congeniality degrees of combinations of hair modes and makeup made by object persons having hair of the mode. The information providing device calculates a congeniality degree using the machine learning model.SELECTED DRAWING: Figure 2

Description

The present invention relates to an information providing device, an information providing method, and an information providing program.

Conventionally, techniques for assisting coordination are known. For example, there is known a technique for generating coordination based on rules regarding fashion combinations.

JP 2013-235528 A

However, the conventional techniques described above have room for improvement in providing useful information to customers ("users" as appropriate). For example, with the conventional technology, it is difficult to propose makeup products that match the hairstyle that the user can provide to the user.

The present application has been made in view of the above, and aims to provide an information providing apparatus, an information providing method, and an information providing program capable of providing useful information to users.

In order to solve the above-described problems and achieve the object, an information providing apparatus according to the present invention includes an acquisition unit that acquires mode information related to a hair mode that can be provided by a user; characterized by comprising a calculation unit that calculates a degree of compatibility indicating the degree of compatibility between the aspect and the makeup, and a generation unit that generates provision information to be provided to the user based on the degree of compatibility. and

Further, an information providing method according to the present invention is an information providing method executed by an information providing device, comprising: an acquiring step of acquiring mode information related to a hair mode that can be provided by a user; , a calculation step of calculating a degree of compatibility indicating the degree of compatibility between the aspect and the makeup, and a generating step of generating provision information to be provided to the user based on the degree of compatibility. characterized by

Further, the information providing program according to the present invention includes an acquisition procedure for acquiring aspect information relating to a hair aspect that can be provided by a user, and indicating the degree of compatibility between the aspect and makeup based on the aspect information. A computer is caused to execute a calculation procedure for calculating the degree of compatibility and a generation procedure for generating provided information to be provided to the user based on the degree of compatibility.

In the present invention, useful information can be provided to users.

FIG. 1 is a diagram illustrating a configuration example of an information providing system according to an embodiment. FIG. 2 is a block diagram showing a configuration example of the information providing device according to the embodiment. 3 is a diagram illustrating an example of a mode information storage unit according to the embodiment; FIG. 4 is a diagram illustrating an example of a makeup information storage unit according to the embodiment; FIG. FIG. 5 is a diagram illustrating an example of a compatibility information storage unit according to the embodiment; 6 is a diagram illustrating an example of a provided information storage unit according to the embodiment; FIG. FIG. 7 is a diagram illustrating a specific example of information provision processing according to the embodiment. FIG. 8 is a flowchart illustrating an example of the flow of information provision processing according to the embodiment. FIG. 9 is a diagram illustrating an example of a hardware configuration;

DETAILED DESCRIPTION OF THE INVENTION Modes (hereinafter referred to as embodiments) for implementing an information providing apparatus, an information providing method, and an information providing program according to the present application will be described in detail below with reference to the drawings. Note that the information providing device, the information providing method, and the information providing program according to the present application are not limited to this embodiment. Also, in each of the following embodiments, the same parts are denoted by the same reference numerals, and overlapping descriptions are omitted.

[Embodiment]
The configuration of the information providing system 100, the configuration of the information providing apparatus 10, the specific example of the information providing process, and the flow of the information providing process according to the embodiment will be described below, and finally the effects of the present embodiment will be described.

[1. Configuration of Information Providing System 100]
The processing of the information providing system (this system as appropriate) 100 according to the present embodiment will be described with reference to FIG. FIG. 1 is a diagram showing a configuration example of an information providing system 100 according to an embodiment. In the following, a configuration example of the system 100, processing of the system 100, set matching technology applied in the system 100, and effects of the system 100 will be described in this order.

(1-1. Configuration example of information providing system 100)
The information providing system 100 shown in FIG. 1 includes an information providing device 10, a user terminal 20, and a makeup information database 30. Here, the information providing device 10, the user terminal 20, and the makeup information database 30 are communicably connected by wire or wirelessly via a predetermined communication network (not shown). The system 100 may include a plurality of information providing apparatuses 10, a plurality of user terminals 20, or a plurality of makeup information databases 30. FIG.

(1-1-1. Information providing device 10)
The information providing device 10 is a device that transmits and receives data to and from the user terminal 20 and the makeup information database 30, and is realized by, for example, a server device or a cloud system. The example of FIG. 1 shows a case where the information providing device 10 is realized by a server device.

(1-1-2. User terminal 20)
The user terminal 20 is used by a user U who browses web pages related to makeup (as appropriate, “makeup”) and the like, and conducts Internet shopping for cosmetics (as appropriate, “cosmetics”) and the like on the web. It is a device (computer). The user terminal 20 receives an operation by the user U. The user terminal 20 is realized by, for example, a smart phone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), or the like. The example of FIG. 1 shows a case where the user terminal 20 is realized by a smart phone.

The user terminal 20 may be referred to as a user U hereinafter. That is, the user U can be read as the user terminal 20 as well. User U may also be used to represent multiple customers.

(1-1-3. Makeup information database 30)
The makeup information database 30 is a storage device that stores makeup information related to makeup, which will be described later.

(1-2. Processing of information providing system 100)
The processing of steps S1 to S6 will be described below as the processing of the information providing system 100. FIG. It should be noted that steps S1 to S6 below may be performed in a different order. Also, some of steps S1 to S6 below may be omitted.

(1-2-1. Processing of step S1)
In this system 100, firstly, the information providing device 10 provides, through the user terminal 20 of the user U, who is a stylist of the beauty salon, style information on hairstyles that can be provided by the user U at the beauty salon. (step S1). Here, the style information is information relating to the style of the hair indicating the overall hairstyle such as hairstyle and hair color. Also, the mode information may be information about the mode of hair using a wig, wig, or the like, and is not particularly limited. In addition, in the example of FIG. 1, the information providing apparatus 10 acquires the mode information from the user terminal 20, but the mode information may be acquired from a terminal device/database (not shown). That is, it is also possible to acquire mode information associated with the mode of hair included in the image selected by the user U from among the images showing the modes of hair displayed on the user terminal 20 .

At this time, the information providing device 10 may acquire the user information of the user U from the user terminal 20 . Here, the user information includes gender, age, generation, occupation, annual income, place of residence, marital status, presence or absence of children, user U's moving image, user U's interests, and user U's interests. This information includes user attributes such as categories owned, user U's search history/browsing history on the website, purchase history in Internet shopping, and action history such as user U's location information. Furthermore, the user information may be information including screen information of the user terminal 20, biometric information of the user U, etc., and is not particularly limited.

(1-2-2. Processing of step S2)
In this system 100, secondly, the information providing device 10 refers to the makeup information database 30 and acquires makeup information (step S2). Here, the makeup information is information about makeup (makeup), including cosmetics such as foundation, lipstick, and eyeshadow, tools and tools used for makeup, and makeup procedures, etc., but is not particularly limited. . In the example of FIG. 1, the information providing apparatus 10 acquires makeup information from the makeup information database 30, but may acquire makeup information from a terminal device/database (not shown).

(1-2-3. Processing of step S3)
In the present system 100, thirdly, the information providing apparatus 10 calculates a degree of compatibility indicating the degree of compatibility between the hair style and makeup that can be provided by the user U from the acquired style information and makeup information (step S3). For example, the information providing device 10 uses a machine learning model to calculate the degree of compatibility so as to take a numerical value of 0 to 1 according to the degree of compatibility of the combination of the appearance of hair and makeup. At this time, the information providing device 10 outputs the degree of compatibility for each combination of the hair style and makeup when style information and makeup information related to the style of hair that can be provided by the user U are input. A degree of compatibility is calculated using a machine learning model such as a DNN (Deep Neural Network) learned as described above. Further, the information providing apparatus 10 may calculate the degree of compatibility based on rules.

Step S3 mentioned above is demonstrated using a specific example. For example, the information providing device 10 uses a machine learning model learned for each attribute of a target person to whom the user U applies makeup and a machine learning model learned for each target person situation (situation) as follows, Affinity can be calculated.

The information providing apparatus 10 can calculate the degree of compatibility using a machine learning model learned for each attribute including the interests and preferences of the target person. For example, the information providing apparatus 10 generates a plurality of machine learning models for a target person who likes "XX (eg, entertainer, age, type of drama (action, etc.))", and correct data for each machine learning model. Generate. Specifically, a machine learning model is learned by using an image in which a subject who likes "XX" is "Like" or an image matching "XX" as correct data. As a result, the information providing device 10 can select a machine learning model that matches the attributes of the target person specified by the user U, and can propose makeup that is highly compatible with the selected machine learning model.

In addition, the information providing apparatus 10 can calculate the degree of compatibility using a machine learning model learned for each situation of the subject. For example, the information providing device 10 uses the image of the target person who passed the document screening at the time of job hunting as the correct data for each situation of corporate recruitment such as new graduate, job change, and mid-career recruitment. Generate machine learning models. Therefore, for example, when the user U selects "job change", the information providing apparatus 10 can propose makeup according to document selection at the time of job change.

(1-2-4. Processing of step S4)
Fourthly, in the system 100, provision information to be provided to the user U is generated based on the calculated degree of compatibility, the acquired mode information, and the user information of the user U (step S4). Here, the provided information is information about makeup specified from the makeup indicated by the makeup information, and is information such as makeup that matches the style of hair that the user U can provide, but is not particularly limited. At this time, the information providing apparatus 10 identifies makeup having a degree of compatibility equal to or higher than a predetermined threshold among the makeup indicated by the makeup information, and generates provision information. For example, the information providing device 10 identifies an image of makeup with a degree of compatibility of “0.7” or higher with a hairstyle that can be provided by the user U, and specifies the image and the cosmetics required for the makeup shown in the image. Generate information as an offer.

Step S4 mentioned above is demonstrated using a specific example. For example, the information providing device 10 can generate the provided information based on the user information of the user U as described below.

The information providing apparatus 10 can generate provision information based on user attributes indicated by user information. For example, the information providing apparatus 10 preferentially proposes, among the makeup candidates, makeup similar to the user U's pre-registered preferred makeup features. Furthermore, the information providing apparatus 10 can preferentially propose makeup images that are relatively similar to the makeup features of the sample image provided by the user U to a hair and makeup reservation site or the like. Therefore, the information providing apparatus 10 can propose makeup with high similarity among candidates specified from makeup information based on user attributes and the like.

Further, the information providing device 10 can generate provision information based on the action history indicated by the user information. For example, when the cosmetics that the user U has already purchased can be estimated based on the action history such as the purchase history of the user U, the information providing device 10 proposes makeup based on the cosmetics.

(1-2-5. Processing of step S5)
Fifthly, in the system 100, the information providing device 10 transmits the generated provided information to the user terminal 20 of the user U (step S5). For example, the information providing device 10 uses provided information including an image of makeup with a degree of compatibility of "0.7" or higher with hair and makeup that the user U can provide, and information on cosmetics and makeup tools necessary for the makeup. It is transmitted to the user terminal 20 of the person U.

(1-2-6. Processing of step S6)
Sixthly, in the system 100, the information providing device 10 learns the calculated degree of compatibility (step S6). For example, using a machine learning model, the information providing apparatus 10 uses a machine learning model, and when information is input that indicates that the combination of the hair style and makeup that can be provided by the user U is good, the information providing device 10 Learning is performed using back propagation or the like so that the degree of compatibility of makeup with respect to is output as a numerical value that approaches “1”. In addition, the information providing apparatus 10 uses a machine learning model to, when information is input indicating that the combination of the hair style and makeup that can be provided by the user U is incompatible, Learning is performed using back propagation or the like so that the degree of compatibility of makeup with respect to is output as a numerical value that approaches “0”.

(1-3. Set matching technology applied in information providing system 100)
Details of the set matching technique applied in the information providing system 100 described above will now be described. As an example of a technique for calculating a score (hereinafter referred to as a "matching score" as appropriate) indicating the degree of harmony between sets based on sets of information indicating feature amounts such as images, set matching and A technique called Set matching is a technique using deep learning, for example, and it is estimated that the higher the matching score, the higher the compatibility between sets. Therefore, set matching can quantitatively evaluate compatibility between sets. Calculation of matching scores using set matching is disclosed, for example, in Y. Saito, T. Nakamura, H. Hachiya, K. Fukumizu "Exchangeable Deep Neural Networks for Set-to-Set Matching and Learning". Note that set matching is an example, and the embodiments are not limited to the use of this technique.

The information providing device 10 applies a technique of calculating matching scores between sets such as set matching to calculate matching scores (degree of compatibility) for each combination of hair style and makeup. Specifically, the information providing device 10 calculates the degree of compatibility with makeup for each type of hair that the user U can provide. At this time, the information providing apparatus 10 can estimate that the higher the degree of compatibility, the higher the compatibility between the hair style and makeup that the user U can provide. To specify makeup for which the degree of compatibility is calculated to be high. In addition, the information providing apparatus 10 estimates that the specified makeup is the makeup with the highest compatibility among the makeup included in the makeup information for one hair aspect. Accordingly, the information providing device 10 can quantitatively determine the compatibility between the appearance of the hair and the makeup, rather than the user U's sensory determination such as appearance.

Further, the information providing device 10 learns a machine learning model as follows. For example, the information providing apparatus 10 identifies images that have been favorably evaluated by viewers of the SNS among the images posted on the SNS. Next, the information providing apparatus 10 identifies the hair style (hairstyle, hair color) and makeup in the specified image. At this time, the information providing apparatus 10 may use various identification models, or obtain identification results through crowdsourcing or the like. The information providing device 10 uses the identification result as correct data. Further, the information providing apparatus 10 may use the result of identification from the image posted by the user U as correct data. Then, the information providing apparatus 10 outputs a high score when a hair style and makeup that are correct data are input, and a low score when a hair style and makeup that are not in the correct data are input. Train a machine learning model to output a score. At this time, the information providing apparatus 10 may generate such a machine learning model for each attribute of the subject to whom the user U applies makeup, or for each situation of the subject. Further, the information providing apparatus 10 may reflect the degree of compatibility in consideration of multiple perspectives by adopting the manipulation of the output values of multiple models as the output of the machine learning model.

(1-4. Effect of Information Providing System 100)
As described above, in the information providing system 100, the information providing apparatus 10 acquires the mode information regarding the mode of hair that the user U can provide, and based on the mode information, the hair that the user U can provide. A degree of compatibility indicating the degree of compatibility between the aspect and the makeup is calculated, and provision information to be provided to the user U is generated based on the degree of compatibility. At this time, the information providing apparatus 10 calculates the degree of compatibility using a machine learning model that has learned the degree of compatibility between the hair style and the makeup applied to the subject having the hair style. Therefore, the system 100 can estimate makeup that matches the hair and makeup that the user U can provide or desires to provide, and can propose cosmetics to the user U. In other words, the system 100 makes it possible to provide useful information to users U such as beauticians who propose hair and makeup on hair and makeup reservation sites and the like.

Further, in the present system 100, the information providing apparatus 10 further acquires user information regarding the user U who provides the hair style, specifies makeup based on the acquired user information, and generates provision information. Therefore, in the present system 100, the user U can provide hairstyles and makeup that can be provided by the user U, and the hairstyles and hairstyles that are favorably used by celebrities such as celebrities are evaluated on SNS. can suggest high-quality makeup. That is, the system 100 makes it possible to provide more specific and useful information to the user U who proposes hair and makeup.

Furthermore, in the present system 100, the information providing apparatus 10 uses a machine learning model that has learned the degree of compatibility between the combination of the hair style and the makeup applied to the subject who has the hair style to determine the degree of compatibility. calculate. At this time, the information providing apparatus 10 uses a machine learning model that learns the degree of compatibility from moving images including hair styles and makeup, or learns the degree of compatibility for each attribute of the subject to whom makeup is applied. The degree of compatibility is calculated by using a machine learning model or by using a machine learning model that has learned the degree of compatibility for each situation of the subject. For this reason, the system 100 generates various machine learning models learned using correct data collected from SNS, television, movies, etc., and proposes more accurate makeup to the user U who wears makeup. be able to. That is, the system 100 can provide more specific and useful information with high precision to the user U who proposes hair and makeup.

[2. Configuration of Information Providing Device 10]
The configuration of the information providing device 10 according to the embodiment will be described with reference to FIG. FIG. 2 is a block diagram showing a configuration example of the information providing device 10 according to the embodiment. As shown in FIG. 2 , the information providing device 10 has a communication section 11 , a storage section 12 and a control section 13 . The information providing apparatus 10 includes an input unit (for example, keyboard, mouse, etc.) for receiving various operations from the administrator of the information providing apparatus 10, and a display unit (for example, liquid crystal display, etc.) for displaying various information. may have.

(2-1. Communication unit 11)
The communication unit 11 is realized by, for example, a NIC (Network Interface Card) or the like. The communication unit 11 is connected to a predetermined communication network (network) by wire or wirelessly, and transmits and receives information to and from various devices.

(2-2. Storage unit 12)
The storage unit 12 is realized by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 12 according to the embodiment has, as shown in FIG. 2, a mode information storage unit 12a, a makeup information storage unit 12b, a compatibility information storage unit 12c, and a provision information storage unit 12d. The storage unit 12 stores various information referred to when the control unit 13 operates and various information acquired when the control unit 13 operates.

(2-2-1. Mode information storage unit 12a)
The mode information storage unit 12a stores various types of information (mode information) regarding the mode of hair that the user U can provide. Here, an example of information stored in the mode information storage unit 12a will be described with reference to FIG. FIG. 3 is a diagram showing an example of the mode information storage unit 12a according to the embodiment. In the example of FIG. 3, the mode information storage unit 12a has items such as "mode ID" and "mode information".

“Mode ID” indicates identification information for identifying the mode of hair. The “mode information” is information about the mode of hair indicating general hairstyles such as hairstyle and hair color, and information about the mode of hair using wigs, wigs, and the like.

That is, in FIG. 3, the mode information of the hair mode identified by the mode ID "HID#1" is "mode information #1", and the mode information of the hair mode identified by the mode ID "HID#2" is " An example of "mode information #2" is shown.

(2-2-2. Makeup information storage unit 12b)
The makeup information storage unit 12b stores various kinds of information (makeup information) about makeup. An example of information stored in the makeup information storage unit 12b will now be described with reference to FIG. FIG. 4 is a diagram showing an example of the makeup information storage unit 12b according to the embodiment. In the example of FIG. 4, the makeup information storage unit 12b has items such as "makeup ID" and "makeup information".

“Makeup ID” indicates identification information for identifying makeup. "Makeup information" is information about makeup, including cosmetics such as foundation, lipstick, and eye shadow, tools and tools used for makeup, and procedures for applying makeup.

That is, in FIG. 4, the makeup information of the makeup identified by the makeup ID "DID#1" is "makeup information #1", and the makeup information of the makeup identified by the makeup ID "DID#2" is "makeup information #2". ” shows an example.

(2-2-3. Compatibility information storage unit 12c)
The compatibility information storage unit 12 c stores the compatibility (compatibility information) for each makeup with respect to the appearance of the hair calculated by the calculation unit 13 b of the control unit 13 . Here, an example of information stored in the compatibility information storage unit 12c will be described with reference to FIG. FIG. 5 is a diagram showing an example of the compatibility information storage unit 12c according to the embodiment. In the example of FIG. 5, the compatibility level information storage unit 12c has items such as "mode ID", "makeup ID", and "compatibility level".

“Mode ID” indicates identification information for identifying the mode of hair. “Makeup ID” indicates identification information for identifying makeup. The “compatibility degree” is a numerical value indicating the degree of compatibility of each make-up with respect to the appearance of the hair.

That is, in FIG. 5, regarding the aspect of the hair identified by the aspect ID “HID#1”, the makeup information “makeup information #1”, the compatibility “compatibility #1”, and the aspect ID “HID#2” An example of the makeup information “makeup information #2” and the compatibility “compatibility #2” is shown for the aspect of the hair identified by .

(2-2-4. Provided information storage unit 12d)
The provided information storage unit 12d stores provided information generated by the generation unit 13c of the control unit 13 . Here, an example of information stored in the provided information storage unit 12d will be described with reference to FIG. FIG. 6 is a diagram showing an example of the provided information storage unit 12d according to the embodiment. In the example of FIG. 6, the provided information storage unit 12d has items such as "mode ID" and "provided information."

“Mode ID” indicates identification information for identifying the mode of hair. "Provided information" is information about makeup that is compatible with the style of hair that the user U can provide, and is information such as cosmetics, makeup tools, and makeup procedures to be provided to the user U.

That is, in FIG. 6 , the provided information about the hair aspect identified by the aspect ID “HID#1” is “provided information #1”, and the provided information about the hair aspect identified by the aspect ID “HID#2” is “provided information #1”. is "provided information #2".

(2-3. Control unit 13)
The control unit 13 executes various programs (equivalent to an example of an information processing program) stored in a storage device inside the information providing apparatus 10 by means of a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like. It is realized by being executed as a work area. Also, the control unit 13 is implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

As shown in FIG. 2, the control unit 13 has an acquisition unit 13a, a calculation unit 13b, a generation unit 13c, a transmission unit 13d, and a learning unit 13e, and implements or executes the information processing functions and actions described below. . Note that the internal configuration of the control unit 13 is not limited to the configuration shown in FIG. 2, and may be another configuration as long as it performs information processing to be described later. Moreover, the connection relationship between the processing units of the control unit 13 is not limited to the connection relationship shown in FIG. 2, and may be another connection relationship.

(2-3-1. Acquisition unit 13a)
Acquisition unit 13a acquires mode information about the mode of hair that can be provided by the user. For example, the acquiring unit 13a acquires, from the user terminal 20, information about the hair style, such as hairstyle and hair color, associated with the image selected by the user U as the style information.

The acquisition unit 13a also acquires makeup related to makeup. For example, from the makeup information database 30, the acquiring unit 13a acquires, as makeup information, makeup-related information such as cosmetics, makeup tools, and makeup procedures. At this time, the acquiring unit 13a may acquire from the makeup information database 30 the makeup information associated by the service provider of the hair and makeup reservation site. The acquisition unit 13a may also acquire makeup information including makeup registered in advance as makeup highly compatible with each hair style.

The acquisition unit 13a also acquires user information about the user U who provides the hair style.
For example, the acquisition unit 13a obtains, from the user terminal 20 of the user U, as user information, user attributes including categories in which the user U has an interest, search history, browsing history, and purchase history of the user U. Acquire action history such as history and location information.

Acquisition unit 13a stores the acquired mode information in mode information storage unit 12a. Further, the acquisition unit 13a stores the acquired makeup information in the makeup information storage unit 12b. Furthermore, the acquisition unit 13 a can store the acquired user information in the storage unit 12 .

(2-3-2. Calculation unit 13b)
The calculation unit 13b calculates a degree of compatibility indicating the degree of compatibility between the hair style and the makeup based on the style information. For example, to explain the calculation method, the calculation unit 13b uses a machine learning model that has learned the degree of compatibility between the combination of the hair style and the makeup applied to the subject who has the hair style, and calculates the degree of compatibility. Calculate As a specific example, the calculation unit 13b is a machine such as a DNN that has been learned to output the degree of compatibility of each makeup with respect to the appearance of hair when style information and makeup information selected by the user U are input. Compatibility is calculated using a learning model. Further, the calculation unit 13b may calculate the degree of compatibility of each makeup with the style of hair selected by the user U on a rule basis.

In addition, regarding the numerical value to be calculated, the calculation unit 13b calculates the degree of compatibility so as to take a numerical value of 0 to 1 according to the degree of compatibility for each makeup. Further, the calculation unit 13b may calculate the degree of compatibility so as to take a numerical value of 0 to 100% according to the degree of compatibility for each makeup, and the range and units of numerical values to be calculated are not particularly limited.

To explain using a specific example, for example, the degree of compatibility between hairstyle H and cosmetics M1 is "0.9", the degree of compatibility between hairstyle H and cosmetics M2 is "0.8", and the degree of compatibility between hairstyle H and cosmetics M2 is "0.8". When the degree of compatibility between the style H and the cosmetics M3 is "0.7", the calculation unit 13b estimates that the cosmetics M1, which have the highest degree of compatibility, are highly compatible with the hairstyle H. As a result, the calculation unit 13b can appropriately optimize the combination of furnishings and hair styles.

Further, learning data of the machine learning model used by the calculation unit 13b will be explained. do. As a specific example, the calculation unit 13b learns the degree of compatibility from videos, still images, etc., in which the appearance of the hair and the makeup applied are collected from websites, SNS, television, movies, etc. Compatibility is calculated using a machine learning model.

Furthermore, the calculation unit 13b calculates the degree of compatibility using a machine learning model that has learned the degree of compatibility for each attribute of the subject to whom makeup is applied. As a specific example, the calculation unit 13b calculates the degree of compatibility using a plurality of machine learning models generated for each user attribute including the target person's age, favorite entertainer, type of drama, and other user attributes. calculate.

In addition, the calculation unit 13b calculates the degree of compatibility using a machine learning model that has learned the degree of compatibility for each situation of the person to whom makeup is applied. As a specific example, the information providing device 10 uses a machine learning model that learns images of job-seeking candidates who passed document screening for each situation, such as new graduates, job changes, and mid-career hiring, to determine the degree of compatibility. Calculate

Note that the calculation unit 13b acquires the mode information about the hair mode selected by the user U from the mode information storage unit 12a. The calculation unit 13b also acquires makeup information indicating makeup from the makeup information storage unit 12b. On the other hand, the calculation unit 13b stores the calculated degree of compatibility in the degree of compatibility information storage unit 12c.

(2-3-3. Generation unit 13c)
The generation unit 13c generates provision information to be provided to the user U based on the calculated degree of compatibility. For example, the generation unit 13c identifies makeup having a degree of compatibility equal to or greater than a predetermined threshold among makeup indicated by the acquired makeup information, and generates provision information. Using a specific example, the generation unit 13c generates cosmetics M1, M2, M3, . , cosmetics M1, M2, and M3 whose degree of compatibility with the hairstyle H is "0.7" or higher, and provides information including images of the cosmetics M1, M2, and M3.

Further, the generation unit 13c identifies makeup based on the acquired user information of the user U, and generates provision information. For example, the generation unit 13c identifies makeup registered as a preference of the user U among makeup having a degree of compatibility equal to or greater than a predetermined threshold, and generates provision information including the makeup.

Note that the generation unit 13c acquires compatibility information from the compatibility information storage unit 12c. Further, the generating unit 13c stores the generated provided information in the provided information storage unit 12d.

(2-3-4. Transmitter 13d)
The transmitting unit 13d transmits to the user U the provided information generated by the generating unit 13c. For example, the transmission unit 13d transmits to the user terminal 20 of the user U an image including makeup whose degree of compatibility with the style of hair that the user U can provide is equal to or greater than a predetermined threshold. At this time, the transmission unit 13d may transmit information on cosmetics required for the makeup and makeup procedure together with the image including the makeup.

In addition, the transmission unit 13d acquires the provided information from the provided information storage unit 12d. Further, the transmission unit 13d may transmit the provided information to a server, business operator terminal, or database (not shown).

(2-3-5. Learning unit 13e)
The learning unit 13e learns a machine learning model so as to output the degree of compatibility of each makeup with the appearance of the hair when the style information and makeup selected by the user U are input. At this time, the learning unit 13e may learn the machine learning model by back propagation or the like.

For example, the learning unit 13e stores information indicating that the combination of the hair style and makeup that can be provided by the user U is good for each of the user U, the attribute of the user U, and the situation of the user U. When input to the learning model, the machine learning model is trained so that the degree of compatibility of makeup with the appearance of the hair is output as a numerical value approaching "1".

On the other hand, the learning unit 13e receives information indicating that the combination of the hair style and makeup that the user U can provide is incompatible for each of the user U, the attribute of the user U, and the situation of the user U. When input to the learning model, the machine learning model is trained so that the degree of compatibility of makeup with the appearance of the hair is output as a numerical value approaching "0."

[3. Specific example of information provision processing]
Next, a specific example of information provision processing according to the embodiment will be described. A specific example of the provision information transmitted to the user terminal 20 of the user U will be described below after explaining the learning process of the machine learning model and the provision information generation process.

(3-1. Learning process of machine learning model)
Learning processing of the machine learning model according to the embodiment will be described. By executing this process, the information providing device 10 can generate a learned machine learning model that is used in the process of generating provided information, which will be described later. Below, correct data collection processing, image data extraction processing, and image data learning processing will be described in this order.

(3-1-1. Correct data collection processing)
First, the information providing device 10 collects correct answer data from the SNS. Here, the correct data to be collected is, for example, an image in which hairstyle and makeup are photographed at the same time, and is an image that satisfies a predetermined evaluation condition. More specifically, the correct data to be collected includes images posted by a predetermined person, images with more than a predetermined number of likes, images with more than a predetermined number of comments and favorable content, and the like. . The information providing device 10 can also collect correct data from moving images such as television commercials and movies in the same way.

(3-1-2. Image data extraction processing)
Next, the information providing device 10 extracts the hair portion and the make-up portion (eg, face portion, limb portion) from the collected correct data image. At this time, the information providing apparatus 10 may extract makeup parts for each eyebrow, eye, nose, mouth, nail, and the like.

(3-1-3. Image data learning processing)
Then, when a combination of an image of a hair part and an image of each makeup part extracted from the same image (that is, a combination considered to be highly evaluated) is input, the information providing apparatus 10 provides information indicating that the combination is high. Train a machine learning model to output (for example, a score of 1). Further, when a combination of an image of a hair part and an image of a makeup part extracted from different images (that is, a combination of which the evaluation is not known to be good) is input, the information providing apparatus 10 provides information indicating that the compatibility is low (for example, , a score of 0, or a random number equal to or less than a predetermined value (0.7)). Through the processing described above, the information providing device 10 generates a machine learning model that has learned the above-described set matching.

(3-2. Provided information generation processing)
A process of generating provision information according to the embodiment will be described. By executing this process, the information providing apparatus 10 provides, for example, makeup posted by each stylist, and when the user U selects makeup, the reservation service of the makeup salon to which the selected stylist belongs (stylist's reservation service). reservation service) can be provided. In the following, the mode information acquisition process, the candidate information generation process, the mode identification process, and the provided information transmission process will be described in this order.

(3-2-1. Mode Information Acquisition Processing)
First, the information providing apparatus 10 is a server that provides information on makeup, for example. At this time, the information providing device 10 acquires an image of hair and makeup registered or selected by the user U. FIG. That is, the information providing apparatus 10 acquires an image of the hair style (mode information) that the user U can provide by the above-described processing.

(3-2-2. Candidate information generation process)
Next, the information providing apparatus 10 generates an image obtained by cutting out the hair portion from the obtained image, and generates candidate information by combining each make-up portion and the image cut out from the hair portion. At this time, the information providing apparatus 10 may receive makeup images (makeup information) from each user U, or may acquire images of makeup portions from the makeup information database 30 .

(3-2-3. Aspect identification processing)
Subsequently, the information providing apparatus 10 inputs each of the generated candidate information to the machine learning model, and identifies makeup and the like included in the candidate information whose outputted score exceeds a predetermined threshold. That is, through the above-described processing, the information providing device 10 identifies makeup that is highly compatible with the appearance of hair that the user U can provide.

(3-2-4. Provided information transmission processing)
Then, the information providing apparatus 10 generates provision information by adding a link to a cosmetics purchase site or the like to the specified image of makeup or the like, and transmits the generated one or a plurality of pieces of provision information to the user terminal 20. . As a result, the information providing device 10 can provide the user U with information such as "Here is makeup that goes well with this hairstyle. Would you like to purchase cosmetics?" The information providing apparatus 10 can also provide the above information to the user U via the server of the hair and makeup reservation site where the user U provides information on beauty salons and the like. At this time, the information providing device 10 can also transmit the provided information directly to the user terminal 20 of the user U. FIG.

(3-3. Specific examples of provided information)
A specific example of the provided information according to the embodiment will be described with reference to FIG. FIG. 7 is a diagram illustrating a specific example of information provision processing according to the embodiment. A specific example of the provided information that is transmitted to and displayed on the user terminal 20 of the user U will be described below.

(3-3-1. Display screen W1)
First, a display screen W1 relating to hair styles such as hairstyles, hair colors, etc. that can be provided by the user U as information displayed on the user terminal 20 of the user U will be described. On the display screen W1, an image H1 of a hairstyle H, which is a hair style uploaded by the user U to the hair/makeup reservation site, and a screen H2 including the treatment fee, treatment details, etc. of the hair style are displayed.

(3-3-2. Display screen W2)
In addition, the display screen W2 including information on makeup for the hairstyle, hair color, etc. that can be provided by the user U as information displayed on the user terminal 20 of the user U will be described. On the display screen W2, screens M1, M2, and M3 are displayed as information on cosmetics X, Y, and Z as "cosmetics that match this hair." At this time, the user U can also browse the details of each cosmetic and the image of the makeup using the cosmetic by clicking the screens M1, M2, and M3, which are the information of the cosmetics X, Y, and Z. In addition, the user U can also transition to the purchase site of each cosmetic by clicking screens M1, M2, and M3, which are information on cosmetics X, Y, and Z. FIG.

[4. Information provision processing flow]
The information processing procedure of the information providing apparatus 10 according to the embodiment will be described with reference to FIG. FIG. 8 is a flowchart illustrating an example of the flow of information provision processing according to the embodiment. Note that steps S101 to S106 below can also be performed in a different order. Also, some of steps S101 to S106 below may be omitted.

(4-1. Mode information collection processing)
First, the acquisition unit 13a of the information providing device 10 executes mode information acquisition processing (step S101). For example, the acquisition unit 13a acquires from the user terminal 20 the mode information about the hair mode that the user U can provide.

(4-2. Makeup information collection processing)
Second, the acquisition unit 13a of the information providing device 10 executes makeup information acquisition processing (step S102). For example, the acquisition unit 13a refers to the makeup information database 30 to acquire makeup information about makeup.

(4-3. Compatibility degree calculation processing)
Thirdly, the calculation unit 13b of the information providing device 10 executes compatibility degree calculation processing (step S103). For example, the calculation unit 13b calculates the degree of compatibility of each makeup with the hair style that the user U can provide from the acquired style information and makeup information.

(4-4. Provided information generation processing)
Fourthly, the generating unit 13c of the information providing device 10 executes the provided information generating process (step S104). For example, the generation unit 13c generates provision information to be provided to the user U based on the calculated degree of compatibility and user information.

(4-5. Provided information transmission processing)
Fifthly, the transmission unit 13d of the information providing device 10 executes the provided information transmission process (step S105). For example, the transmission unit 13d transmits the generated provided information to the user terminal 20 of the user U.

(4-6. Compatibility learning process)
Sixthly, the learning unit 13e of the information providing device 10 executes compatibility degree learning processing (step S106). For example, the learning unit 13e learns the degree of compatibility of each makeup with the calculated hair style that the user U can provide.

[5. Effect of Embodiment]
Finally, effects of the embodiment will be described. Effects 1 to 7 corresponding to the processing according to the embodiment will be described below.

(5-1. Effect 1)
In the processing according to the above-described embodiment, mode information regarding the mode of hair that can be provided by the user U is acquired, and based on the acquired product information, the degree of compatibility indicating the degree of compatibility between the mode of hair and makeup is obtained. Calculate Therefore, useful information can be provided to the user U in this process.

(5-2. Effect 2)
In the processing according to the above-described embodiment, the degree of compatibility is calculated using a machine learning model that has learned the degree of compatibility between the combination of the hair style and the makeup applied to the subject having the hair of that style. Therefore, in this process, useful information can be provided to the user U more effectively.

(5-3. Effect 3)
In the processing according to the above-described embodiment, the degree of compatibility is calculated using a machine learning model that has learned the degree of compatibility from moving images including hair styles and makeup. Therefore, in this process, useful information can be provided to the user U more effectively and with high accuracy.

(5-4. Effect 4)
In the processing according to the above-described embodiment, the degree of compatibility is calculated using a machine learning model that has learned the degree of compatibility for each attribute of the subject to whom makeup is applied. Therefore, in this process, useful information can be provided to the user U with high accuracy for each attribute of the target person.

(5-5. Effect 5)
In the processing according to the above-described embodiment, the degree of compatibility is calculated using a machine learning model that has learned the degree of compatibility for each situation of the person to whom makeup is applied. Therefore, in this process, useful information for the user U can be provided with high accuracy for each target person's situation.

(5-6. Effect 6)
In the above-described processing according to the present embodiment, makeup information related to makeup is further acquired, makeup having a degree of compatibility equal to or higher than a predetermined threshold among the makeup indicated by the makeup information is specified, and provision information is generated. Therefore, in this process, by using the makeup information, useful information can be provided to the user U more effectively.

(5-7. Effect 7)
In the processing according to the present embodiment described above, user information regarding the user U who provides the hair style is further acquired, makeup is specified based on the user information, and provision information is generated. Therefore, in this process, by using the user information, useful information can be provided to the user U more effectively.

[Hardware configuration]
Also, the information providing apparatus 10 according to the above-described embodiment is implemented by a computer 1000 configured as shown in FIG. 9, for example. The information providing device 10 will be described below as an example. FIG. 9 is a diagram illustrating an example of a hardware configuration; The computer 1000 is connected to an output device 1010 and an input device 1020, and an arithmetic device 1030, a primary storage device 1040, a secondary storage device 1050, an output I/F (Interface) 1060, an input I/F 1070, and a network I/F 1080 are buses. It has a form connected by 1090.

Arithmetic device 1030 operates based on programs stored in primary storage device 1040 and secondary storage device 1050, programs read from input device 1020, and the like, and executes various types of processing. The arithmetic unit 1030 is implemented by, for example, a CPU (Central Processing Unit), MPU (Micro Processing Unit), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), or the like.

The primary storage device 1040 is a memory device such as a RAM (Random Access Memory) that temporarily stores data used for various calculations by the arithmetic device 1030 . The secondary storage device 1050 is a storage device in which data used for various calculations by the arithmetic device 1030 and various databases are registered. State Drive), flash memory, or the like. The secondary storage device 1050 may be an internal storage or an external storage. Also, the secondary storage device 1050 may be a removable storage medium such as a USB memory or an SD (Secure Digital) memory card. Also, the secondary storage device 1050 may be a cloud storage (online storage), a NAS (Network Attached Storage), a file server, or the like.

The output I/F 1060 is an interface for transmitting information to be output to the output device 1010 that outputs various information such as a display, a projector, and a printer. (Digital Visual Interface), HDMI (registered trademark) (High Definition Multimedia Interface), and other standardized connectors. Also, the input I/F 1070 is an interface for receiving information from various input devices 1020 such as a mouse, keyboard, keypad, buttons, scanner, etc., and is realized by, for example, USB.

Also, the output I/F 1060 and the input I/F 1070 may be wirelessly connected to the output device 1010 and the input device 1020, respectively. That is, the output device 1010 and the input device 1020 may be wireless devices.

Also, the output device 1010 and the input device 1020 may be integrated like a touch panel. In this case, the output I/F 1060 and the input I/F 1070 may also be integrated as an input/output I/F.

Note that the input device 1020 includes, for example, optical recording media such as CDs (Compact Discs), DVDs (Digital Versatile Discs), PDs (Phase change rewritable discs), magneto-optical recording media such as MOs (Magneto-Optical discs), and tapes. It may be a device that reads information from a medium, a magnetic recording medium, a semiconductor memory, or the like.

Network I/F 1080 receives data from other devices via network N and sends the data to arithmetic device 1030, and also transmits data generated by arithmetic device 1030 via network N to other devices.

Arithmetic device 1030 controls output device 1010 and input device 1020 via output I/F 1060 and input I/F 1070 . For example, arithmetic device 1030 loads a program from input device 1020 or secondary storage device 1050 onto primary storage device 1040 and executes the loaded program.

For example, when the computer 1000 functions as the information providing device 10 , the arithmetic device 1030 of the computer 1000 implements the functions of the control unit 130 by executing a program loaded on the primary storage device 1040 . Further, arithmetic device 1030 of computer 1000 may load a program acquired from another device via network I/F 1080 onto primary storage device 1040 and execute the loaded program. Further, the arithmetic unit 1030 of the computer 1000 may cooperate with another device via the network I/F 1080, and call functions, data, etc. of the program from another program of the other device for use.

〔others〕
Although the embodiments of the present application have been described above, the present invention is not limited by the contents of these embodiments. In addition, the components described above include those that can be easily assumed by those skilled in the art, those that are substantially the same, and those within the so-called equivalent range. Furthermore, the components described above can be combined as appropriate. Furthermore, various omissions, replacements, or modifications of components can be made without departing from the gist of the above-described embodiments.

Further, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed manually. All or part of this can also be done automatically by known methods. In addition, information including processing procedures, specific names, various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

Also, each component of each device illustrated is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to the illustrated one, and all or part of them can be functionally or physically distributed and integrated in arbitrary units according to various loads and usage conditions. Can be integrated and configured.

For example, the information providing apparatus 10 described above may be implemented by a plurality of server computers, and depending on the function, may be implemented by calling an external platform or the like using an API (Application Programming Interface), network computing, or the like. can be changed flexibly.

Also, the above-described embodiments and modifications can be appropriately combined within a range that does not contradict the processing content.

Also, the above-mentioned "section, module, unit" can be read as "means" or "circuit". For example, the control unit can be read as control means or a control circuit.

10 information providing device 11 communication unit 12 storage unit 12a mode information storage unit 12b makeup information storage unit 12c compatibility information storage unit 12d provision information storage unit 13 control unit 13a acquisition unit 13b calculation unit 13c generation unit 13d transmission unit 13e learning unit 20 User terminal 30 Makeup information database 100 Information providing system

Claims (9)

an acquisition unit that acquires mode information about the mode of hair that can be provided by a user;
a calculation unit that calculates a degree of compatibility indicating a degree of compatibility of a combination of the aspect and makeup based on the aspect information;
a generation unit that generates provision information to be provided to the user based on the degree of compatibility;
An information providing device comprising: The calculation unit calculates the degree of compatibility using a machine learning model that has learned the degree of compatibility between the mode of hair and the makeup applied to the subject having the hair of the mode.
2. An information providing apparatus according to claim 1, characterized by: The calculation unit calculates the degree of compatibility using the machine learning model that has learned the degree of compatibility from moving images including hair styles and makeup.
3. The information providing apparatus according to claim 2, characterized by: The calculation unit calculates the degree of compatibility using the machine learning model that has learned the degree of compatibility for each attribute of the subject to whom makeup is applied.
4. The information providing device according to claim 2 or 3, characterized in that: The calculation unit calculates the degree of compatibility using the machine learning model that has learned the degree of compatibility for each situation of the subject to whom makeup is applied.
5. The information providing device according to any one of claims 2 to 4, characterized by: The acquisition unit further acquires makeup information related to makeup applied to the subject,
The generation unit identifies the makeup having a degree of compatibility equal to or higher than a predetermined threshold among the makeup indicated by the makeup information, and generates the provision information.
6. The information providing device according to any one of claims 1 to 5, characterized by: The acquisition unit further acquires user information about the user who provides the aspect,
The generating unit identifies the makeup based on the user information and generates the provided information.
7. The information providing device according to any one of claims 1 to 6, characterized by: An information providing method executed by an information providing device,
an acquisition step of acquiring mode information about the mode of hair that can be provided by the user;
a calculation step of calculating a degree of compatibility indicating the degree of compatibility of a combination of the aspect and makeup based on the aspect information;
a generating step of generating provided information to be provided to the user based on the degree of compatibility;
An information provision method characterized by comprising: an acquisition procedure for acquiring mode information about the mode of hair that can be provided by the user;
a calculation procedure for calculating a degree of compatibility indicating a degree of compatibility of a combination of the aspect and makeup based on the aspect information;
a generation procedure for generating provision information to be provided to the user based on the degree of compatibility;
An information providing program characterized by causing a computer to execute

Images